The max-age=3600 is added by default if HttpResult.MaxAge wasn’t specified, what’s is the server code you’re using to set the caching headers?
Can you try removing the plugin to see if that restores the existing functionality:
this.Plugins.RemoveAll(x => x is HttpCacheFeature);
Although HttpCacheFeature shouldn’t have any effect unless you’re returning a HttpResult which you wouldn’t be doing inside a Filter which I believe from earlier comments is what you’re using. Can you post an example of the server code where the behavior has changed and I’ll see if I can spot what might have affected it.
RE: adding ‘max-age=3600’ If that is the result of the HttpCacheFeature being ON by default.
(I am verifying this now… will take a few minutes for CI to report)
IMHO I think it is a grave mistake (in general) for services to ‘advise’ HTTP clients (i.e. HttpCacheFeature ON by default) to cache all responses. It will likely cause unintended side-effects in all SS services for people’s existing implementations, and they wont know why this is suddenly happening to them.
For example, if their service is HTTPS only, then the default on the internet is not to cache any responses, and they wont be expecting clients and intermediaries to save their responses. This feature breaks that rule (because it is ON by default).
Second example, if my service is consumed by a web browser app, then the responses will be cached for a minute in the browser, which may interfere with freshness of the data in the app.
If the goal of the implementer of HTTP caching advice (implementer of the SS service) is to keep its cached data (from a SS service) fresh, then HTTP expiration (max-age) cannot stand alone, and requires a HTTP Validation strategy as well. HTTP caching ‘advise’ form the server needs to be an intentional and planned strategy.
RE: max-age=3600
Early indications from our CI, are that by removing HttpCachingFeature (this.Plugins.RemoveAll(x => x is HttpCacheFeature)) does NOT fix the max-age=3600 header being set in all responses, and overriding our own ‘Cache-Control’ header being set in response.
We do this:
In a ResponseFilterAttribute of our own, (set on each service operation):
public static void AddCachingExpirationHeaders(this IResponse response, TimeSpan timeToLive, string etag)
{
response.AddHeader(HttpHeaders.CacheControl, "no-cache, max-age={0}".Fmt(timeToLive.TotalSeconds));
response.AddHeader(HttpHeaders2.Expires, DateTime.UtcNow.ToNearestSecond() .AddSeconds(timetoLive.TotalSeconds).ToString("r"));
response.AddHeader(HttpHeaders2.Pragma, string.Empty);
if (etag.HasValue())
{
response.AddHeader(HttpHeaders.ETag, etag);
}
}
That sounds very strange, can you show mean example of what your Service returns? e.g. is it a vanilla Response DTO, HttpResult, ToOptimizedResult or something else?
Ammendment.
For service operations where we are not setting any HTTP caching headers (i,e. Cache-Control, Expires, Pragma etc) we are NOT getting max-age=3600 returned by the service by default. In fact, we don’t get any HTTP caching headers set, which is entirely what we expect.
It is only, when we set the HTTP caching headers (i,e. Cache-Control, Expires, Pragma etc as in above code example), ourselves that the result in the response is ‘max-age=3600’.
BTW: this is with the HttpCacheFeature turned OFF (this.Plugins.RemoveAll(x => x is HttpCacheFeature))
Can you please provide an example of the Service response which exhibits this behavior? i.e are you returning a raw response DTO?
In all cached responses we return object, that is either returned by ToOptimizedResultWithCache() from the cache (if cache hit), or generated by our service as a vanilla DTO (if cache miss).
This is the code we use to do it in each service operation (abridged):
[CacheResponse(CacheExpirationDuration.Medium)]
[HttpClientShouldCacheResponse(CacheExpirationDuration.VeryShort)]
public object Get(GetAResource request)
{
return ProcessCachedRequest(request, HttpStatusCode.OK, () =>
{
return new GetResourceResponse
{
Resource = new TestResource
{
Id = request.Id,
}
};
});
}
...
//where ProcessCachedRequest ultimately directly calls ToOptimizedresultFromCache(), and sets corresponding response headers, and status code.
It is all vanilla ServiceStack stuff, by the book, no HTTPResult, no custom return types, nothing like that. The ResponseFilters we have just set the HTTP headers.
RE: 2. the exception ServiceStack.WebServiceException: 400 - Bad Request: Bad Request: An item with the same key has already been added
Removing the HttpCacheFeature (this.Plugins.RemoveAll(x => x is HttpCacheFeature)) HAS worked, and gotten rid of the exception from our codebase.
And ideas what would have caused that in our code?
I’m having a hard time trying to work out how this can be possible given the default MaxAge is on the HttpCacheFeature and if the plugin is removed I don’t see how it could have access to it to even attempt to add it. I’ve modified the existing tests to remove the feature to confirm that it’s not being added in ToOptimizedResults.
Just to be perfectly clear you’re removing the plugin from AppHost.Configure() right? i.e.
public override void Configure(Container container)
{
Plugins.RemoveAll(x => x is HttpCacheFeature);
}
Ok just saw this now, removing the plugin should remove the feature - I was going mental trying to find out how it could still be adding the CacheControl headers.
No idea from here, do you have a full stack trace?
My bad Mythz, I am sorry, my fark up.
The AppHost we were using for integration testing (derived from AppSelfHostBase) is not the same one we made the change on which is the base class all our services inherit from (derived from AppHostBase).
When I add the Plugins.RemoveAll(x => x is HttpCacheFeature); to the testing AppSelfHostBase instance, it removes the max-age=3600 problem.
Sorry to fark you about on that one.
OK, so that confirms it: the new HttpCacheFeature being ON by default, is confirmed to cause both the issues we reported. [and turning the HttpCacheFeature OFF fixes both issues)
So, since we roll our own code for doing both service-side caching and client side-caching, and we don’t want to use the new HttpCacheFeature (at the moment, perhaps we will go there later when it matures).
Are you still planning to leave HttpCacheFeature ON by default in v55?
(Which means will have to remove that feature by default for our services to use v55?)
No stack trace yet.
We are adding our headers using the IResponse.AddHeader() method. Which has not been a problem until the HttpCacheFeature entered the picture.
It is possible that, with the HttpCacheFeature turned on, calling IResponse.AddHeader(Headers.CacheControl, “no-cache, max-age=300”); from our ResponseFilterAttribute might throw the dictionary exception?
Yes, it’s what implements the new Cache attributes on HttpResult and is supported by the new CacheClients so it needs to be enabled by default. But making it a plugin means it can be easily removed/substituted.
Yeah adding duplicate headers would cause that, I’ve added an extra check in ToOptimizedResults so it doesn’t add LastModified/CacheControl headers if there’s an existing Cache-Control header in this commit. This should support your use-case where you’re adding Caching headers manually.
I could change AddHeader so it overwrites existing headers so it doesn’t throw but that would mask that there are conflicting headers being added so I’ll leave it as-is.
OK thanks for the confirmation, and fix.
I still think that if the HttpCacheFeature is ON by default, and if it always sets the Cache-Control: max-age=3600 by default, that you are introducing potential issues with existing services that upgrade to v55, from implmenters who are not aware that their responses are now saying “cache me for 60secs” (but who have no HTTP validation strategy in place using ETag or LastModified.). That is a lot of work to add for most people to setup I suspect.
It does however force implementers to start to explore HTTP caching, which is good, but not easy to get right, or even functional (a lot of confusion and poor guidance out there on RFC2616. RFC7234 is better). Not many implementers understand it very well, and learning it well enough is a steep learning curve. So having a guidance page on wiki on how to think about doing it is essential here to support them.
Happy to help with that, or provide a blog post on how we are doing it (in principle) and (in practice, since we tailor for a more elaborate declarative approach) with servicestack, if you wanted to reference that from the wiki?
Yeah that’s a good point ideally the default behavior should be transparent so I’ve changed the default Cache-Control for ToOptimizedResults to max-age=0 so clients should revalidate each time unless error, adds a round-trip but still saves on serialization + bandwidth. (Change is now available on MyGet).
Yeah if you can create a blog post we’ll be able to reference/include it from the wiki where we’ll document the new built-in HttpCachingFeature.
One last change which might affect you if you were previously using NotModifiedFilter is I’ve now renamed it to ExceptionFilter and changed it to handle any exception. This will also allow Cache Clients to return a cached Response for any Exception in this commit.
Which is what the existing cache clients will do for any NoCache/(MustRevalidate+Expired) response. This change is now on MyGet.
Hey Mythz,
Blog post, on how we achieved comprehensive client and service caching, which I hope will help give some guidance to others the SS community: http://www.mindkin.co.nz/blog/2016/1/5/caching-anyone