JsonServiceClient HTTP Client Cache

ServiceStack Caching
1

Hey Demis,

Another design question/challenge.

We are designing a HTTP Client Cache component that can be used by a JsonServiceClient that implements HTTP Expiration (with ETag) and HTTP Validation (Cache-Control: no-cache). The idea is that we can just wire in this client cache to a JsonServiceClient, and it would honor HTTP caching headers, just like most browsers do. It is critical for enhancing the performance of our services. Also, we were thinking of contributing this client to SS community for general use, as it is missing from the framework right now for anyone who wants to do the same.

We are having some challenges finding a way to hook this client cache into a JsonServiceClient.

Now, we are aware of the ResultsFilter and ResultsFilterResponse that are described on the wiki page: https://github.com/ServiceStack/ServiceStack/wiki/C%23-client#custom-client-caching-strategy, and the sample gives a good sense of how to go about hooking into the client.
However, in practice, we are finding that this hook is just slightly insufficient when it comes to implementing HTTP Validation (using ETags).

Here is the scenario:

We have a client cache and it is wired into the ResultsFilter and the ResultsFilterResponse of a JSC instance, in the same way as the wiki page demonstrates.

When a GET request by the JSC is made, the ResultsFilter hook fires, and our client cache either responds with null (nothing found in the cache yet) or it responds with the cached response (that it had previously saved with some expiry period). Implementing HTTP Expiration is a cinch, but there are times when HTTP Validation requires us to make a quick validation call to the [same] service with the the same request, and include the If-None-Match header with the the ETag.

As it goes with HTTP Validation, the server can either respond (by throwing in a MS client) a 304 - Not Modified, or return a 2XX fresh response with a new ETag and new Cache-Control headers. The client cache is then supposed to update its expiration and ETag values, and (if 2XX then cache the new response), and then serve its cached version of the response.

So, the issue is that in order for the client cache to make the addtional validation call with the ‘If-None-Match: ETag’ it needs a another instance of the JSC to do it. Since the current JSC is blocked in the process of handling a ResultsFilter hook from the previous GET.

Now I haven’t tested this yet in a running environment, but I suspect that we can’t issue a second 'If-None-Matched validation call from the same JSC that is calling us in the ResultsFilter hook? That would be nice, but I doubt it’s accomadated?

If not, we will need another instance of a JSC to do that for us, and in practice, for that to work, we will need the second JSC instance to include the original request object, to the original request URI and with all the original headers of the original request that was hooked by the ResultsFilter hook. Those original request headers are not available to us in the hook [at present].

I am certain that there is a sound reason why the headers are not in the call from ResultsFilter hook, but without those request headers we cannot complete the ‘Íf-None-Match’ validation call. Since to make the ‘If-None-Match’ validation call, we must have the same request headers and request object and same request URI as the hooked call. Because for us, those original headers included things like oAuth Authorization headers and others that we have setup in the RequestFilter of our JSC for each call.

From cursory testing, it also seems that the RequestFilter of our JSC is not processed until after the ResultsFilter hook returns, so we cannot get the current headers in our hooked call either.

Are you aware of this problem? Any ideas on a reasonable workaround? (Perhaps another hook, we can use where the headers are all set in the request by the RequestFilter)? Can you think of anything we can do here? Ideally, we would not want to create another instance of the JSC, but we certainly can and will do at a pinch.

Regards

2

Wasn’t aware of an issue with the hooks that exist now, but the hooks can’t be executed after request filters since they’re executed in SendRequest() which needs to return a populated and executed WebRequest whereas Results Filter returns the Response DTO.

There shouldn’t be an issue with re-using the instance to make another request as each request uses a new HttpWebRequest and doesn’t maintain interim state about the request on the instance. One thing you have to watch out for is the ResultsFilter/RequestFilter will get called again if you use the public API’s to make the request (i.e. instead of SendRequest()). But there’s also no harm in using a 2nd instance since a new HttpWebRequest which behind the scenes is using connection pooling, so the performance would be the same. You can configure the 2nd instance to not have ResultsFilter/RequestsFilter so it might be preferred to use instead, note you’ll need to share the cookie container if you want to maintain the same session in the new instance.

Since there’s little opportunity to change the ResultsFilter hooks, if it doesn’t do what you need you may need to create a new higher level Service Client that uses JsonServiceClient under the hood. This would be my preferred approach rather than trying to introduce and manage cache state inside the base service client implementation.

3

Are you suggesting I do something like this?

public class MyJsonServiceClient : ServiceStack.JsonServiceClient
{
        public MyJsonServiceClient(string baseUrl)
            : base(baseUrl)
        {
            RequestFilter = ExecuteAllRequestFilters;
            ResultsFilter = FetchResponseFromClientCache;
            ResultsFilterResponse = CacheResponseInClientCache;
        }

        public IServiceClientCache ClientCache { get; set; }

        private void ExecuteAllRequestFilters(HttpWebRequest request)
        {
            // Add any headers/cookies we need to the request etc...
        }

        private void CacheResponseInClientCache(WebResponse webResponse, object response, string httpMethod,
            string requestUri, object request)
        {
            if (ClientCache == null)
            {
                return;
            }

            ClientCache.CacheResponse(webResponse, response, request, httpMethod, requestUri);
        }

        private object FetchResponseFromClientCache(Type responseType, string httpMethod, string requestUri, object request)
        {
            if (ClientCache == null)
            {
                return null;
            }

            // Used by IServiceClientCache to make additional 'If-None-Match' validation request, using same requestDto and same requestUri as the current call, plus any headers and cookies that would be added by the RequestFilter of the current instance of MyJsonServiceClient
            var validationClient = new MyJsonServiceClient(BaseUri)
            {
                RequestFilter = RequestFilter,
                CookieContainer = CookieContainer,
                ResultsFilter = null,
                ResultsFilterResponse = null,
            };
            Headers.ToDictionary().ForEach((name, value) =>
            {
                validationClient.Headers.Add(name, value);
            });

            return ClientCache.GetCachedResponse(request, httpMethod, requestUri, validationClient);
        }

}
4

Found another issue with the ResultsFilter mechanism:

Take a look at: https://github.com/ServiceStack/ServiceStack/blob/363ea397c3fee3bbbdc23354510f23a9eb4a8a75/src/ServiceStack.Client/ServiceClientBase.cs#L560

In the delegate for ResultsFilterResponse, we are being passing in the actual WebResponse, as well as the response DTO, but as it turns out at runtime the WebResponse has already been disposed of, so accessing its headers is now impossible.

Looks like HandleResponse https://github.com/ServiceStack/ServiceStack/blob/363ea397c3fee3bbbdc23354510f23a9eb4a8a75/src/ServiceStack.Client/ServiceClientBase.cs#L1613 is disposing of the actual WebResponse before we can use it in ResultsFilterResponse, nullifying its purpose or use.

Without those response headers, we cannot tell how to cache the response in the client. In HTTP Expiration an HTTP Validation our client cache behaves differently based on the headers (ETag, Cache-Control, Expires etc) that come back in every response.

5

I wasn’t thinking of using inheritance, my initial approach would be to create a high-level wrapper service client that made use of an internal service client instance, but if you can make it work using inheritance that works too.

ok thx, I’ve split HandleResponse into multiple parts so ResultsFilterResponse is called before the webResponse is disposed in this commit.

This change is available from v4.0.55 that’s now available on MyGet.

6

Thanks Mythz, that is fabulous!
now trying out the 55 pre-release.

RE inheritance or wrapping:

In either solution, the second JSC instance (the one that does the 304 validation call) will need exactly the same request context (headers, cookies, etc) that the first instance would have. (Just like the first instance would have if it was to make the request after the call to ResultsFilter=null).

Can I confirm with you that I would have it covered here if we do this:

        var firstInstance = new JsonServiceClient("http://someurl");
        ... other code that could use the Headers, Cookies, and RequestFilter


        var validationClient = new JsonServiceClient(firstInstance.BaseUri)
        {
            RequestFilter = firstInstance.RequestFilter,
            CookieContainer = firstInstance.CookieContainer,
            ResultsFilter = null, // so we dont loop forever
            ResultsFilterResponse = null, // so we dont loop forever
        };
        firstInstance.Headers.ToDictionary().ForEach((name, value) =>
        {
            validationClient.Headers.Add(name, value);
        });
7

It’s not a complete clone but it does have a copy of headers and shared CookieContainer.

Maybe I haven’t thought about this enough but I was only going to use 1 client instance internally, which just adds the If-None-Match header for any url it has the cached results for then if the client throws a 304 Exception I’d just return the cached result I have. Inside the ResultsFilterResponse is where I’d be populating the cache for any responses which contains an ETag.

8

If you are working on client side caching for JSC, can I offer you a class (+tests) that does what it should do with 304’s, ETags, CacheControl and Expires? Would save you some time, I suspect. The only remaining issue that I am wrangling with, is how to make the validation request.

9

Sure would be happy to review a PR.

Is the reason that you’re trying to support all caching headers because you need to use all of them or just because you want to make a general purpose cache-aware client?

10

Good challenge.

For our JSC clients (to our SS services) we need ‘If-None-Match’ with ‘ETag’ for HTTP validation and ‘Cache-Control: max-age’ for HTTP Expiration, but we are choosing to support ‘Expires’ as well for HTTP Expiration at our services for other clients. Strictly speaking, our JSC client cache only needs to deal with Etag and Cache-Control, but was designing it for a contribution to more general audience, who may be using it with their services which may be using ‘Expires’.
We have not bothered yet with implementing ‘Last-Modified’ for HTTP Validation at this stage, because our services provide ETag only.

Happy to submit a PR for you, but this IServiceClientCache will not be integrated with anything yet, and its designed to integrate with the JSC ResultsFilter pattern at this stage.

Still useful? at this stage? or do you want me to finish proving it working seemlessly with pre-release 55 first?

Or perhaps you just want to see the IServiceClientCache as standalone? Let me know, I would want to help if you are working in this area.

11

That’s ok if it’s integrated with other classes it wont be a useful general purpose test case we could test against a custom cache-enabled IServiceClient implementation. If I get time I may look at adding better caching support/hooks in client/server, but don’t have the time atm.

12

OK, did you want me to share the code (at this stage) somehow?

13

No, that’s ok. Thought you had a decoupled test suite that would verify a correct cached IServiceClient implementation.

14

Hey Mythz,

A related how to question:

In the ResultsFilterDelegate, we get the: request, httpMethod, and requestUri.

Given a JSC, how would I make a new call using just those variables?
I thought I would do this: JSC.Get<WebResponse>(request); but it does not feel right, since how would the JSC know which URL to send to?

How should I be doing it?

15

Call it with the full requestUri?

using (var webRes = client.Get<HttpWebResponse>(requestUri))
{
   //...
}
16

haha! can you believe all this time, I didn’t know about that overload of Get()! thanks

1 Like
17

Hey Mythz,

Struggling with last piece of this client caching puzzel with ResultsFilter.

When we are called with ResultsFilterDelegate and asked the question “Do you have the response cached”, AND we go through HTTP validation (ETag+If-None-Match) and DONT receive a 304 - Not Modified, then we cache that new response, and return it to the ResultsFilter.

According to line 551 (ServiceClientBase): https://github.com/ServiceStack/ServiceStack/blob/e392265456e12077087c1cd014c918913f409bc9/src/ServiceStack.Client/ServiceClientBase.cs#L551
We would have to return a response that is of type TResponse, otherwise the JSC continues to makes the request (on line 555) regardless, even though we have cached it!. Which defeats the purpose of client side caching.

So, the challenge is this: when we make the HTTP Validation call (ETag+If-None-Match with our other instance of the JSC) we call validationClient.get(requestUri), becuase we need to interrogate the response headers.But we have to return the TResponse from our delegate instead.

How would I safely convert a HttpWebResponse response into a TResponse response?
(Bearing in mind that the HttpWebResponse could be a DTO or Stream perhaps)

1 Like
18

You can’t unless the client called the generic method with a HttpWebResponse generic parameter, the ResultsFilter is meant to return the generic response the user is expecting or it can’t work. The idea is to return the cached response from within the ResultsFilter as we show in the client caching strategy example. If you can’t do that you wont be able to make use of the delegate.

19

So I think we are saying that: if the server responds with a 200 to a ETag+If-None-Match call (which means the ETag has changed), then to return null to the ResultsFilter? and then client will make another regular request to get the new response, and call ResultsFiilterResponse to store that response and new ETag.

(that will work but it is not so efficient when the If-None-Match fails validation)

20

So this begs the more general question:
Is there a way in SS to make a client.Get<??>(requestUri) that returns the typed response and gives us access to the response headers? (since client.Get<HttpWebResponse>(requestUri) cant provide us with both?