JsonHttpClient + Azure multi-instance = fail

Clients
1

We have an application that posts JSON data from one service to another using SS’s JsonHttpClients. Each service has N instances, and we’ve found that the traffic winds up overwhelming high on a small amount of instances since it appears to be pooling the connections.

From what I see online, this is built on .NET’s HttpClient, so we should (in theory) be able to do this as a fix:

ServicePointManager.FindServicePoint(new Uri(baseUri)).ConnectionLeaseTimeout = 0;

This should make every connection fresh, letting the Azure traffic controller round-robin appropriately. This did not work at all - still the majority of the traffic on 1-2 instances of 8.

As another test, I did change one call to instantiate an HTTPClient directly with settings I wanted (abandoning the JsonHttpClient entirely), and do a post that way, and it worked - the load balanced perfectly distributed.

Are there other properties I can use to modify this behavior or am I stuck with implementing my own concept for clients?

Thanks!

2

I believe the ServicePointManager class only impacts HttpWebRequest based clients like JsonServiceClient, not HttpClient-based clients like JsonHttpClient.

3

These are 2 examples using ServicePointManager to impact this behavior with HttpClient. YMMV - they are just random internet posts.

Of course, as I said, this property doesn’t seem to work for JsonHttpClient regardless…

4

The 2nd comment from the first link suggests HttpClient is not using the ServicePointManager.

5

Ok, so let’s assume HttpClient doesn’t use ServicePointManager at all. Can you help with the original question - that is how can I control (or disable) connection pooling for JsonHttpClient?

Thanks.

6

There’s no customizations available in JsonHttpClient itself, any customizations would need to be made to the HttpClient instance it uses to make requests, which you can access from:

var httpClient = client.GetHttpClient();

I don’t see any options on HttpClient for disabling connection pooling itself, but if the issue is due to reusing the same HttpClient instance then you can just create a new JsonHttpClient instance on each usage which will use a new HttpClient instance.

Otherwise you can modify the HttpClientHandler the JsonHttpClient uses with::

JsonHttpClient.GlobalHttpMessageHandlerFactory = () => new new HttpClientHandler
{
    UseCookies = true,
    CookieContainer = client.CookieContainer,
    UseDefaultCredentials = client.Credentials == null,
    Credentials = client.Credentials,
    AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate,
};

You can also override the entire HttpClient instance with:

client.HttpClient = CreateCustomHttpClient();
7

Bringing back an old message, but I did override the entire HttpClient instance as recommended.

  Client = new JsonHttpClient(uri);
        Client.HttpClient = new CustomHttpClient();

public class CustomHttpClient : HttpClient
{
public CustomHttpClient() : base()
{
DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue(“application/json”));
Timeout = new System.TimeSpan(0, 0, 20);
}
}

This still wound up pushing the majority of traffic to a small amount of instances.

As I mentioned originally, if I remove the JsonHttpClient completely and just create an HttpClient and do a raw json data post to a url through that, everything balances perfectly.

I am also creating a new JsonHttpClient (and a new HttpClient) per request as well - no impact.

It still seems as though something is different between using JsonHttpClient (even with a custom HttpClient implementatoin) and just using HttpClient. Any other suggestions? I can put it a support ticket if that would be a better avenue for exploration of this.

Thanks

8

You can check the source code for the JsonHttpClient to see if there’s something different to what you’re doing, but it’s just using the normal HttpClient APIs as normal. One thing we could try is changing it to use async/await instead of manual Task continuations, but other than that I don’t see what would be different.

Can you try changing your manual client to use .ContinueWith() instead of async/await and let me know if that makes a difference in behavior?