I’m using the JsonServiceClient as a static variable in a proof of concept for a service gateway and I was wondering if it is thread-safe?
The gateway factory is fairly straightforward
public class GatewayFactory : ServiceGatewayFactoryBase
{
private static JsonServiceClient client = new JsonServiceClient("http://localhost:50834/"); // for testing purposes this server remote does a Task.Delay to simulate a load.
public override IServiceGateway GetServiceGateway(IRequest request)
{
if (request.AbsoluteUri.Contains("marketing"))
return client;
return base.GetServiceGateway(request);
}
public override IServiceGateway GetGateway(Type requestType)
{
return base.localGateway;
}
}
It is registered like to in AppHost’s Configure method:
container.Register<IServiceGatewayFactory>(x => new GatewayFactory());
The service uses async method and is defined like this:
public class CampaignGatewayService : Service
{
public Task<Campaign> Any(GetCampaignRequest request)
{
return Gateway.SendAsync(request);
}
}
While load testing the service gateway I found that it sometimes stopped responding when using JsonServiceClient but if I use JsonHttpClient the issue is not present. I found an old post (2013) where you said it that it was thread safe so I was wondering if it was still the case.
It should be ThreadSafe, each request creates a new HttpWebRequest instance for each Request, the only instance shared between requests is the CookieContainer. So if you’re not using Cookies, using multiple JsonServiceClient instances should behave the same using a singleton instance (since each creates a new WebRequest for each request).
Although HttpWebRequest uses a shared connection pool behind the scenes so it’s not as isolated as .NET’s Frameworks API suggests, where for instance if you don’t dispose external HttpWebRequest’ (e.g. created/used outside of the client libraries) it can hang all future HttpWebRequest’s until they’re disposed. The connection pool is configurable with System.Net’s ServicePoint and ServicePointManager classes.
I had a bit more time to work on my gateway POC and I found that I get request timeouts when using JsonServiceClient while benchmarking the performance of the gateway.
The benchmark is fairly simple: it uses 20 connections to hit the gateway which forwards the request to a backend server. With 400 requests I get ~5-10% of request timeout. Is this the expected performance of JsonServiceClient? I do not have these timeouts when benchmarking the backend server directly.
The solution is configured to start both web sites automatically. Once the web sites are up and running you can execute load-test.cmd. This batch file uses Apache’s load testing tool (ab.exe) to load test the application and you will get an output that looks like this:
Still catching up with the after effects of last weeks release but I’ve just had a look into it, I can see the issue with using JsonServiceClient but it’s unlikely a Thread Safety issue as the same issue persists when returning a new instance, i.e:
return new JsonServiceClient("http://localhost:20690/");
Where each request uses a new instance so there’s no shared state or ThreadSafety issues. IIS Express has implicit concurrent request limitations which I’ve seen hang when too many concurrent HttpWebRequest have been issued so instead of using an ASP.NET Gateway.Web I’ve added a new SelfHost AppHost, e.g:
class Program
{
static void Main(string[] args)
{
new AppHost().Init()
.Start("http://*:20248/");
Thread.Sleep(Timeout.Infinite);
}
}
public class AppHost : AppSelfHostBase
{
public AppHost()
: base("GatewayService.SelfHost !", typeof(AppHost).Assembly) { }
public override void Configure(Funq.Container container)
{
this.Plugins.Add(new GatewayPlugin());
container.Register<IServiceGatewayFactory>(x => new GatewayFactory());
}
}
In which case returning the same JsonServiceClient instance, e.g:
private static JsonServiceClient client =
new JsonServiceClient("http://localhost:20690/");
public override IServiceGateway GetServiceGateway(IRequest request) => client;
