ServerEvents sometimes hang while executing Start()

ServiceStack Server Events
1

Hi guys
I have a really strange issue I’m not able to understand. I’m using SSE and sometimes it hangs till timeout when executing Start() method.

This is my apphost




public class MaresApiHost : AppHostHttpListenerPoolBase
{



public override void Configure(Container container)
        {
            Plugins.Add(new SwaggerFeature());
            Plugins.Add(new ServerEventsFeature());
            Plugins.Add(new AuthFeature(() => new CustomAuthUserSession(),
                new IAuthProvider[]
                {
                    new PinAuthProvider(),
                    new UserNameAuthProvider()
                }
            ));

            //registro i database
            container.Register<IDbConnectionFactory>(ServiceEnvironment.Mares40ConnectionFactoryName, c => new OrmLiteConnectionFactory(ApplicationVariables.Instance.ConnectionStringToMasterDatabase, SqlServerDialect.Provider));
            container.Register<IDbConnectionFactory>(ServiceEnvironment.Analysis40ConnectionFactoryName, c => new OrmLiteConnectionFactory(ApplicationVariables.Instance.ConnectionStringToAnalysis40Database, SqlServerDialect.Provider));

            //registro la cache sul database
            container.Register<OrmLiteCacheClient>(c => new OrmLiteCacheClient() { DbFactory = c.ResolveNamed<IDbConnectionFactory>(ServiceEnvironment.Mares40ConnectionFactoryName) });
            container.Register<ICacheClient>(c => c.Resolve<OrmLiteCacheClient>());
            container.Resolve<ICacheClient>().InitSchema();

            //registro tutte le rotte
            Routes.AddFromAssembly(base.ServiceAssemblies.ToArray());

            //Configs the host
            SetConfig(new HostConfig
            {
                DebugMode = true,
                EnableFeatures = Feature.All,
            });

            //Prevents Json serialized to add __type part on json string for anonymous types serialization
            //Read this: -http://stackoverflow.com/questions/18842685/servicestack-sessions-doesnt-work-when-using-jsconfig-excludetypeinfo
            JsConfig.Init(new Config
            {
                DateHandler = DateHandler.ISO8601,
                ExcludeTypeInfo = true,
                ExcludeDefaultValues = true,
                IncludeNullValues = false
            });
        }
}

on my consumer I’m using a wrap class to execute the Start in this way


public bool Start()
        {
            try
            {
                lock (locker)
                    this.EventClient.Start();

                this.IsStarted = true;
            }
            catch (Exception ex)
            {
                Log.Error("Start", ex);
                this.IsStarted = false;
            }

            return this.IsStarted;
        }

where EventClient is


public ServerEventsClient EventClient { get; private set; }

created in this way


this.Url = $"{ApplicationVariables.Instance.APIProtocol}://{ipAddress}:{ApplicationVariables.Instance.APIPort}/";
this.EventClient = new ServerEventsClient(this.Url);

and the ServerEventsClient sometimes hangs while executing start() till a Timeout exception. My apphost is running on a server that is always reachable. It seems like if it is waiting for a previous task to complete, but I don’t know where I can search.

Is it a problem on the apphost side or in the client side? Is the ServerEventClient threadsafe?

Best regards
Enrico

2

Calling Start() is ThreadSafe so you don’t need to add your own locking around it (it only allows 1 call through to Start the client), this might be what’s causing your client to hang.

Otherwise if you’re running into multiple concurrent request issues you can try increasing the ServicePointManager.DefaultConnectionLimit, e.g:

ServicePointManager.DefaultConnectionLimit = 10;
3

I tryed changing the DefaultCollectionLimiti to 10 as suggested without luck. If I execute my application in the same apphost computer everything works fine, I asked the IT guys to check for any network issues, and they didn’t find anything strange. I try to repro the issue using a console app that execute the same code and it works in every condition, but my application keep on hang. I’m trying to repro with a smaller wpf application instead of mine to see if it might be due to some external component that we use.

I will let you know
Enrico

4

I have news, some bad some good: BAD: problem keep on persist; GOOD: I was able to reproduce the issue in a new application.

I was focusing on why the start method hang, and I found some deadlock I removed. After this fix, the scenario changed, now the start does not hang anymore, but now hangs every rest methods executed with a JsonServiceClient after the creation of a second instance of a SSE.

Let me explain better: my application has 2 different views: the first one is the header of the app, the second one is a kind of dynamic content. When I load the app, the header create its own SSE client to listen to a specific message and then it load the dynamic content. the dynamic content create its own SSE to listen to a different kind of message. The issue raise once the second SSE is created.

Due to the fact that I don’t trust anyone, neither my code, I create I blank application and keep on add code until I finally reproduce the problem.

What is really really strange is this: connecting to a local apphost it works, connecting to a remote apphost is does not.
Ideas?

What is happening is exactly what is described here

changing the ServicePointManager.DefaultConnectionLimit does not fix the problem. Keep in mind that mine is a selfhost app host. Should I use a singleton?

I can give you the working project to reproduce the issue

Enrico

5

Yes please publish a repo on GitHub and send me the link.

6
7

You need to configure DefaultConnectionLimit on the client App making the .NET HttpWebRequest connections, not on the server. Add this to your HeaderViewModel:

System.Net.ServicePointManager.DefaultConnectionLimit = 10000;
8

ok, perfect. What is the downside of increase that number? What if I set it to int.MaxValue?

Regards
Enrico

9

It restricts the number of concurrent connections per domain, if you set a large number it wont enforce the restriction, simple as that.

10

great, thanks. I move the limit setup on the client side and it works
Thanks
Enrico