I’d like to discuss a design issue with having an SSE client running from within a ServiceStack service.
I cannot seem to wrap my head around the lifecycle part of the SSE client in the lifecycle of my SS service. I’ve not had to think about keeping anything else alive with my SS service until now. But the SSE client needs to be alive from the moment the service starts.
Basically, I’ll need a (single) instance of an SSE client to be alive for as long as my service is alive, so that it instruct my service to do stuff when a SSE event comes through from some SSE server elsewhere.
So initially, I thought creating the SSE client in my AppHost.Configure() method was the right way to go.
I have subsequently discovered that if I do that, and hold the SSE client instance in my AppHost, then if the SSE client throws an exception (say because it lost connection or something with SSE Server) than it seems to throw and bring down my SS service. Presumably, killing my AppHost instance?
This is what seems to happen. I can detect that my SSEClient.OnException delegate is called and shortly after my service dies. I am assuming they are related, since we had not had problems with the service dying before we added this SSE client.
If this is by design, then how would you manage the lifecycle of an SSE client to be alive as long as the SS service without bringing the service down when the SSE client throws an exception?
The client should just be like any other dependency, if you want to keep an instance around you should assign the instance to a field in AppHost or register the singleton instance in the Container so there’s a reference to it.
But this shouldn’t be happening, if the SSE client connection is broken it should be automatically be retrying to connect. Do you have a StackTrace showing what’s causing this or a small repro that exhibits this behavior?
If it’s bringing down the ASP.NET App Domain down I’m assuming it’s due to an UnobservedTaskException which you can handle by registering a handler in App_Start, e.g:
This should prevent unhandled async exceptions from bringing down the Web App, you’ll also want to add some logging around this as well to see what’s causing this. There’s some more details about unhandled ASP.NETasync Exceptions in this blog post.
I tried to debug ServiceStack.Client.dll for you to try and identify the exact line throwing the exception, but my debugger cannot load the symbols for this current version (v55) from the known symbols places.
I can’t repro any unhandled exception, I’ve got it running with the network off and it’s just retrying continually as expected. Are you sure your exception is an unhandled exception and it’s not just stopping on a normal exception? i.e. does it say anywhere it’s an unhandled exception?
I’m going to let it run with the network off for another 10 more minutes with continual failed heartbeats + retries and see if I can get it to repro an unhandled exception, will let you know if it reconnects or when the networks back or not.
Nope still can’t repro any unhandled exception, I’ve left my network disconnected for over 20 minutes and as soon as I reconnected the network again it automatically re-established the SSE connection to the remote http://chat.servicestack.net SSE Server and started receiving messages again. AFAICT it’s working as expected.