C# client heartbeat timeout issue

There isn’t, but it just wraps an in memory ConcurrentDictionary so it shouldn’t be noticeable.

You could call GetKeysByPattern(“*”) which will return dictionary.Keys which will save you an Enumeration.

No the cache key pattern is hard-coded against the IdUtils.CreateUrn<IAuthSession> to create a predictable urn per type.

Hi mythz,

Do you have a typical(recommended) IIS/Asp.Net configuration for a rather large scale user site? Say the number of concurrent online users is over 10,000.

Thanks,

No but general IIS/ASP.NET questions like this are best asked on StackOverflow.

Hi mythz,

The reason why I asked the question here is because of the design of the servicestack SSE. It seems to keep all the HTTP requests alive for the channels. So if there are 10,000 online users, the active requests in IIS is 10,000 and won’t release until the user logs out. Is this expected behavior?

Each SSE connection needs to be kept alive by design, i.e. it needs an active connection to be able to send notifications to, but each User/SSE Subscription should only need 1 connection even if they’re subscribed to multiple channels.

Thanks, mythz!
I have one more question. Is there a recommended settings for the IIS recycling? Since all the connections need to be kept alive, if the application pool is recycled by IIS, all the connections are lost. Is there a standard way to deal with this situation?
Hong

When the AppDomain recycles the SSE connections will drop, but all clients should have built-in auto-reconnect.

Yeah, I understand that. But if we have 30,000 connections, they will all be connecting to the server simultaneously after recycling. That could be a problem. Do you have experience in that situation?
The problem we are experiencing is the authentication is quite slow and it fails constantly when a lot of users are trying to authenticate. What do you suggest to persist the auth sessions?

I’d avoid to have clients re-authenticating, you could use a distributed caching provider like redis or a stateless auth provider like JWT which wont be affected by AppDomain restarts.

I tried to use Redis to replace the MemoryCacheClient, but I encountered a strange error.
I am wondering if you have seen this error before and what I did wrong.
Thanks.

Here is what I did.
I installed the Redis 3.2.100
I change the config to use RedisClient as suggested
public sealed override void Configure(Funq.Container container)
{
var hostConfig = GetConfiguration();

		SetConfig(hostConfig);

        container.Adapter = new IOC.WindsorContainerAdapter();
        // container.Register<ICacheClient>(new MemoryCacheClient());
        container.Register<IRedisClientsManager>(c =>
            new PooledRedisClientManager("localhost: 6379"));

        container.Register(c => c.Resolve<IRedisClientsManager>().GetCacheClient());

When I try to authenticate a user, it throws an exception.
Msg=“Unknown authentication error System.Net.Sockets.SocketException (0x80004005): The attempted operation is not supported for the type of object referenced
at System.Net.IPAddress.get_Address()
at lambda_method(Closure , IPAddress )
at ServiceStack.Text.Common.WriteType2.WriteProperties(TextWriter writer, Object value) at ServiceStack.Text.Common.WriteType2.WriteProperties(TextWriter writer, Object value)
at ServiceStack.Text.JsonSerializer.SerializeToString(Object value, Type type)
at ServiceStack.Text.JsonSerializer.SerializeToString[T](T value)
at ServiceStack.StringExtensions.ToJson[T](T obj)
at ServiceStack.Redis.RedisClient.ToBytes[T](T value)
at ServiceStack.Redis.RedisClient.<>c__DisplayClass26_01.&lt;Set&gt;b__0(RedisClient r) at ServiceStack.Redis.RedisClient.Exec(Action1 action)
at ServiceStack.Redis.RedisClient.Set[T](String key, T value, TimeSpan expiresIn)
at ServiceStack.Redis.RedisClientManagerCacheClient.Set[T](String key, T value, TimeSpan expiresIn)
at ServiceStack.ServiceExtensions.CacheSet[T](ICacheClient cache, String key, T value, Nullable1 expiresIn) at ServiceStack.ServiceStackHost.OnSaveSession(IRequest httpReq, IAuthSession session, Nullable1 expiresIn)
at Mck.EIWI.WebCommunication.WebAppHost.OnSaveSession(IRequest httpReq, IAuthSession session, Nullable1 expiresIn) in e:\work\WorkflowIntelligence\Ongoing\EnterpriseWorklist\Mck.EIWI.WebCommunication\WebAppHost.cs:line 61 at ServiceStack.Auth.AuthExtensions.SaveSession(IAuthProvider provider, IServiceBase authService, IAuthSession session, Nullable1 sessionExpiry)
at ServiceStack.Auth.CredentialsAuthProvider.OnAuthenticated(IServiceBase authService, IAuthSession session, IAuthTokens tokens, Dictionary2 authInfo) at ServiceStack.Auth.CredentialsAuthProvider.Authenticate(IServiceBase authService, IAuthSession session, String userName, String password, String referrerUrl) at ServiceStack.Auth.CredentialsAuthProvider.Authenticate(IServiceBase authService, IAuthSession session, Authenticate request) at Mck.EIWI.WebCommunication.Auth.MonitorAuthProvider.Authenticate(IServiceBase authService, IAuthSession session, Authenticate request) in e:\work\WorkflowIntelligence\Ongoing\EnterpriseWorklist\Mck.EIWI.WebCommunication\Auth\MonitorAuthProvider.cs:line 89" /> <Event Time="2017/05/11 14:40:35.3750448" Logger="Mck.EIWI.Server.PicardAppHostBase" Level="ERROR" Thread="25" Msg="Unhandled service exception for request '/meiwi.WebCommunication/api/json/reply/Authenticate'&#xA;&#xD;[request DTO]&#xA;&#xD;ServiceStack.Authenticate&#xA;&#xD; System.Net.Sockets.SocketException (0x80004005): The attempted operation is not supported for the type of object referenced at Mck.EIWI.WebCommunication.Auth.MonitorAuthProvider.ProcessAuthException(Exception ex) in e:\work\WorkflowIntelligence\Ongoing\EnterpriseWorklist\Mck.EIWI.WebCommunication\Auth\MonitorAuthProvider.cs:line 142 at Mck.EIWI.WebCommunication.Auth.MonitorAuthProvider.Authenticate(IServiceBase authService, IAuthSession session, Authenticate request) in e:\work\WorkflowIntelligence\Ongoing\EnterpriseWorklist\Mck.EIWI.WebCommunication\Auth\MonitorAuthProvider.cs:line 95 at ServiceStack.Auth.AuthenticateService.Authenticate(Authenticate request, String provider, IAuthSession session, IAuthProvider oAuthConfig) at ServiceStack.Auth.AuthenticateService.Post(Authenticate request) at lambda_method(Closure , Object , Object ) at ServiceStack.Host.ServiceRunner1.Execute(IRequest request, Object instance, TRequest requestDto)” />

Don’t include a space with the port, just try:

container.Register<IRedisClientsManager>(c =>
            new PooledRedisClientManager("localhost:6379"));

Well, I guess that is not the issue. The problem was in my auth session there is an IPAddress object. It seem when the object is serialized to Json it fails. I am not sure how servicestack serializes the IPAddress object though.

Only serialize data types, change it to a string on your custom session.

Sorry, another question. I can save the session now in Redis, however when IIS recycles, it doesn’t seem to use the old session and sends 401 error when the client tries to reconnect. I see someone is talking about using RedisAuthRepository but I have no idea if I need to configure it and how it is used. Can you help me with that?
Thanks a lot!

That doesn’t sound right, if the same session ids are sent with the request then they’ll be referencing their authenticated User Session that’s stored in Redis.

My servicestack version is 4.5.6 signed. servicestack.redis is also 4.5.6 signed.

I can confirm the same session ids (both ss-id and ss-pid) are sent to the server. And also in the log it shows it gets the auth object from Redis with the same permanent session id. It is not supposed to return 401

2017-05-12 16:58:15,833: DEBUG 10 RedisNativeClient - S: GET urn:iauthsession:uK55l47eapRG7CyOLJQC
2017-05-12 16:58:15,834: DEBUG 10 RedisNativeClient - R: $1635

also confirmed the isAuthenticated is true

I guess the code below is returning 401 error. I have no idea why session.IsAuthenticated is false.
var session = req.GetSession();
if (feature.LimitToAuthenticatedUsers && !session.IsAuthenticated)
{
session.ReturnFailedAuthentication(req);
return TypeConstants.EmptyTask;
}

I don’t understand why either, the users session is resolved directly from the cache using the session ids which looks like is properly populated. Do you have any custom configuration which might interfere with the session? Can you try explicitly registering:

container.Register(c => c.Resolve<IRedisClientsManager>().GetCacheClient());

If it’s still an issue please post the full raw HTTP Request and 401 Response.

Thanks, mythz
It turns out to be my bad. I have some code to update the auth session which saves it as my user auth session type. I changed it to save as IAuthSession and it works fine now.

1 Like

Hi mythz,

I am a little confused about Redis Server Events. I read the description on servicestack github page and it seems to replace MemoryServerEvents. When I tried it, it did persist the connections in Redis. However, it seems to publish events to Redis instead of to the SSE client like MemoryServerEvents does. Can you give me a little more information on how Redis Server Events works?

Thanks,

Hong