Query for Service Stack

1

We’ve developed services running in Kubernetes Cluster on Azure Cloud with multiple pods. During the run time, multiple pods tried to read from Azure Redis Cache via Service Stack Redis Cache client. Sometimes we’re getting continuously below error, it corrects over a period of time.

at ServiceStack.Redis.RedisClient.GetValue(String key) in C:\BuildAgent\work\b2a0bfe2b1c9a118\src\ServiceStack.Redis\RedisClient.cs:line 263
   at DTCC.ReportHub.Framework.CacheStore.Application.Implementation.EntityTypeRedisCacheProvider`1.<>c__DisplayClass12_2.<<Get>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Polly.AsyncPolicy.<>c__DisplayClass40_0.<<ImplementationAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Polly.Retry.AsyncRetryEngine.ImplementationAsync[TResult](Func`3 action, Context context, CancellationToken cancellationToken, ExceptionPredicates shouldRetryExceptionPredicates, ResultPredicates`1 shouldRetryResultPredicates, Func`5 onRetryAsync, Int32 permittedRetryCount, IEnumerable`1 sleepDurationsEnumerable, Func`4 sleepDurationProvider, Boolean continueOnCapturedContext)
ServiceStack.Redis.RedisException: Exceeded timeout of 00:00:05
---> System.Net.Sockets.SocketException (110): Connection timed out
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 count)
   --- End of inner exception stack trace ---

We need your help in finding the root cause for the same

2

Exceeded timeout means it the Redis Client wasn’t able to establish a TCP Connection with the server within the configured Retry Timeout.

The Automatic Retries defaults to 10 seconds so I’m assuming it’s been configured with a lower timeout.

Increasing it to a higher timeout instructs the client to keep retrying until it’s able to establish a connection for that configured time, e.g:

RedisConfig.DefaultRetryTimeout = 10000;

If it’s unable to establish a connection it will throw this error.