Sentinel call is failing because of network latency

We are using service stack library (v 5.7)to connect to Redis using Sentinel. But, using this library we can’t access sentinel if latency is greater than 200 ms. We have remote developer in Singapore and he tried to hit our US-West Redis server and it is not working. Is there any configuration we can set so that sentinel call to Redis works for higher latency about 300 ms? Waiting for quick reply.

Thank you.

You can configure Network configuration changes from the RedisConfig class.

The RedisConfig.DefaultConnectTimeout is already configured to have no timeout, you may instead be exceeding the RedisConfig.HostLookupTimeoutMs which defaults to 200ms:

I have pasted our code for your reference. Whatever we set 0 , 1, 200, 300, 600, 1000, it is not working. Odd thing is, when developer is working from our Harrisburg(Pennsylvania USA) office it works for all values including 0 and 1. The latency time from Harrisbug machine is 75. But, when our remote developer works from Singapore VM, Latency time is 160 it doesn’t work I think Setting RedisConfig.HostLookupTimeoutMs` is not working. Is there anything I can do to make it working?

static void Main(string[] args)
{

        string sentinelHostsString = ConfigurationManager.AppSettings["SentinelHosts"];
        string redisIP = ConfigurationManager.AppSettings["RedisIP"];
        string DirectToRedis = ConfigurationManager.AppSettings["DirectToRedis"];
        //string[] sentinelHosts = new string[] { "10.8.0.7:26579" };
        string[] sentinelHosts = sentinelHostsString.Split(',').ToArray();
        Console.WriteLine("Please wait..");
        if(DirectToRedis.ToLower() == "yes")
        {
            DirectlyConnectToRedis(redisIP);
        }
        else
        {
            GetMasterIPAddress(sentinelHosts, ConfigurationManager.AppSettings["MasterName"]);
        }
        
        Console.WriteLine("-------------------------------------");
        Console.ReadLine();

    }

    static string GetMasterIPAddress(string[] sentinelHosts, string masterName)
    {
        RedisConfig.HostLookupTimeoutMs = int.Parse(ConfigurationManager.AppSettings["HostLookupTimeoutMs"]);

        RedisSentinel sentinelSet = new RedisSentinel(sentinelHosts, masterName);

        sentinelSet.Start();
        sentinelSet.SentinelWorkerConnectTimeoutMs =  int.Parse(ConfigurationManager.AppSettings["SentinelWorkerConnectTimeoutMs"]);
        sentinelSet.SentinelWorkerReceiveTimeoutMs = int.Parse(ConfigurationManager.AppSettings["SentinelWorkerReceiveTimeoutMs"]);
        sentinelSet.SentinelWorkerSendTimeoutMs = int.Parse(ConfigurationManager.AppSettings["SentinelWorkerSendTimeoutMs"]);

        sentinelSet.HostFilter = host => $"ppsCache1739@{host}";
        do
        {
            try
            {
                List<RedisEndpoint> slaveRedisEndpoint = sentinelSet.GetSlaves();

                RedisEndpoint masterRedisEndpoint = sentinelSet.GetMaster();

                if (masterRedisEndpoint != null)
                {
                    Console.WriteLine("Successfully got master ip address: " + $"{masterRedisEndpoint.Host}:{masterRedisEndpoint.Port} @ {DateTime.UtcNow.ToString("yyyy-MM-dd HH:mm:ss.fff")}");
                }
            }
            catch (Exception ex)
            {
                EventLog logger = new EventLog();
                logger.Source = "Application Error";
                logger.WriteEntry(ex.Message, EventLogEntryType.Error);
                Console.WriteLine("Could not retrieve master Ip Address: " + ex.Message);
            }
        } while (true);
    }

    static string DirectlyConnectToRedis(string redisIP)
    {
        bool pinged = false;
        do
        {

            try
            {

                using (var redisManager = new PooledRedisClientManager(redisIP))
                using (var redis = redisManager.GetClient())
                {
                    pinged = redis.Ping();

        
                    string masterRedisEndpoint = redis.GetHostString();

                    if (masterRedisEndpoint != null)
                    {
                        Console.WriteLine("Successfully got master ip address: " + $"{masterRedisEndpoint} @ {DateTime.UtcNow.ToString("yyyy-MM-dd HH:mm:ss.fff")}");
                    }


                }


                
            }
            catch (Exception ex)
            {
                EventLog logger = new EventLog();
                logger.Source = "Application Error";
                logger.WriteEntry(ex.Message, EventLogEntryType.Error);

                if(pinged)
                {
                    Console.WriteLine("Pinged Successfullt but Could not retrieve master Ip Address: " + ex.Message);
                }
                else
                {
                    Console.WriteLine("Could not retrieve master Ip Address: " + ex.Message);
                }
            }

        } while (true);
    }
}

}

Please post the full Exception StackTrace, it’s not clear what the issue is exactly.

Here it is.

This shows the Auto Retry Limit for trying to resolve a Redis Sentinel Host was exceeded, i.e. you have more networking connectivity issues than just latency, this StackTrace suggests it wasn’t able to establish a TCP connection within 10 seconds.

You can increase the Auto Retry limit with:

RedisConfig.DefaultRetryTimeout = 30 * 1000; // e.g. 30 seconds