Faulty Redis configuration hangs indefinitely when trying to connect

ServiceStack.Redis
1

Hi,

I stumbled upon this error when testing our configurations for connecting to Redis Cache

If I provide a faulty configuration, in this case I run a redis instance locally via Docker which do not have Ssl enabled but when using the RedisNativeClient I set Ssl to true the SslStream hangs indefinitely

var endpoint = new RedisEndpoint("localhost", 6379, "") {Ssl = true};

new RedisNativeClient(endpoint).Ping();

As I can see this hangs at RedisNativeClient_Utils.Connect():

sslStream.AuthenticateAsClient (Row 175/178)

However, if I prior to that set sslStream.ReadTimeout to a value it seems to work properly though and times out after that value has expired

Now, my question is if there is a workaround for setting a timeout so our application wont hang if our configuraiton is not set up properly?

2

The default Redis Config options are available on RedisConfig, e.g. you can configure the default TCP Socket Connection timeout with:

RedisConfig.DefaultConnectTimeout = 20 * 1000;

This is also configurable in the RedisEndpoint, e.g:

new RedisEndpoint(...) {
    ConnectTimeout = 20 * 1000
}

And ?connectTimeout=... on the Redis Connection String.

4

Thank you for fast reply,

I tried creating a simple test for the issue, maybe Im missing something but this hangs as well unfortunately

[TestMethod]
public void CanHandleFaultyConfig()
{
    var endpoint = new RedisEndpoint("localhost", 6379)
    {
        Ssl = true,
        ConnectTimeout = 20 * 1000
    };
    RedisConfig.DefaultConnectTimeout = 20 * 1000;

    var client = new RedisNativeClient(endpoint);

    Assert.AreEqual(false, client.Ping());
}
5

If you have an instance connected on port 6379 then it does make a connection so the connection timeout does not apply. If you’re trying to establish an SSL connection to a non SSL endpoint it will hang on this line:

I’m not seeing any overload that accepts a Timeout and multiple Google searches suggests this API does hang. Still looking through results to see if there’s a way to configure a timeout.

6

Thank you,

I managed to sort it out by setting the ReceiveTimeout for that particular scenario, it all works well now (except that the timeout message seems to reflect 10s instead of 20s)

    [TestMethod]
    public void CanHandleFaultyConfig()
    {
        var endpoint = new RedisEndpoint("localhost", 6379)
        {
            Ssl = true,
            ReceiveTimeout = 20 * 1000
        };

        var client = new RedisNativeClient(endpoint);

        Assert.AreEqual(client.Ping(),  false);
    }

ServiceStack.Redis.RedisException: Exceeded timeout of 00:00:10 —> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

7

That’s the auto retry timeout configurable with RedisConfig.DefaultRetryTimeout.

8

.NET Core + .NET v4.7.2 builds now uses the ConnectTimeout when waiting for AuthenticateAsClientAsync() to establish an SSL connection, otherwise throws.

This change is available from v5.10.3+ that’s now available on MyGet.

9

Thanks, that was fast indeed

Maybe it is already catered for but another type of misconfiguration seems to cause a stackoverflow exception as well. It’s when I use a hostname for a redis instance that has ssl enabled and I dont set Ssl to true, like the configuration below.

    [TestMethod]
    public void CanHandleFaultyConfig()
    {
        var endpoint = new RedisEndpoint("my.sslenabled.redisinstance.com", 443)
        {
            Ssl = false
        };

        var client = new RedisNativeClient(endpoint);

        Assert.AreEqual(client.Ping(),  false);
    }

The error raised here should be ““Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.”” and it seems like GetRetryableException sets socket to null and it has the impact that Connect() gets stuck in an eternal loop, I’m not totally sure a simple Ping() should try to reconnect like that since it causes a Stackoverflow Exception in the end. Not a big issue for us but might be worth looking into as well perhaps.

10

There’s no “simple” commands in Redis, i.e. all commands are treated the same way which all go through the same resilient auto retry connection loop. But I’ll look into how the SSL misconfiguration is impacting the auto reconnection handling.