I am still fiddling around with the appropriate settings for HeartbeatInterval and IdleTimeout. To get an impression what is going on on my server I wrote the following code:
Plugins.Add(new ServerEventsFeature
{
HeartbeatInterval = TimeSpan.FromSeconds(15),
IdleTimeout = TimeSpan.FromSeconds(60),
OnConnect = (subscription, dict) =>
{
// some debugging output
},
OnCreated = (subscription, request) =>
{
// some debugging ouput
},
OnPublish = (subscription, response, data) =>
{
Log.Info($"PUBLISH EVENT ..."); // This receives also HEARTBEATS!!
// for testing only:
using (var service = HostContext.Resolve<ServerEventsSubscribersService>())
{
Log.Info("SUBSCRIPTION STATS:");
var subscriptionInfos = service.ServerEvents.GetAllSubscriptionInfos();
if (subscriptionInfos.Count == 0) Log.Info("There are currently NO subscriptions registered!");
else
{
Log.Info($"There are currently {subscriptionInfos.Count} registered subscriptions.");
foreach (var subscriptionInfo in subscriptionInfos)
{
Log.Info($"subscription: {subscriptionInfo.SubscriptionId} | {subscriptionInfo.UserName}");
}
}
}
Log.Info($"Subscription: {subscription} | Response: {response} | Data: {data} | Size of data: {data.Length} chars.");
},
OnSubscribe = (subscription) =>
{
// some debugging output
},
OnUnsubscribe = (subscription) =>
{
// some debugging output
}
});
The interesting bit is the OnPublish() handler which also covers the heartbeat. The log output on my dev-machine looks as follows:
2018-02-22 16:38:58,539; [14]; INFO ; BizBusLicenseServer; [AppHost+<>c.<Configure>b__15_8]; - PUBLISH EVENT ...
2018-02-22 16:38:58,549; [14]; INFO ; BizBusLicenseServer; [AppHost+<>c.<Configure>b__15_8]; - SUBSCRIPTION STATS:
2018-02-22 16:38:58,557; [14]; INFO ; BizBusLicenseServer; [AppHost+<>c.<Configure>b__15_8]; - There are currently 1 registered subscriptions.
2018-02-22 16:38:58,566; [14]; INFO ; BizBusLicenseServer; [AppHost+<>c.<Configure>b__15_8]; - subscription: xmWc9QHQ9YbwPzkIQrGN | BizBusLicenseAdmin
2018-02-22 16:38:58,575; [14]; INFO ; BizBusLicenseServer; [AppHost+<>c.<Configure>b__15_8]; - Subscription: ServiceStack.EventSubscription | Response: ServiceStack.Host.HttpListener.ListenerResponse | Data: id: 4
data: cmd.onHeartbeat {"userId":"1","isAuthenticated":"true","displayName":"BizBusLicenseAdmin","channels":"LsChannel","createdAt":"1519313923498","profileUrl":"https://raw.githubusercontent.com/ServiceStack/Assets/master/img/apps/no-profile64.png"}
| Size of data: 257 chars.
What really surprises me is the size of 257 characters for every heartbeat message. My software will be part of a cloud application which runs in a production environment with a Docker / Kubernetes / OpenShift on CentOS managed by a partner company. We expect over time several thousand concurrent users.
My current settings are one heartbeat every 15 seconds, if four heartbeats are missed, the connection will be closed. So for a 1000 connections this means at least 1 MB network noise just for heartbeats every minute. Given the fact, that some users may be connected through mobile devices (slow networks) this looks like a lot too much traffic to me.
I don’t know, maybe I get this wrong or I am looking at the wrong data, but if this is really the content of every heartbeat it should be only a very small amount of bytes something like an OK or just a boolean. I have the impression that this heartbeat is providing data, which should not belong to a keep alive.
Maybe the idea is to send a heartbeat only every 60 seconds (not ideal too, since it would require an idle timeout of maybe 5 minutes which may keep a lot of unnecessary connections on the server also consuming ressources)? What is the idea behind this large datagram? Would it be possible, to reduce the size of this message a bit, maybe through configuration?
Thanks for some feedback and clarification. The guys responsible for the production cloud environment will HATE me, if I deliver software which is ‘noisy on the network’!