Nicklas Laine Overgaard - 469 - Mar 14, 2014

I’m facing some trouble with a SS v4 API hosted in Amazon AWS via the Elastic Beanstalk deployment platform. I can’t remember the exact version of SS running on the server, but it’s Windows 2012 with .net 4.5.

Basically all the API is doing right now is sitting on amazon, on one small EC2 instance behind a load balancer. Each 30 seconds the load balancer is pinging a heartbeat route and all this route does is to construct a response saying OK.

What we are experiencing is a linear increase in the response-time of our application. I’ve pulled the IIS request log files and plotted the average of the measured response times on a per day basis:
http://d.pr/i/7fLs

When the response time breaches 5 seconds (which is quite a lot) the load balancer is marking the instance as unhealty.

Is this something other customers have reported?
For now I’ll try to update to the latest version of SS and redeploy.

Nicklas Laine Overgaard:

I updated to SS v4.0.13 and it’s been running in amazon for the last 3 days. The latency is still slowly building up:
http://screencast.com/t/C5G9V1HhacU

What can we do to try and resolve this? 

Drazen Dotlic:

The latency shown is monstrous for (if I understood correctly) such a simple request/response. Have you tried with other cloud ‘providers’ or is that not an option? In our own internal tests (so not cloud) SS is extremely low overhead so I find your empirical observations very surprising, especially the fact that latency rises over time.

Nicklas Laine Overgaard:

+Drazen Dotlic Thanks for getting back to me! I’m also very surprised to see these results which is why I reached out here :slight_smile:

I haven’t tried any other cloud providers yet, but I’ve been looking at the windows servers the code is running from, and they all seem fine in regard to CPU / RAM / Disk load.  Could it have something to do with the fact that the IIS AppPool is never recycled since the ELB is kepping it alive?

What I’m going to do is setup a local windows environment, attach a profiler to the IIS site, and simulate the ELBs requests, which will hopefully reveal what is responsible for the slowdown.

I was hoping that +Demis Bellot had some insights on this matter though.

Nicklas Laine Overgaard:

+Drazen Dotlic +Demis Bellot  I have managed to handle this issue, and it was some of my own code and not the ServiceStack framework causing the problems. More specifically i’ve ported the SQL storage for miniprofiler to MySQL.