Hello, Demis and All,
I’m using SS for a new healthcare API. Message-based is the way to go, and SS works great.
My Questions:
I’m not seeing a whole lot of docs on the RequestLogs feature, at least for this: what is the mechanism and rules around how long log entries are kept? Is it configurable?
I did dive in to the src some. InMemory logger uses .Net ConcurrentQueue <-- maybe my answer is in there, but I’m a little new to SS and am hoping for a quick answer.
Jon Kennerly:
Thanks, Richard Safier. That is the information I was looking for.
+Jon Kennerly ok yeah, this also applies to all InMemory collections/providers/etc, which are cleared on every AppDomain recycle/restart.
Carlos Mendes:
Hi Jon,
When you add the plugin you can set the buffer capacity:
plugins.Add(new RequestLogsFeature(capacity: 100));
Hi Jon, the main description about the RequestLogs feature is at: https://github.com/ServiceStack/ServiceStack/wiki/Plugins#request-logger
Like any plugin, it can configured at registration when adding the Plugin. The different config options for RequestLogsFeature is at: https://github.com/ServiceStack/ServiceStack/blob/master/src/ServiceStack/RequestLogsFeature.cs
e.g. you can change how long the buffer is with:
Plugins.Add(new RequestLogsFeature {
Capacity = 2000
});
By default it uses an InMemoryRollingRequestLogger which has a default capacity of 1000.
Carlos Mendes:
The default capacity is 1000:
https://github.com/ServiceStack/ServiceStack/blob/master/src/ServiceStack/Host/InMemoryRollingRequestLogger.cs#L15
Jon Kennerly:
I saw Capacity. Quantity of logs entries isn’t an issue.
It’s how long the entries are valid.
Ex: 5 calls to service, 5 calls show up in the list under RequestLogsService, which is expected. Then some time later (no other requests made) those 5 calls no longer appear in the log. That is 15 min or so, and those entries are gone.
Is this garbage collection happening on the queue. Maybe I’m misunderstanding. Maybe increasing the buffer size would change the time factor.
It’s uses a rolling InMemory log by default, so it always keeps the maximum Capacity and only evicts the oldest ones once that Capacity is reached.
Note: it doesn’t hold onto the Request/Response itself, just metadata about the request (in the DTO that gets returned).
Richard Safier:
+Jon Kennerly If your hosting on most IIS versions the default app pool config would shutdown the process after 20 minutes of idle time, 0 is off. You also might want to alter the default recycle period if you need your logs longer than 29 hours.
Jon Kennerly:
wrap up:
+Richard Safier 's tip worked. That said, it’d be nice to have a sample implementation of a CustomRequestLogger that implements Redis another queuing mechanism that isn’t tied in to IIS, much in the same way a Redis ICacheClient state isn’t tied in to IIS.
I’m in a kind of rapid dev mode for a client, and publish new code quite often. The result is the logs are requestlogs lost, but I’d still like to have that history.
thanks again! (having fun here)
A custom request logger would just need to implement this interface:
https://github.com/ServiceStack/ServiceStack/blob/master/src/ServiceStack.Interfaces/Web/IRequestLogger.cs
Which can then registered with:
Plugins.Add(new RequestLogsFeature {
RequestLogger = new MyRequestLogger()
});
The custom logger could also make use of most of the impl from the InMemoryLogger:
https://github.com/ServiceStack/ServiceStack/blob/master/src/ServiceStack/Host/InMemoryRollingRequestLogger.cs
What type of logger would you prefer? Redis or RDBMS (OrmLite).
Jon Kennerly:
Redis. I plan to use that as the Cache store as well, which is already working.
It’s in the ServiceStack.Server NuGet package and it’s now called RedisMqServer.
Hi Jon, I’ve just added a RedisRequestLogger in this commit: https://github.com/ServiceStack/ServiceStack/commit/b02352a86b8e9ab57af86bf6620a8a36576d64ee
It supports an optional capacity if you want to configure it as a rolling log otherwise it will keep it until you flush redis db manually. Since logs are stored in redis the logs will survive AppDomain restarts.
RedisRequestLogger is in the ServiceStack.Server package available from v4.0.34+ that’s just finished deploying on MyGet: https://github.com/ServiceStack/ServiceStack/wiki/MyGet
Jon Kennerly:
This is great. Exactly what I was looking for. I have the pre-release code and it is working.
I’m new to Redis, but there seems be some spin-up time. Getting the list of requests using the RequestLogs service (C# client) takes around 6 second, both with 5 or 1000 requests logged.
6s sounds off, I’ve just deployed RedisRequestLogger to http://test.servicestack.net (an over saturated m1.small AWS instance) and 1000+ requests is <1s:
http://test.servicestack.net/requestlogs
And even then time spent is mostly on html rendering in the browser, as it’s even quicker if viewed with any of the data formats, e.g:
http://test.servicestack.net/requestlogs.json
http://test.servicestack.net/requestlogs.csv
Jon Kennerly:
I’m going to try and standup a different Redis instance. Some suggest stick with the *nix flavor, not Redis-on-Windows. I see you have a Redis Admin UI with the entries also. http://redisadminui.servicestack.net . I might try that tool if it is available.
I’m finally opening this project back up. Since I am more experienced with Couchbase, I’ve decided to move the log storage over to it instead of trying to resolve the performance issue.
I’ve made some headway in writing a CouchbaseRequestLogger based on the RedisRequestLogger. It has dependencies on the Couchbase C# client SDK. It may not be the most IOC-friendly implmentation, but it is a start.
It saves and retrieves as need, except for one issue.
Couchbase uses Newtonsoft vs ServiceStack for serialization. There is an issue in serialization though. The Request and Response DTO and other properties are not typed during the save.
Example: “__type”: “ServiceStack.ErrorResponse, ServiceStack.Interfaces”
“__type” is not include.
RequestLogEntry needs to be modified? I’m hoping not to have to write a full JavaScriptConverter.
Any help would be appreciated. I’m thinking Couchbase is very popular and the CouchbaseLogger would be a good fit for the SS stack when complete.
Here is the code so far: using System;using System.Collections.Generic;using System.Linq;using Serv - Pastebin.com
Thank you!
Switching to use different JSON serializers isn’t going to work to well if you’re relying on latebound type info, ServiceStack uses __type whilst JSON.NET uses $type supporting different formats.
Since you already have a dependency on the Couchbase C# client SDK, you should just be able to only use JSON.NET for serialization.
Got it. Serializes and Deserializes entire RequestLogEntry hierarchy now as expected.
Seems Couchbase was ignoring my Json.net Serialization settings. I have it set according to the latest SDK now, and so “$type” is noted in the json string as you expected.
Note that ServiceStack is mentioned here and here regarding the later ability of the SDK to use other serializers. You just implement a Couchbase.Core.Serialization.ITypeSerializer, which is probably quite easy, but I have exhausted my time on the subject.