We’re having this problem again, and it’s hard to track, especially since it happens without any warning.
So to recap:
I’ve got a class which I serialize to our DB. Since 2021/2022 these lines are in my code to force JSON always:
SqlServerDialect.Provider.StringSerializer = new JsonStringSerializer();
SqlServer2017Dialect.Provider.StringSerializer = new JsonStringSerializer();
SqliteDialect.Provider.StringSerializer = new JsonStringSerializer();
// followed by this:
ServiceStack.JS.Configure(); // this fixes a lot, don't remember why
Dialect provider is set explicitly to SqlServer2017Dialect, except for unit testing where Sqlite is used:
var dialectProvider = connectionString.StartsWith("Server=") ? SqlServer2017Dialect.Provider : SqliteDialect.Provider;
var dbFactory = new OrmLiteConnectionFactory(connectionString, dialectProvider);
services.AddSingleton<IDbConnectionFactory>(dbFactory);
–
At one point we had to add [Serializable] because we were getting deserialization error, but this was a very explicit error that threw exceptions. However it was around the same time (november) when we got the errors the first time (timeline later).
[Serializable] // to be able to serialize it in GlobalState, ref. https://docs.servicestack.net/releases/v6_06#msvr-76883
public class ActionInstanceReference
{
// some public properties
}
So what happens is that we have a class which we save to the db.
It is basically
public class ProcessInstance
{
[AutoIncrement]
public int Id { get; set; }
public Dictionary<string, object> GlobalState { get; set; } = new Dictionary<string, object>();
The GlobalState is where things sometimes goes wrong, it will only serialize the first level to JSON, then it’ll revert to JSV for the deeper nested objects.
Incorrect:
{
"employee": "{fullName:xxxx yyyy,firstName:xxxx,lastName:yyyyy,...}",
"endDate": "2025-01-31",
"__REFS__": "{confirmEmailMgr:{__type:MasterProcessServer.ServiceModel.Interfaces.ActionInstanceReference, MasterProcessServer.ServiceModel,type:Message,prettyPrint:Message to MDK 169},confirmEmailEmp:{type:Message,prettyPrint:Message to MDK 965}}"
}
Correct (taken from another row in the DB):
{
"employee": {
"fullName": "xxxx yyyy",
"firstName": "xxxx",
"lastName": "yyyy",
...
},
"endDate": "2024-11-30",
"__REFS__": {
"confirmEmailMgr": {
"__type": "MasterProcessServer.ServiceModel.Interfaces.ActionInstanceReference, MasterProcessServer.ServiceModel",
"type": "Message",
"prettyPrint": "Message to MDK 1597"
},
"confirmEmailEmp": {
"type": "Message",
"prettyPrint": "Message to MDK 580"
},
....
}
This has happened intermittently. From our logs and our backups, we can see that the GlobalState object was correctly JSON-only, then turned bad at the same time.
237 2024-10-28 14:34:56.023 BAD
240 2024-10-29 15:23:05.633 GOOD
236 2024-10-31 10:18:42.287 GOOD
243 2024-11-27 14:12:45.287 GOOD
238 2024-12-16 11:00:16.037 GOOD
239 2024-12-23 18:22:03.313 BAD
241 2024-12-23 18:22:03.653 BAD
242 2024-12-23 18:22:03.900 BAD
246 2024-12-27 08:00:12.860 GOOD
From the dates, we can assume that no one was working at the time.
I could find this in my logs:
2024-12-23 18:22:03.191 +01:00 [ERR] Activity 8557 of process instance 239 failed w. exception The best overloaded method match for 'MasterProcessServer.ServiceModel.Utils.ExpandoObjectHelper.ToExpando(System.Collections.Generic.IDictionary<string,object>)' has some invalid arguments
2024-12-23 18:22:03.970 +01:00 [INF] Failed to write error to response: {0}
ServiceStack.ConfigurationErrorsException: ServiceStack: AppHost does not exist or has not been initialized. Make sure you have created an AppHost and started it with 'new AppHost().Init();' in your Global.asax Application_Start() or alternative Application StartUp
at ServiceStack.HostContext.AssertAppHost() in /home/runner/work/ServiceStack/ServiceStack/ServiceStack/src/ServiceStack/HostContext.cs:line 34
at ServiceStack.HostContext.RaiseAndHandleException(IRequest httpReq, IResponse httpRes, String operationName, Exception ex) in /home/runner/work/ServiceStack/ServiceStack/ServiceStack/src/ServiceStack/HostContext.cs:line 261
at ServiceStack.Host.Handlers.HttpAsyncTaskHandler.HandleException(IRequest httpReq, IResponse httpRes, String operationName, Exception ex) in /home/runner/work/ServiceStack/ServiceStack/ServiceStack/src/ServiceStack/Host/Handlers/HttpAsyncTaskHandler.cs:line 180
So once we got the error about “ToExpando” then I traced back and finally arrived at a DB ORMLite save call. It was guarded by this:
// just a bug hunt:
if (SqlServerDialect.Provider.StringSerializer.GetType() != typeof(JsonStringSerializer)
|| SqliteDialect.Provider.StringSerializer.GetType() != typeof(JsonStringSerializer)) {
throw new MasterServiceException("Wrong db serializer! ");
}
However I note that it’s missing SqlServer2017Dialect and only has SqlServerDialect so of course there was nothing in the logs.
Question: Is there any case where OrmLite would revert to JSV ?
- Like if somehow my Dictionary<string,object> (where object can be another dict of same type) got messed up somehow into another class or something?
- Or could it be something with that last line in the log about the AppHost. Because after that I have no more log lines for the failed processes (so probably that’s when they failed). Perhaps some “default apphost” started after that? One without the JSON serialization options set?