We are moving our .net 4.8 web services from Web API/AutoFaq to ServiceStack/Func. We have moved perhaps 100 types and created more than 300 routes with no issues. We had to create DTOs but the code in the service stayed largely the same. We recently stated getting the attached exception. We dutifully marked up the impacted types with the [Serializable] attribute. The exception persisted. The exception was the same but the identified class name changed so we knew/thought we were making progress. We started adding the attribute to the entire codebase (100’s of types), until it became obvious there was something else going on and this exception was a red herring.
The server was working flawlessly until about 1 month ago. About that time we added a 3rd party SDK (Lead Technologies) to the server and start performing some image processing. Its only then when we get this exception. To be clear, the entire system works flawlessly until we go through a code path that uses func and Lead.
We had an issue about a month ago where the ServiceStack serialization could not serialize a Lead type. We had to design around that. If I had to guess I would bet there is something in Lead that func does not like.
This exact tech stack works in Web API/Autofaq and fails consistently in ServiceStack/func. If we wire the interfaces and types manually, take func out of the stack it works just fine. I’m fairly confident we narrowed the issue to func and Lead.
I’m guessing the binary serialization is happening to cross app domain boundaries.
We are stuck. We want to continue to use func but we cant - at least as it is configured now. I have read the IOC page on your site - unsure if any there will help us. Please advise.
This StackTrace uses BinaryFormatter which doesn’t use or have anything to do with any of ServiceStack Serializers, none of which use the [Serializable] attribute, which is only used by .NET’s serializers like BinaryFormatter.
But I don’t understand what’s going on here, there shouldn’t be any serialization happening in the IOC. This StackTrace suggests something is trying to serialize some of Funq’s types, namely its IOC generic class registration entry which was never intended to be serializable. It may even be trying to serialize a compiled Func<Container,IDocumentDirector> delegate? which is even worse.
My only suggestion is to not attempt a solution that relies on anything like this. Maybe you’ll have better luck resolving instances from the IOC and serializing those (i.e. instead of the whole IOC), but I’d personally only consider solutions that serialize serializable models, e.g. DTOs.
Thank you for your prompt reply. I have no idea what is going on either and agree that there should be no serialization going on with IOC. The text version of the stack trace is easier to read than the image (we cant upload text) and in the text version there is “System.Runtime.Remoting.Channels.CrossAppDomainSerializer”. The IOC container is initialized AppHost.Configure and accessed ServiceStack.TryResolve in the “controller”. In asp.net/web api these are two different threads, guessing servicestack is similar. The types in question are not DTOs, not POCOs, they are part of of DAL. (Data Access Layer) and were never intended to be serialized and for the life of me I don’t know why they are but stack traces don’t lie. The same stack work in web api/autofaq which is even more confusing.
Here is a little more information on this. We are using Visual Studio, the testing framework and the JsonServiceClient for testing. When using this stack we get this error. I have looked at the JsonServiceClient and ultimately it calls HTTPWebRequest.GetRequest so no magic here. When we use this stack for testing I get the error. When I execute the the API via swagger in a browser I do not. Unsure what is the diff between an HTTPWebRequest inside VS and an HTTP request via Chrome. Both calls should end up going through IIS…
Can you check the AppHost used in the tests you are running? Possible the tests running are standing up their own AppHost which might differ from the one you deploy to be hosted in IIS.
Can you also replicate these failing tests in a build server/CI environment?
Thank you for your reply. We use the testing framework in Visual Studio. BOMK, the tests are making calls to IIS. We literally just create a JsonServiceClient, give it an URL (localhost:80) that points to a service on IIS, make request and wait for a response. Funny thing is it only fails on one API call, works fine on 100’s of others. This one API call just “happens” to be the call we had a struggle with the ServiceStack Json serializer and had to resort to Newtonsoft. We are currently only running unit tests in CI. We are not standing up servers just yet for “integration tests” however, I will run this test on my machine and point it at another server. That is a good idea. It “smells” like Studio/Testing is doing something magic with IIS since its local and pointing it at another server would take that out of the equation.
So this error is back. We had a work around in place knowing that before production we would have to develop a production fix. We can reproduce this using IIS or self-hosting. The new stack trace again shows a type being serialized that is not intended to be serialized. Its the cache object for Lead we had to extend.
The problem is a combination of ServiceStack/funq and Lead. We had no problem with Lead for years while using ASP.net/Autofaq in our old server. We had no problems with ServiceStack in our new server until we added Lead. The problem only presents itself when we visit methods that use API’s from both SDKs.
The issue is trying to serialize IOC’s internals, which is never attempted in ServiceStack. So the proper fix would be to find out what’s triggering the serialization so you can look at preventing that from happening.
The last item in the StackTrace uses CrossAppDomainSink which I’ve never heard or can find any information of, but sounds like it’s doing some cross AppDomain serialization. Maybe the runtime is doing this behind the scenes during an async hop. Are you using any Request Scoped dependencies? If you are can you change them to Transient (or Singleton if they’re ThreadSafe).
Otherwise can you try catching the exception to go to see what’s at the top of the callstack to hopefully identify what’s causing it.
