Reflection.Emit

Johan · August 1, 2018, 7:54am

Hi all,

I currently have an cross platform Xamarin Forms (Android , iOS) and native UWP app , using .net standard 2 as my base line. I can no longer compile my UWP app in release mode as Servicestack uses Reflection.Emit and this is a known “new issue” in UWP compilation and throwing an error. Looking at the issue list in github I can see you guys are also not happy about it and I was hoping MS can roll out a fix in the past 2 months. However it does not look like the fix is forthcoming until .net standard VNext leaving me stranded with ServiceStack that can’t compile. As SS and .net standard 2.0 are now fairly deeply rooted in my current platform I dont really want to go back to PCL implementation.

Any ideas ??

mythz · August 1, 2018, 9:31am

Removing the dependency to Reflection.Emit from ServiceStack.Text is not an option, but it wont be used at runtime if Env.SupportsEmit=false; but we can’t do anything if it’s a compile-time error.

But compile time errors should be a bug as UWP is meant to support .NET Standard 2.0 and Reflection.Emit has a .NET Standard 1.1 build so there shouldn’t be a compile-time error.

I’ve tried creating a new UWP App and added a reference to v5.1.0 ServiceStack.Client which works at as expected in Debug mode and it does build in Release mode unfortunately it fails at runtime with:

Method 'DynamicMethod..ctor(string, Type, Type[], Module, bool)' from assembly 'System.Reflection.Emit.Lightweight' was not included in compilation, but was referenced in ReflectionExtensions.GetConstructorMethodToCache(Type). There may have been a missing assembly, or a dependency on a more recent Windows SDK release.

Which is surprising behavior as it the Exception sounds like the method doesn’t exist even though it does and runs fine in Debug mode. The effect of this looks like just having a method reference existing but not calling it will still cause a runtime exception which leaves very few options.

I’ve also tried adding rules in UWP’s Default.rd.xml file to avoid any optimizations with:

<Assembly Name="*ServiceStack.Text*" Dynamic="Required All" DoNotInline="true" DoNotOptimize="true"/>
<Assembly Name="*ServiceStack.Client*" Dynamic="Required All" DoNotInline="true" DoNotOptimize="true"/>
<Assembly Name="*ServiceStack.Interfaces*" Dynamic="Required All" DoNotInline="true" DoNotOptimize="true"/>

But that doesn’t help either.

So I’m out of solutions. Creating fork of ServiceStack.Text without any references to Reflection.Emit could work but that’s not an option for us so we’re back to waiting for UWP to add better support for UWP, at least if it had empty stubs that throws a runtime PlatformNotSupportedException will help as we can handle it at rutnime where it shouldn’t fail if you’re not calling unsupported methods.

Johan · August 1, 2018, 9:50am

Hi mythz ,

I was hoping mult-targing of .net standard libs were an option. Is this issue only related to .net standard or does it also include .net framework ?

mythz · August 1, 2018, 10:02am

I think its an issue with UWP’s .NET Native AOT compiler, .NET Standard is just the API contract so the surface area should exist everywhere on all platforms, but for some reason the method reference seems to have gotten removed under UWP .NET Native.

Johan · August 1, 2018, 10:14am

Thanks, rather confusing. Reading the following thread I assumed it was working under .net core + .net framework.

github.com/castleproject/Core

System.Reflection.Emit package has been delisted

opened 03:08PM - 13 Jun 18 UTC

closed 07:09AM - 15 Jul 19 UTC

jonorossi

discussion

See https://github.com/dotnet/corefx/issues/29365#issuecomment-385102487. Acc…ording to that thread the package was never cleanly implemented as a .NET Standard 1.x package and had a hack that it could see the internals of the runtime's classes, it was then recently decided for .NET Standard 2.0 not to continue that hack (because CoreRT doesn't have the hack). No .NET Standard 2.x package was published and recently on the 2018-04-28 `System.Reflection.Emit` and `System.Reflection.Emit.Lightweight` were delisted on NuGet.org. That now leaves us [using a delisted package while targeting .NET Standard 1.3 and 1.5](https://github.com/castleproject/Core/blob/7ef5e620693e9548dbc0246adc91c3f9ba734b3d/src/Castle.Core/Castle.Core.csproj#L35-L45). We need to follow this thread, however it likely means we'll need to drop our .NET Standard 1.3 and 1.5 targets and move to `netcoreappx.x` (.NET Core) targets, obviously resulting in our downstream users doing the same. The .NET Framework targets (`net35;net40;net45`) wouldn't change. @stakx is this your understanding?

Sorry for all the questions. If the problem is the AOT compiler , will older versions of UniversalWindows then work ? I’m trying to figure out if there is a point in time that I can reference.

mythz · August 1, 2018, 10:16am

Yeah should work before .NET Native since it works fine in Debug mode and Release is normally just an optimized build whereas AOT uses a different compiler / toolchain.

Johan · August 3, 2018, 6:58pm

Hi @mythz,

For me to continue with UWP I’m thinking about taking out SS altogether on all my clients and replacing it with https://flurl.io/

Server side will stay as is with SS running in .net core. I’m currently using JWT so it should work with generic rest client.

The only problem that I can see is Server Send Events where I don’t have a client that will work.

Would appreciate your thoughts and suggestions on above.

mythz · August 3, 2018, 7:04pm

The source code for the ServerEventsClient is available in these 2 classes:

You can try taking a modified copy and changing it to use your different HTTP Client and JSON serializer you want to use.

Johan · August 3, 2018, 7:47pm

Thanks @myth , I think easiest is probably just decoupling SSE on server and use SignalR for real-time data than me trying to hack together “custom code” for SSE ;)) damm MS

mythz · August 4, 2018, 11:32am

I’ve ended up refactoring out the Reflection.Emit dependencies from ServiceStack.Text’s .NET Standard 2.0 build into platform-specific .NET Core 2.1 and .NET v4.5 builds in this commit.

This change is available from the latest v5.1.1 which is now available on MyGet.

With this change the Client.UWP App in HelloMobile project now works in Release build, but Xamarin iOS/Android/Forms has an issue with the new efficient Span<T> types which has only been resolved from VS.NET 15.8 Preview 3+:

So you would need to wait for the next v15.8 release or use the latest VS.NET preview to use ServiceStack’s Client v5.1.1 libraries with Xamarin.Forms.

Johan · August 4, 2018, 12:12pm

Hi @myth, you are the man What can I say , thank you very much. I’m already on latest VS15.8 preview so that’s not an issue. You just saved me a lot of pain, IMHO reading between the lines online, is that Reflection.Emit is not worth pursuing going forward and although they potentially planning to include it in .net standard Vnext it looks like it’s going to come with special build config to make it work. Too much friction if you ask me.

github.com/dotnet/runtime

Assembly.ReflectionOnlyLoad replacement - TypeInfo and family implementation over System.Reflection.Metadata

opened 04:07AM - 14 Aug 15 UTC

closed 09:33AM - 31 Jan 20 UTC

davidfowl

enhancement api-suggestion area-System.Reflection

### Proposed Surface Area ```c# namespace System.Reflection { public abs…tract class MetadataAssemblyResolver { public abstract Assembly Resolve(MetadataLoadContext context, AssemblyName assemblyName = null); } public class PathAssemblyResolver : MetadataAssemblyResolver { public PathAssemblyResolver(IEnumerable<string> assemblyPaths); public override Assembly Resolve(MetadataLoadContext context, AssemblyName assemblyName); } public sealed partial class MetadataLoadContext : System.IDisposable { public MetadataLoadContext(MetadataAssemblyResolver resolver, string coreAssemblyName = null); public Assembly CoreAssembly { get; } public void Dispose(); public System.Collections.Generic.IEnumerable<Assembly> GetAssemblies(); public Assembly LoadFromAssemblyName(AssemblyName assemblyName); public Assembly LoadFromAssemblyName(string assemblyName); public Assembly LoadFromAssemblyPath(string assemblyPath); public Assembly LoadFromByteArray(byte[] assembly); public Assembly LoadFromStream(System.IO.Stream assembly); } } ``` ### Scenario: Inspecting IL metadata assemblies in custom directories. ```C# var mscorlibpath = @"D:\temp\myselfcontainedapp\mscorlib.dll"; var mydllpath = @"D:\temp\myselfcontainedapp\myap.dll"; var resolver = new PathAssemblyResolver(new string[] {mscorlibpath, mydllpath}); using (var context = new MetadataLoadContext(resolver, "mscorlib")) { Assembly a = context.LoadFromAssemblyPath(@"D:\temp\myselfcontainedapp\myapp.dll"); var type = a.GetType("MyApp.MyClass", throwOnError: true); } ``` This creates a load context with a policy to probe for external assemblies in a specific directory. An external assembly is loaded on demand when there is a type reference to that assembly. `MetadataLoadContext` implements `IDisposable`. Disposing releases any file locks that may be held on the underlying assembly files. Once disposed, any Assembly objects dispensed by the MetadataLoadContext (and any Reflection objects dispensed by those Assemblies) must no longer be called. Their behavior is undefined. (We'll make a good faith effort to throw `ObjectDisposedExceptions` but some properties may returned cached data. The primary goal is to avoid the really anti-social behavior such as accessing freed native memory.) You determine the bind algorithm by selecting a pre-existing resolver (MetadataAssemblyResolver-derived class) or writing you own MetadataAssemblyResolver-derived class and overriding the `Resolve` method. The Resolve method should return either `null` or an `Assembly` object dispensed by one of the `Load` methods on the `MetadataLoadContext` that was passed to the Resolve method. The provided DirectoryAssemblyResolver policy probes each directory in order to find a given assembly by name and stops once the first provided assembly file is found. It does not compare assembly versions or try to find the best matching assembly (if the assembly exists in more than one location). It appends ".dll" to the assembly name in order to find the file. If no assembly is found, a FileNotFoundExcepton is raised. DirectoryAssemblyResolver also sets the context.CoreAssemblyName property to "mscorlib", but that can be changed by the consumer before the first access to the core assembly is requested; the first access to the core assembly occurs by a reference to primitive type such as System.Object or System.Int32. ### Scenario: Inspecting IL metadata assemblies using the runtime loader. In .NET Framework, a common pattern is creating a new AppDomain (using the existing AppDomainSetup) just to load a single assembly, inspect its types and custom attributes, then unload in order to free the memory. We've looked into the idea of allowing runtime-supplied Assemblies to be injected into the TypeLoader but this is a snake pit. For this idea to work at all, the cooperation needs to work both ways and runtime Assemblies aren't designed to interoperate with externals. In particular, Type.MakeGenericType() is not copacetic with instantiating runtime types on non-runtime type arguments. A possible midway solution is for TypeLoader (or the calling app) to reload the runtime assembly (from it's Location property) as a TypeLoader-supplied assembly. This allows users to use the binding algorithm of the current runtime if that's what they want. But it does mean "loading the assemblies twice" so there is a perf hit. Currently there is no MetadataAssemblyResolver that supports this policy, but it would not be difficult to create. ### Earlier Proposal and Notes ```C# namespace System.Reflection { public sealed class TypeLoader : IDisposable { public TypeLoader(); public TypeLoader(string coreAssemblyName); public Assembly LoadFromAssemblyPath(string assemblyPath); public Assembly LoadFromByteArray(byte[] assembly); public Assembly LoadFromStream(Stream assembly); public Assembly LoadFromAssemblyName(AssemblyName assemblyName); public Assembly LoadFromAssemblyName(string assemblyName); public string CoreAssemblyName { get; set; } public event Func<TypeLoader, AssemblyName, Assembly> Resolving; public IEnumerable<Assembly> GetAssemblies(); public void Dispose(); } } ``` #### Scenario : Legacy ReflectionOnlyLoad emulation ``` Assembly a = TypeLoader.Default.Load("Foo, Version=1.2.3.4, PublicKeyToken=xxxxxxxxxx"); ``` The static `Default` property returns a global `TypeLoader` that serves as a (compatible within reason) replacement for `Assembly.ReflectionOnlyLoad`. As with `ReflectionOnlyLoad`, you must preload all dependencies (other than mscorlib) either in advance or on-demand by subscribing to the `AppDomain.ReflectionOnlyAssemblyResolve` event. This approach is not recommended for new code. It's for porting old code only. #### Issue - Throw FileLoadException or FileNotFoundException when the Resolving event handler returns null? ReflectionOnlyLoad (the feature we're officially replacing) throws `FileLoadException`. AssemblyLoadContext (the new feature) throws `FileNotFoundException.` The former is more compatible, the latter is more logical. #### Issue - Semantics of LoadFromAssemblyName() - be like ReflectionOnlyLoad or like AssemblyLoadContext? On the surface, `LoadFromAssemblyName()` looks like the equivalent of `ReflectionOnlyLoad(AssemblyName)`. But in fact, it's not. `Assembly.ReflectionOnlyLoad(AssemblyName)` only returns an Assembly that had already been loaded via a prior operation. If no assembly matching the name had been loaded, it throws `FileNotFoundException` (unlike when it fails a dependency load, which throws `FileLoadException`.) It does not invoke the `ReflectionOnlyAssemblyResolve` event. So the name is misleading since it never _loads_ an assembly. It only fetches a previously loaded Assembly. I'm not sure what the use case for it is - if it return `null` rather than throwing, it could be seen as the `TryGet` api but given that it throws without giving the resolve event handler a chance to bind, what is the rationale for this behavior? `AssemblyLoadContext.LoadFromAssemblyName()`, on the other hand, works much more like I'd expect. It triggers the `Load` overload and the event handler as needed. It's basically a manual way of triggering the same process that happens when the runtime internally resolves an assembly reference. I'd rather adopt `AssemblyLoadContext`'s behavior here as we're adopting the api name and the behavior is less surprising. On the other hand, we're supposed to be replacing `ReflectionOnlyLoad`. I see three choices: 1. Adopt the `AssemblyLoadContext` behavior. 2. Adopt the `ReflectionOnly` behavior. 3. Have both `Load` and `LoadFromAssemblyName` overloads.

Johan · August 4, 2018, 2:19pm

I’m seeing the following error with V5.1.1 in release mode (UWP)

Microsoft SDKs\UWPNuGetPackages\microsoft.net.native.compiler\2.1.8\tools\Microsoft.NetNative.targets(788,5): warning : MCG : warning MCG0007: Unresolved P/Invoke method ‘kernel32.dll!GetCurrentPackageFullName’ for method ‘System.Int32 ServiceStack.Text.Env.GetCurrentPackageFullName(System.Int32, System.Text.StringBuilder)’. Calling this method would throw exception at runtime. Please make sure the P/Invoke either points to a Windows API allowed in UWP applications, or a native DLL that is part of the package. If for some reason your P/Invoke does not satisfy those requirements, please use [DllImport(ExactSpelling=true) to indicate that you understand the implications of using non-UWP APIs.

C:\Program Files (x86)\Microsoft SDKs\UWPNuGetPackages\microsoft.net.native.compiler\2.1.8\tools\Microsoft.NetNative.targets(788,5): error MSB6006: “ilc.exe” exited with code -1073741819.

mythz · August 4, 2018, 8:02pm

Sigh, looks like we’re back to not being able to reliably determine at runtime if its running in UWP, I’ve refactored it to use some heuristics that should work with the latest version of UWP in this commit.

This change is available in the latest v5.1.1 on MyGet.

Johan · August 29, 2018, 7:38am

Thanks @mythz. Finally got things working again after numerous third party libs that uses Reflection.Emit. Was a hugely painful exercise.

BTW plse update servicestack website(home page) to reflect latest twitter feed, I see the website is out of sync.

mythz · August 29, 2018, 3:44pm

There is no twitter feed on the website, just a collection of curated favourited tweets.

Johan · August 29, 2018, 7:15pm

Cool , just thought you might want to update that. It’s like reading an old newspaper

mythz · August 29, 2018, 7:18pm

It’s for public quotes people have said (since we don’t ask Customers for marketing quotes), it’s not a Twitter feed for news, for that subscribe to the @ServiceStack Account:

https://twitter.com/ServiceStack