ServiceStack.Discovery.Redis RFC

rsafier · April 16, 2016, 9:05pm

I’ve created a plugin which gives basic service discovery abilities to any ServiceStack instance that has access to Redis, more details in the README of repo.

Source can be found @ https://github.com/rsafier/ServiceStack.Discovery.Redis

If there is interest I will look at getting a NuGet package setup.

mythz · April 16, 2016, 9:13pm

Awesome thx Richard - another provider to add to the Release Notes! Yeah I think a NuGet package would greatly ease the effort to adoption. I’d also include the necessary instructions for downloading/enabling the plugin, e.g:

Plugins.Add(new RedisServiceDiscoveryFeature());

I’d also include an example of using the base.Gateway and mention that if the Request DTO is implemented internally it will call that otherwise it will query Redis to determine which external Service it should call using a JsonServiceClient instead.

Mac · April 16, 2016, 9:50pm

Sounds good

Setting up a nuget package is fairly easy.

I use an appveyor account linked to my github account to build the repo on commits and it creates a nuget package from a nuspec file, then you just need an nuget api for running a deployment to nuget.org.

If you need any help with it, let me know.

rsafier · April 16, 2016, 10:07pm

Sounds good. I just committed after I pulled it out of my ServiceStack.SimpleCloudControl project (which I’ll also be releasing later, it sits on top of this discovery layer) and did a quick set of sample services.

I’ll improve some of the documentation, plus it is also missing an ExcludeServiceDiscoveryAttribute which I know I’ll need in my production environment (we “proxy” a bunch of services which reuse the DTOs of the internal services they are calling, in these cases we want to make sure we attribute the “gateway” services so they are not picked up as valid entry points).

I’ll take a look at AppVeyor, I have had NuGet packaging on the list of things to explore for a while now, just haven’t had the need until now.
Thanks!

rsafier · April 16, 2016, 11:28pm

That was fairly simple NuGet Gallery | ServiceStack.Discovery.Redis 1.0.0

Mac · April 17, 2016, 10:11am

Looks really good, SCC project too.

One question, why use ExcludeServiceDiscovery over the existing ExcludeAttribute or RestrictAttribute?

It couples the plugin code to the request dto’s so perhaps an additional enum value could be added to one of the above enums so that both plugins can use

[Exclude(Metadata.ServiceDiscovery)] or [Restrict(VisibilityTo = RequestAttributes.ServiceDiscovery)]

rsafier · April 17, 2016, 3:55pm

I need an alternative flag because we have a sisuation were I have internal service A, it is called via a “proxy” service on the DMZ which uses the exact same DTO as the internal service (same typename), on the proxy service I would want to mark those DTOs as not discoverable on that node.

I was figuring because my existing implementation is a stand alone attribute I can easily have a a process that at AppHost init can dynamically attribute the DTOs that should be ignored for discovery. For my DMZed proxy situation it would be easiest to just have a method which marks all the exposed services before startup.

I’m all for either of these suggested alternatives, but I’m not sure if I can “append” to existing attributes or how that would work?

I’m going to add a public HashSet<Type> ExcludedTypes property on the feature. I’ll wait to hear what opinion @mythz might have on the attribute situation. I do think a global policy would be best.

mythz · April 18, 2016, 7:48am

@rsafier hey sorry for the delay, been tied up working on the release notes. I’ve added Feature.ServiceDiscovery in this commit.

So you can now use it with the [Exclude] attribute, e.g:

[Exclude(Feature.ServiceDiscovery)]

This change is now available from v4.0.55 on MyGet but I’m planning for a v4.0.56 Release on NuGet on Tuesday morning in-case you prefer to wait till then.

rsafier · April 18, 2016, 2:36pm

Made the changes and pushed out to NuGet

Mac · April 18, 2016, 8:44pm

Hey @rsafier, I’ve updated the exclusions

One thing I came across which you might want to update is when filtering the metadatatypes which I tidied up into (tested!) extensions methods you might find useful. I also added support for the RestrictAttribute

https://github.com/MacLeanElectrical/servicestack-discovery-consul/blob/master/src/Servicestack.Discovery.Consul/Discovery/DefaultDiscoveryRequestTypeResolver.cs#L22

One question that the RestrictAttribute use raised is how this would work with a couple of scenarios

when a request is restricted to secure but is registering only as http
when requests are restricted to certain formats (xml,csv) etc how to create the correct IServiceGateway client for the request. I’m toying with the idea of creating variable prefixes on the dto tags to denote the format with json being the preferred and default.

@mythz I swear it’s a coincidence but that PR merge puts the nuget package version as 4.0.56!

mythz · April 18, 2016, 9:05pm

It’s-a-coming - release notes are especially long this time around - But stay tuned for a NuGet drop tomorrow.

rsafier · April 19, 2016, 12:08am

I’m not sure about [Restrict(RequestAttributes.External)] being ignored by default, I think I was just coding without thinking about it. I’m fairly certain there are plenty of times one would want a DTO discover-able internally but its restricted external.

I am just going to provide a Func<,> FilterTypes that the user can optionally fully filter if required just before building my final cached list.

You do bring up good points about needing an alternative format, HTTPS, etc:
I think it can be done now that I pass the requestType as well to the SetServiceGateway delegate here

At least for the moment this isn’t going to present an issue for me, but I will add some notes about the limitations/assumptions of the current implementation so others might not expect to get too creative with this initial release.

rsafier · April 19, 2016, 7:25am

Insomnia can be productive. Put in basic support to detect proper service client (Json, Jsv, Xml preference order), also will adjust baseUrl if it thinks it must be secure.

github.com

rsafier/ServiceStack.Discovery.Redis/blob/ffab83d512faf5b4eed2582e8cf21f58fc208555/src/ServiceStack.Discovery.Redis/TypeExtensions.cs#L45


            return nativeTypes == null


                ? types
                : types
                    .Where(x => !nativeTypes.MetadataTypesConfig.IgnoreTypes.Contains(x))
                    .Where(x => !nativeTypes.MetadataTypesConfig.IgnoreTypesInNamespaces.Contains(x.Namespace));
        }
         
        public static bool HasXmlClientSupport(this Type type) => !type.AllAttributes<ExcludeAttribute>().Any(t => t.Feature.HasFlag(Feature.Xml)) &&
                                                                  type.AllAttributes<RestrictAttribute>().All(t => t.HasAccessTo(RequestAttributes.Xml)); 
        public static bool HasJsvClientSupport(this Type type) => !type.AllAttributes<ExcludeAttribute>().Any(t => t.Feature.HasFlag(Feature.Jsv)) &&
                                                                  type.AllAttributes<RestrictAttribute>().All(t => t.HasAccessTo(RequestAttributes.Jsv));
        public static bool HasJsonClientSupport(this Type type) => !type.AllAttributes<ExcludeAttribute>().Any(t => t.Feature.HasFlag(Feature.Json)) &&
                                                                  type.AllAttributes<RestrictAttribute>().All(t => t.HasAccessTo(RequestAttributes.Json)); 
    }
}

github.com

rsafier/ServiceStack.Discovery.Redis/blob/ffab83d512faf5b4eed2582e8cf21f58fc208555/src/ServiceStack.Discovery.Redis/RedisServiceDiscoveryFeature.cs#L371


    var feature = HostContext.GetPlugin<RedisServiceDiscoveryFeature>();
    
    if (!feature.NeverRunViaLocalGateway.Contains(requestType) && HostContext.Metadata.RequestTypes.Contains(requestType))
        return localGateway;
    var baseUrl = feature.ResolveBaseUrl(requestType);
    if (baseUrl.IsEmpty())
        throw new RedisServiceDiscoveryGatewayException("Cannot resolve request type to local or remote service endpoint.");
    return feature.SetServiceGateway != null ? feature.SetServiceGateway(baseUrl, requestType) : ResolveBestFitClient(baseUrl, requestType);
}


public static IServiceGateway ResolveBestFitClient(string baseUrl, Type requestType)
{
    var restricts = requestType.AllAttributes<RestrictAttribute>();
    var inSecureAllowed = restricts.All(t => t.HasAccessTo(RequestAttributes.InSecure)) || restricts.All(t => !t.HasAccessTo(RequestAttributes.Secure));
    if (!inSecureAllowed && baseUrl.StartsWith("http", StringComparison.InvariantCultureIgnoreCase))
    {
        baseUrl = baseUrl.Insert(4, "s"); //tack in the secure if required and not in base listening url
    }
    IServiceGateway gateway = null;
    if (requestType.HasJsonClientSupport())
    {

Mac · April 19, 2016, 8:41am

good point, will probably go with the Func<,>

Will have a look later at the service client detection

mythz · April 19, 2016, 3:22pm

@Mac @rsafier FYI v4.0.56 is now released! https://github.com/ServiceStack/ServiceStack/blob/master/docs/2016/v4.0.56.md

Thanks for your contributions on this

Mac · April 19, 2016, 3:42pm

Well done on latest big release!
Updated nuget package.

One correction to the readme, you have ServiceStack.EventStore Install-Package command is incorrect

mythz · April 19, 2016, 3:43pm

whoops copy+paste fail, fixed now!