ServiceStack.Discovery.Consul RFC

Hey Scott, I’m looking to formalize the API’s for calling both Internal and External Services with the new IServiceGateway interfaces which I’m going to start recommending everyone use to call other Services in future as they can be easily substituted and integrated with Discovery Services like Consul, so I’m also hoping you could add support for it in your Consul plugin.

It wont take much effort to support as the interfaces are already implemented in all Service Clients as as well as the new InProcessServiceGateway so your plugin would only need to alternate between the two - I show an example of this at the bottom of this post.

The InProcessServiceGateway is the default implementation of base.Gateway in ServiceStack’s base Service class which can now be used instead of ResolveService<T>() or ExecuteRequest() to call another Service, where you can execute a Service with just a Request DTO, e.g:

public class MyServices : Service
{
    public object Any(Request request)
    {
        return Gateway.Send(request.ConvertTo<ExternalRequest>());
    }
}

As I want to make calling Internal and External Services indistinguishable any Exceptions are automatically converted into WebServiceException - the same as would be thrown from a Service Client.

The IServiceGateway interfaces represent the minimal surface area required to support all of ServiceStack’s different calling conventions. GET, POST, etc requests can still be made by adding IGet, IPost, interface markers on the Request DTO’s and the ServiceGatewayExtensions.cs provide nicer convenience API’s when Request DTO’s implement IReturn<T> and IReturnVoid.

There’s both Sync:

public interface IServiceGateway
{
    /// Normal Request/Reply Services
    TResponse Send<TResponse>(object requestDto);

    /// Auto Batched Request/Reply Requests
    List<TResponse> SendAll<TResponse>(IEnumerable<object> requestDtos);

    /// OneWay Service
    void Publish(object requestDto);

    /// Auto Batched OneWay Requests
    void PublishAll(IEnumerable<object> requestDtos);
}

And Async versions:

public interface IServiceGatewayAsync
{
    Task<TResponse> SendAsync<TResponse>(object requestDto, 
        CancellationToken token = default(CancellationToken));

    Task<List<TResponse>> SendAllAsync<TResponse>(IEnumerable<object> requestDtos, 
        CancellationToken token = default(CancellationToken));

    Task PublishAsync(object requestDto, 
        CancellationToken token = default(CancellationToken));

    Task PublishAllAsync(IEnumerable<object> requestDtos, 
        CancellationToken token = default(CancellationToken));
}

With tests for testing each calling convention in both ServiceGatewayTests.cs and ServiceGatewayTestsAsync.cs which run the same under different scenarios:

This is the easiest to support where you can just register a ServiceClient against the IServiceGateway interface and it will use that for all base.Gateway requests:

public override void Configure(Container container)
{
    container.Register<IServiceGateway>(c => new JsonServiceClient(baseUrl));
}
  • MixedServiceGatewayTests - Uses InProc for Internal Services and Service Client for external ones. When you need to support internal and external calls you’ll need to implement the IServiceGatewayFactory instead as you’ll need to capture the current IRequest in order to make in-process calls:
public interface IServiceGatewayFactory
{
    IServiceGateway GetServiceGateway(IRequest request);
}

To make it easy for you to create a IServiceGatewayFactory for Consul I’ve created a ServiceGatewayFactoryBase base class which allows you to implement it with the least effort required, basically you just need to implement IServiceGateway GetGateway(Type requestType) as seen in MixedServiceGatewayNativeAsyncTests tests.

This example Factory uses a convention to determine which Services should be called internally or externally based on whether the Request DTO Type Name contains “External”:

class ServiceGatewayFactory : ServiceGatewayFactoryBase
{
    public override IServiceGateway GetGateway(Type requestType)
    {
        var gateway = requestType.Name.Contains("External")
            ? new JsonServiceClient(baseUrl)
            : (IServiceGateway)localGateway;
        return gateway;
    }
}

Whereas your Consul implementation would need to look up the Request DTO Type based on the registry.

To get ServiceStack to use your implementation your plugin would just need to register an IServiceGatewayFactory in ServiceStack’s IOC. As the factory captures the current IRequest it’s not thread-safe to use as a singleton and needs to be registered with ReuseScope.None so a new instance is returned each time, e.g:

class AppHost : AppSelfHostBase
{
    public AppHost() 
        : base("Mixed AppHost Tests" typeof(MyServices).Assembly) { }

    public override void Configure(Container container)
    {
        container.Register<IServiceGatewayFactory>(x => new ServiceGatewayFactory())
            .ReusedWithin(ReuseScope.None);
    }
}

The latest v4.0.55 on MyGet has these latest bits, I hope you can extend your Consul plugin to also register a Consul-aware IServiceGatewayFactory as seen above so users using this new API will be able to seamlessly adopt Consul discovery services when they want to decouple their application. Please let me know if you have any questions I can help with.

1 Like

This looks like it will fit with a plugin what I’m coding currently (tentatively namespaced ServiceStack.SimpleCloudControl). I’ve had many of the sub-plugins running solo for a while now. I am just finally getting everything in a single project so I can finally have more cluster-wide control of my various deployed services.

I had been looking at Consul, but i felt like it was too much extra stuff for our environment right now, and every service we have talks to a single redis endpoint already. So I have basic Redis-based discovery services working, as well as a shared PubSub channel for additional control plugins (caching policy, request logging policy, MQ control policy, AppSettings, etc.).
I’ll try wiring a IServiceGatewayFactory implementation and see if I can get a working sample up I could share. I’ve been hacking away with extension methods to call external services for a year or so, but this would be much cleaner.

1 Like

@rsafier Cool, a redis-backed discovery service could save an extra infrastructure process to manage for people who are already using Redis.

That was what I was thinking, its pretty lightweight, the core plugin is ~200 LOC.

I couldn’t find ServiceGatewayFactoryBase in the current nuget release (even after cleaning cache), but mimicked it, and everything just worked!

I need to do some more claenup/refining, but thinking this should cover the 100s of service-hosts / 10s of servers sized environment I typically deal without much fuss.
Nice work @mythz, the features just keep on coming :smile:

Looks good @mythz, I’ll take a look at updating the plugin in the next couple of days but it looks straightforward.

@rsafier We created the plugin as ServiceStack.Discovery.[Provider] as we thought others might implement against different providers (etcd, zookeeper etc), your Redis one sounds really interesting.

A few of the reasons we picked Consul was not necessarily it’s scalability (although it’s a plus) but it’s ability to run health checks against services (we hope to expand this feature to make it easier to just add delegates to the plugin config and have them autowired to an endpoint), the predictability and speed of round trip times for lookups using local agents across a loopback address (we aren’t interested in implementing gossip, elections, RAFT etc) and the round-trip times for auto selecting the closest/fastest route to a service.
We also have an implementation of IAppSettings using consul shared config running using their k/v store (still to release as a plugin).

As part of our future strategy we also want to be able to remotely manage service instances in a similar way to netflix’s simian army with conformity and janitor style services looping through consul registered services to interrogate servicestack metadata endpoints, and if required add failing health checks for them to deny them being returned to any clients, raise system notifications routed to service technical contacts etc, this way we can control or influence those services without those services needing to taking any additional dependencies other than service discovery or implement in each of them potentially fragile RPC style interfaces.

2 Likes

Just having a look at this and it raises a question for me. If the apphost knows all of its own operations and requestDTO types when it builds the metadata, why not check this in the base factory first and just provide a single method for the external resolver. It would simplify the process to avoid naming conventions and/or unnecessary lookups?

One additional comment unrelated to the above change.
If I wanted to automatically add health checks (a simple open connection test probably) for all registered db connections via IDbConnectionFactory or OrmLiteConnectionFactory, the registered names in the latter are not exposed and there isn’t anything like ‘State’ in the former’s interface. Any suggestions?

That application logic is not in the base class as it’s the purpose of the subclass to workout which IServiceGateway should be returned. You may not want to use the localGateway directly and return a decorated IServiceGateway with logging or any other app logic attached instead.

Anyway it’s just a simple check to workout whether the Request DTO is available locally, i.e:

var isLocal = HostContext.Metadata.RequestTypes.Contains(requestType)

That’s the changes up now @mythz

1 Like

Awesome, thx Scott! - it will serve as a perfect example of what’s enabled with the new ServiceGateway in the next release.

1 Like

no probs. should I release the updated package to nuget?
It has min dependency version of .55 set so shouldn’t auto-update existing clients…but I’m never 100% confident in nuget’s behaviour; or is it better to wait until you release .55? Don’t mind either way.

lastly, as before, happy to get any feedback (good or bad), PR’s etc from the community. We don’t have any contrib guidelines on our repo’s yet but it’s on the list!

No better not yet, hoping to publish v4.0.56 early next week - will ping you here just after I do.

I’m refactoring now to go with that pattern for the basic discovery component as it makes sense.

I cut the fat out of what I had so its just the core functionality. If either of you have suggestions I’m all ears.

https://gist.github.com/rsafier/4d28e484904404219805f11b56e89373

2 Likes

Is there any way I get get a list of the registered db connections in the factory, seems like you can only register and recall a connection using a name string?

OrmLiteConnectionFactory.NamedConnections maintains a dictionary of OrmLiteConnectionFactory - you can inspect the DB connection string from OrmLiteConnectionFactory.ConnectionString.

Haven’t gotten round to wiring up the auto health checks against any registered IDBConnections yet but forgot to say thanks for the pointer, looks like it’ll do the job.

On an unrelated note, I completely forgot (and it isn’t documented anywhere yet) but our discovery solution now supports DNS SRV DTO level discovery.

You can issue a DNS SRV Query against the Consul agent’s DNS service (on port 8600 by default):

C:\>dig @127.0.0.1 -p 8600 echoa.api.service.consul. SRV

; <<>> DiG 9.10.2-P4 <<>> @127.0.0.1 -p 8600 echoa.api.service.consul. SRV
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 42770
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;echoa.api.service.consul.      IN      SRV

;; ANSWER SECTION:
echoa.api.service.consul. 0     IN      SRV     1 1 8091 m.node.dc1.consul.

;; ADDITIONAL SECTION:
m.node.dc1.consul. 0     IN      CNAME   http://127.0.0.1:8091/.

;; Query time: 0 msec
;; SERVER: 127.0.0.1#8600(127.0.0.1)
;; WHEN: Fri May 20 19:19:16 GMT Daylight Time 2016
;; MSG SIZE  rcvd: 170

Pretty sweet huh!

This should ensure pretty consistent sub 1ms lookups (which are already pretty quick over the consul http api)

Sadly, the built in DNS support for issuing and parsing SRV queries in .NET is non-existent so I haven’t had time to figure out the interop calls to hook up support for this in the plugin. It’s on the list though.

1 Like

With the latest commit, I’ve cleaned up the discovery interface and refactored out any type dependencies to the plugin.

Is there any value in adding this interface to ServiceStack.Interfaces as it could avoid some of our other discovery-related (but not consul specific) plugins taking a direct dependency on our consul one?

It doesn’t really affect us as we are using consul anyway but thought I’d suggest it.

I don’t think so, it seems more like an internal interface than an external user-facing interface with multiple providers.

I’d say you’re better off maintaining your own dep-free Interfaces/abstactions package that everything depends off like we do with ServiceStack.Interfaces and .NET Core does with all their *.Abstractions packages.