Issues with MQ upgrading from ServiceStack 6 to 8

ServiceStack Rabbit MQ
1

I’m coming back to a project after years of being away, trying to get all the dependencies updated (including ServiceStack).

So far everything has worked fine, except our MQ system. Locally we run RabbitMQ with Docker, in the cloud we run AWS SQS. This post is specifically for local (RabbitMQ). I haven’t gotten to AWS SQS yet.

Our project setup runs something like this:

SS API → We mostly queue messages from here to go to SS Console App, and receive one type of message update from the SS Console App.
SS Console App (MQ Host) → This is the service that processes the bulk of commands queued from the SS API project.

What seems to be happening is that as soon as I queue a message from the API, the RabbitMqServer.RunLoop enters into the WorkerOperation.Stop. The only errors I’m getting in my seq are:

System.NullReferenceException: Object reference not set to an instance of an object.
   at ServiceStack.ServiceStackHost.ApplyRequestConvertersAsync(IRequest req, Object requestDto) in /home/runner/work/ServiceStack/ServiceStack/ServiceStack/src/ServiceStack/ServiceStackHost.Runtime.cs:line 45
   at ServiceStack.Host.ServiceController.ExecuteMessage(IMessage dto, IRequest req) in /home/runner/work/ServiceStack/ServiceStack/ServiceStack/src/ServiceStack/Host/ServiceController.cs:line 604
   at ServiceStack.ServiceStackHost.ExecuteMessage(IMessage mqMessage) in /home/runner/work/ServiceStack/ServiceStack/ServiceStack/src/ServiceStack/ServiceStackHost.cs:line 1787
   at ServiceStack.Messaging.MessageHandler`1.ProcessMessage(IMessageQueueClient mqClient, IMessage`1 message) in /home/runner/work/ServiceStack/ServiceStack/ServiceStack/src/ServiceStack/Messaging/MessageHandler.cs:line 151

And additionally:

System.NullReferenceException: Object reference not set to an instance of an object.
   at RabbitMQ.Client.Impl.AutorecoveringModel.Abort()
   at RabbitMQ.Client.Impl.AutorecoveringModel.Dispose(Boolean disposing)
   at RabbitMQ.Client.Impl.AutorecoveringModel.System.IDisposable.Dispose()
   at ServiceStack.RabbitMq.RabbitMqProducer.Dispose() in /home/runner/work/ServiceStack/ServiceStack/ServiceStack/src/ServiceStack.RabbitMq/RabbitMqProducer.cs:line 202

Relevant code:
From SS API Init:

container.Register<IMessageService>(c =>
                {
                    var rabbitMqServer = new RabbitMqServer(mqConfig.HostAndPort, mqConfig.Username, mqConfig.Password);
                    container.Register<ITempQueueCreator>(new RabbitMqTempQueueCreator(rabbitMqServer.MessageFactory));
                    return rabbitMqServer;
                });
var messageService = container.Resolve<IMessageService>();
messageService.RegisterHandler<InvestigationUpdate>(ExecuteMessage);
logger.Debug("Starting messaging service");
messageService.Start();

From SS Console App:

container.Register<IMessageService>(c =>
                    new RabbitMqServer(mqConfig.HostAndPort, mqConfig.Username, mqConfig.Password)
                    {
                        PublishMessageFilter = (queue, props, message) =>
                        {
                            logger.Info($"Publishing message {message.Body} to {queue}.");
                        },
                        CreateQueueFilter = (queue, dict) =>
                        {
                            logger.Info($"Creating queue {queue} with properties {dict}");
                        }
                    });
// register handlers ....
var mqServer = Container.Resolve<IMessageService>();
mqServer.Start();
2

It’s likely trying to process the messages before the AppHost has finished initializing. If you’re not configuring RabbitMQ with a Startup configuration like Configure.Mq.cs:

[assembly: HostingStartup(typeof(MyApp.ConfigureMq))]

public class ConfigureMq : IHostingStartup
{
    public void Configure(IWebHostBuilder builder) => builder
        .ConfigureServices((context, services) => {
            services.AddSingleton<IMessageService>(c => 
                new RabbitMqServer(context.Configuration.GetConnectionString("RabbitMq") ?? "localhost:5672"));
        })
        .ConfigureAppHost(afterAppHostInit: appHost => {
            appHost.Resolve<IMessageService>().Start();
        });
}

Then can you start the AppHost with an After AppHost Init callback, e.g:

AfterInitCallbacks.Add(host => mqServer.Start());