Not understanding the difference between inq vs outq

#1

I’m just now starting to play around with the messaging feature. Although I understand how messaging works outside the realm of ServiceStack, I’m struggling to understand it within the context of ServiceStack. Outside of ServiceStack, you simply publish a message to a single queue and have a listener service monitor that queue and process the message. Why do we have two queues (inq, outq) in ServiceStack?

My first question is, what is the purpose of the inq and outq? Although the documentation points these two out, it doesn’t make clear what they are actually for.

For example, I have a publisher ServiceStack service which simply sends a Hello message to an SQS queue. Following the example on the website produces a single entry in the inq and an ever expanding list of messages to the outq. Why? I then have a listener ServiceStack service which should listen for messages in the SQS queue to process. Would this be the inq or the outq?

Any clarification on this topic would be appreciated.

#2

The Message Workflow in the Messaging docs explains its behavior:

Messages with no responses are sent to ‘.outq’ Topic

When a handler returns a null response, the incoming message is re-published as a
“transient” message to the out queue, e.g: mq:Hello.outq having the effect of notifying
any subscribers to mq:Hello.outq each time a message is processed.

We can use this behavior to block until a message gets processed with:

IMessage<Hello> msgCopy = mqClient.Get<Hello>(QueueNames<Hello>.Out);
mqClient.Ack(msgCopy);
msgCopy.GetBody().Name //= World

Also shown in this example is an explicit Ack (which should be done for each message you receive) to tell the MQ Broker that you’ve taken responsibility of the message so it can safely remove it off the queue.

It’s a way to be able to get notified that a one-way MQ message has been processed. It’s a common pattern in pub/sub systems for decoupled processes to listen to notification topics to trigger their evented workflows. When the MQ Server supports Pub/Sub topics the .outq is a transient topic that notifies multiple subscribers that are subscribed to the topic, when it’s not available it’s just a transient MQ. If you’re not using it you can disable messages being sent to the .outq when registering the MQ Server with DisablePublishingToOutq=true or only allow a whitelist of outq messages using PublishToOutqWhitelist.

The .inq is the actual queue for the message.

#3

I guess the part I’m struggling with is: “Messages with no responses are sent to ‘.outq’ Topic”

Does this mean a response within the publish event? Since I’m doing a fire and forget, wouldn’t this always be the case? For example, from the publisher service, I don’t care if and when the message has been processed by the processor service. So for example I’m just calling PublishMessage(new Hello(){Name=“Joe”}) and want to move on. Then, any processor service that happens to pick up that message will process it and remove from the queue.

But when I tried this, a single message was published to the .inq and the .outq seemed to be continuously growing until I killed the service. This is where I’m getting really confused. I would expect a single message to be published to some queue (.inq or .outq??) for some other service to pick it up and process. Why is it publishing the message to both queues?

#4

“No response” is when the Service that processses the message either returns a null response or has a void or Task method signature. i.e. it’s dependent on what the message that processes it returns not how it’s called.

If you don’t need to use it you can disable it with DisablePublishingToOutq=true.

#5

But it seems that ServiceStack is determining that there is “no response” immediately after calling PublishMessage(). But how can that be when the very nature of pub/sub is to fire and forget. It’s very possible that the service the processes the message does it minutes, hours, days later.

I don’t want to just disable something. I’m trying to understand what ServiceStack is doing here, but I’m just failing to grasp it so far.

#6

Messages does not get sent to the .outq before it’s processed, the actual code that sends it is here:

After the message is processed which only gets sent to the .outq when the message response is null response and it’s not a Request/Reply message.

If you’re not listening to the .outq anywhere you’re not using it, so just disable it with DisablePublishingToOutq=true.

#7

So I disabled the publishing to the outq and now after a very short period of time of sitting in the .inq, it gets moved to the .dlq

Is there a setting somewhere to set the time it goes to the dead letter queue? Seems way to short as it is.

#8

Messages don’t get moved into the .dlq on their own, what ever processed the .inq message has failed. Check the message in the .dlq for info.

#9

{“errorCode”:“NotImplementedException”,“message”:“Unable to resolve service ‘TestMq’”,“errors”:[]}

So I commented out public object Any(TestMq request) method from this app to try and test how this is going to work if I have a different “producer” service and “consumer” service. My assumption was that without the Any(TestMq request) method running anywhere, it would just sit in the .inq waiting for the “consumer” service that does have this method to come online and then process anything in .inq. But it seems I’m still not quite understanding this.

#10

It doesn’t try to execute messages on their own, do you still have the TestMq MQ Handler registered?

#11

And I’m still not grasping why I need a .in and .out queue. When writing SQS services normally, you simply have one queue that you publish to and listen to that same queue to process messages. So I’m still failing to understand why with ServiceStack, we publish to an .in queue and it then gets moved to an .out queue. Why can’t it just sit in the .in queue waiting to be processed by another service?

#12

I feel like I’m explaining the same things in circles.

and you’ve just said you’ve disabled the .outq. It doesn’t get moved, the message gets processed according to the MQ Message Flow in the docs and re-iterated in my comments above.

If you have an MQ Handler it processes the message, if you don’t want it processed, then don’t register the MQ Handler that processes it.

#13

Ok, that was it. The handler was still registered in that service. I removed that. Created another service and left this code in. It correctly processed the message from the queue that the other service published. This is exactly what I was expecting. Thanks.

#14

Sorry for all the trouble. ServiceStack seems to be a very different way of “doing things” and so it is sometimes hard to get your head wrapped around it if you’re used to doing things the “traditional” ASP.NET way. But once you grasp it, it clicks and it makes a lot of sense and you can see and appreciate the increased productivity that ServiceStack provides.

Thanks

1 Like