Currently we have a search endpoint which has to do a lot of processing of results before returning to the caller. It's starting to take far too much time. So, I'm trying to figure out a solution and wondering if message MQ stuff can help.
In short, our endpoint fetches search results very quickly, but post processing to add details like user costs is very expensive. I'd like to find a way to return the results right away and then send the post-processing to another service which returns the results to the browser when they are ready.
After reading over the messaging MQ stuff I'm wondering if this is the best direction.
My assumption is that I could fetch my results, send those off to be processed via message MQ service and return the un modified results right away. Then, within the MQ service I could use Reply To to send the return data to our browser messaging service. This would then allow the browser UX to update the results as necessary.
Is this a valid approach or am I understanding this incorrectly? Or maybe there is a better approach?
I wouldn't use MQ for time coupled operations like read queries, it's ideal for idempotent async one-way time decoupled write operations that aren't time sensitive (i.e. waiting in response to a query).
It sounds like you want to return multiple results which you can either return via Server Events to notify the client as new results are available, the simpler alternative is to make different API calls (i.e. to slow + fast results) and update the UI as the APIs complete.
Ok, well good to know I shouldn't take that path.
We've had problems with server events in the past - but actually use a different service to do the same. But taking that approach could work. I'd have to send out an initial event of the search results and then additional events as those get updated and processed. We're talking a total time right now of about 5-10 seconds depending on the user. So having the long running endpoint should be fine.