Provide support for multi-staged message processing

[tomkerkhove]

Provide support for multi-staged message processing.

An example of where this is useful is when one process needs to:

• Persist an order in DB
• Track AI event
• Send email communication
• Send webhook

Every step is a small piece of code doing one thing after which it will be considered "done". Every step that is done will be flagged as completed and should not be re-executed.

The messaging infrastructure should:

1. Dequeue messages
2. Check if the context contains a previously successfully executed state
3. Skip all the executed steps and only process what remains.
4. Abandon the message and stamp it with last succesfully processed state

Important: Not all ours sources will support this and will only be Azure Service Bus at first sight.

[tomkerkhove]

I've written this before and can take this up/advise

[stijnmoreels]

This will maybe made easier once we introduced the message handlers.

[tomkerkhove]

Yes and no, yes because every stage will have a handler but the pump should orchestrate thing, but not a prio for now.

[stijnmoreels]

Like we talked about offline.
What if we used the context filtering of the message handling to determine which step to use, and provide in the specific Azure Service Bus message handler a Service Bus MessageSender which can be used to send the message back on the queue/topic (with a different message context so the next message handler will be executed)?

[stijnmoreels]

Hmm, we may want to hold on with this so we can first migrate to the Azure.Identity and therefore the new Azure.Messaging.ServiceBus package. That would mean that we can use the ServiceBusSender instead of the MessageSender in the specific Azure Service Bus message handlers.

[stijnmoreels]

Hmm, with the current setup, you could already to that yourself (a bit), as the router is registered in the dependency injection container, you could retrieve it in your custom message handler, change the messaging context, and send the received message again through the message router.

[tomkerkhove]

Yes but it should be simpler, but I'll provide some more examples later on

[stijnmoreels]

The message router needs to have a ServiceBusSender instance, so it may be good to have this injected, and for that, we may want to use the injected stuff for the entire message pump (see #34) in v2.