Creating a State Machine Saga With MassTransit (Orchestrated Saga)

[@MilanJovanovicTech]

Get the source code for this video here → the-dotnet-weekly.ck.page/masstransit-saga Want to master Clean Architecture? Go here: bit.ly/3PupkOJ Want to unlock Modular Monoliths? Go here: bit.ly/3SXlzSt

[@dcernach]

The most clear crystal video I've seem about MassTransit Sagas. This video must referenced on masstransit documentation. Great video as well. Please, bring us a video about the MassTransit Courier Service, it is great for distributed transactions with compensation routines!

[@MilanJovanovicTech]

I think it would've been even better if I had used separate services for each handler. But I felt that would overcomplicate the matter and distract the user from the actual Saga pattern. I'm glad that you enjoyed this one :) P.S. Thanks for the video suggestion!

[@aakashpoojary3968]

@@MilanJovanovicTech It would had been far better, if handlers would had been split in to 3 different services. It somewhat simulates the real world applicatoin, just my thoughts.

[@baranacikgoz]

Great video. This concept feels like "do I really need this? Can't I create an entity and update it with its methods instead?" But I think if you make another video about implementing a complex flow with traditional ways and saga pattern in the same video and show differences, people will have an "ahaa" moment

[@MilanJovanovicTech]

That's a great suggestion for a future video!

[@stephendgreen1502]

The best topic you have covered, in my opinion. Very much appreciated.

[@MilanJovanovicTech]

Now I need to move these handlers into separate services, and add distributed tracing on top.

[@stephendgreen1502]

@@MilanJovanovicTech And hire some code-skilled support engineers to deal with data lossiness.

[@stephendgreen1502]

@@MilanJovanovicTech There are two big weaknesses with sagas. One is the expertise required to write and maintain them well. The other is that they can never plug all the data leaks: There are always edge cases where data gets lost. The workflows are partly non-deterministic because they are weakened by issues such as latency, plus the workflows tend in some cases to never complete such as with timeouts.

[@PhatBoyG]

@@stephendgreen1502 if you end up with "data leaks" in your system, it's a result of poor programming, not message loss. There are many companies using extensive sagas with MassTransit that have no lost transactions, and they're doing 400,000 an hour in real-time. So, your experience may vary, but blaming the technology/pattern is misguided.

[@drhdev]

I would definitely pay for a MassTransit course. Chris's videos are great, but would love some more real examples of Routing Slips (Choreography) and State Machines (Orchestrators). State/Status/Processing steps are one of my biggest struggles. Would love to know more using examples like Invoices/Reserving Inventory/Billing Customer/Shipping Items/etc. Also would be helpful if you explained how to correlate using an integer as an Id.

[@MilanJovanovicTech]

Noted! Maybe I can do a longer YT video first, and see how that goes.

[@TuanNguyen-ed9rb]

I find Chris' videos are hard to follow for beginner

[@ryoman76]

Great! Milan Now I am waiting for a video where you explain a video on how to handle an undo/rollback operation

[@MilanJovanovicTech]

Here's the conceptual approach: kzbin.info/www/bejne/fIG5q5isoZuFn5o

[@dy0mber847]

Thanks for valuable content, Milan.

[@MilanJovanovicTech]

Most welcome!

[@rconn5641]

Awesome content. I appreciate the effort you put into your videos.

[@MilanJovanovicTech]

Glad you enjoy it! Thanks :)

[@Helza99]

I don’t know what it is, but for some reason this feels “off”. Just can’t put my finger on it. You have the regular publish/subscribe handlers which all just accept a command and send out a new one, but then you have one central state machine which accepts those and then sends the new publish command. I guess it feels off because it is all an 1-on-1 for the state and commands. You’re pretty much just resending the commands because of that. I guess this becomes a lot more useful with more complex/involved flows and it’s easier to track and continue flows in this way. I guess you made it too simple to follow along 😊

[@MilanJovanovicTech]

I have to make these example simple so that they're easy to demonstrate. Think of an order processing flow online. You have a few steps: - Placing an order - Reserver items on stock - Making the payment for the oder - Shipping the order to the customer This is difficult to implement as one long-lived transaction. Which is why we can introduce a saga (state machine) to manage the individual steps. Moreover, these steps can be handled by different physical services which makes it even harder to track. So a central orchestrator that can tell you the current state of the order processing flow is a unique advantage.

[@allen9504]

Isn't saga about compensating? Why is a step fails and you need to roll back?

[@apatock]

Hi Milan, first of all - I like your tutorials very much. There are many of them which helps me out. I have a question: does the saga continues if let's say the process getting stopped between the Welcome and the FollowUp state? If so, does this work automatically or have one to send a message to trigger continuation?

[@MilanJovanovicTech]

Yes, if you use a saga persistence mechanism that isn't in-memory. The saga stores the current state, so it can pick up where you left of.

[@rakeshkumarreddymudda]

🫡🫡🫡Hatsoff to you and efforts with quality content that you create.

[@MilanJovanovicTech]

Most welcome! :)

[@salihcan1991]

Thanks Milan. It's very useful video.

[@MilanJovanovicTech]

Glad it was helpful!

[@KolpaPlak]

super good explanation and nice video edit. perfect

[@MilanJovanovicTech]

Thank a lot!

[@carlosportillo6915]

Great video, thanks for sharing

[@MilanJovanovicTech]

You bet!

[@tzurdo1]

Great video, as always :)

[@MilanJovanovicTech]

Thank you! Cheers!

[@qua11q7]

Awesome tutorial. I have one question. At 14:34, we see that you landed in the last step, which is OnboardingCompletedHandler. In this step, the Consume method hasn't finished yet. When we take a look at the saga state in DB at 14:41, it is at the Final state, which I assume that the saga has finished. But we haven't still complete the Consume method in the last step. What would happen if the last OnboardingCompletedHandler throws an error? Will the saga state be still Final, how can we detect such issues where the saga seems complete but actually the last step erred. Thank you.

[@MilanJovanovicTech]

"What would happen if the last OnboardingCompletedHandler throws an error? Will the saga state be still Final, how can we detect such issues where the saga seems complete but actually the last step erred" - Yes in this case the Saga would transition to the final state. We can add another step, where we wait for a message while in the Onboarding state. - During(Onboarding, ...) I'll have to do a separate video on error handling, but essentially MassTransit allows you to handle faults whenever you publish a message.

[@idgafa]

It would be great if you cover also how to rollback if some state we cannot move forward? Is there some builtin functionality also in MassTransit?

[@MilanJovanovicTech]

I did, albeit not with MassTransit: kzbin.info/www/bejne/fIG5q5isoZuFn5o

[@s1kky]

What a legend!

[@MilanJovanovicTech]

Much appreciated!

[@drhdev]

I would love if you would go in depth on this topic more. Not very many well made MT vids online.

[@MilanJovanovicTech]

Can do!

[@ryoman76]

Hello Milan, I have a question if I need to build a saga between two microservices from A (with its database) to B (with its database) the saga in a clean architecture where should I write it? In microservice A or B? Thanks

[@MilanJovanovicTech]

Which one should control the flow?

[@ryoman76]

@@MilanJovanovicTech Microservice A

[@novaploca2080]

I literally been watching tutorials about this last week

[@MilanJovanovicTech]

Cool. Mind sharing a few that you watched? I'd love to see it

[@malikbrahimi7504]

This is what I'm looking for with a POC I want to develop that allows users to define jobs in which one "consumer" is ran after the previous consumer's condition has been met. The only problem is that I need something that is dynamic since the user would be defining these states via UI, is that possible?

[@MilanJovanovicTech]

Looks like some sort of dynamic saga generator?

[@malikbrahimi7504]

@@MilanJovanovicTech Basically want to create an application where users can define data flows via UI that will allow them to specify logic gates on incoming stream data. I did more digging into it and it could probably be defined with sagas but I think Apache Flink is probably a better choice

[@smnb6652]

EDIT: Nvm, forgot the AddConsumers call in the setup. ---- Do you need to start the bus or have a hosted service running or something? If I try this, I don't hit the breakpoint after using bus.Publish, however there are no errors or anything in my logging. Just nothingness.

[@MilanJovanovicTech]

Glad you have it sorted 😁

[@eduardoandrescastilloperer4810]

Why did you declare your commands as records but not your events?

[@MilanJovanovicTech]

Easier serialization

[@sunzhang-d9v]

I don't know much about the roles of RabbitMq and Postegresql

[@MilanJovanovicTech]

Research

[@emanuelrodriguez3155]

what happens when we face an error during the steps?

[@MilanJovanovicTech]

We have a few options, mainly we want to retry processing the step so we can complete the saga. However, if we can't continue, we can publish a "failure" message and consider how we can reverse the effects of previous steps. Like implementing an "undo" operation.

[@roberteru25]

To a compensation

[@emanuelrodriguez3155]

@@MilanJovanovicTech would be really interesting a video on how to handle an undo/rollback operation

[@dcernach]

@@emanuelrodriguez3155 Tottaly agree, in masstransit it is called courier service to handle these cases...

[@AndrewCao-iw5by]

I applaud for the effort made for the vide but there are too many topics to explain in a short vide. it is not detailed enough for begineers.

[@MilanJovanovicTech]

This isn't really a beginner-friendly topic

[@geepy5708]

Your videos are getting more complicated 😢

[@MilanJovanovicTech]

The topic is complicated...

[@torrvic1156]

This is a very complicated stuff. Do you think that it is a good idea to use Redis to store SAGAs instead of Ef database Milan?

[@MilanJovanovicTech]

Maybe, if Sagas are short-lived. You can always switch it with a SQL DB