Bulletproof Workflows with Temporal | Microservices orchestration the easy way

What if we could build bulletproof workflows just by defining a simple function in our favorite programming language? Wouldn’t that be great? Well that's exactly what Temporal has to offer.
Intro
We have discussed many times how difficult is to coordinate complex flows in a distributed system such as a microservices architecture or a highly available monolith.
Temporal solves the issue in the most elegant way possible. We can simply define a function in our favorite language. Java, Go, Python, PHP, Typescript, .NET and many more are supported and interoperable.
Why Temporal?
The two main selling points of Temporal are:
1. It promises us to code smarter and faster. We can pick our favorite language between Java, Go, Python, TypeScript, PHP (and many more are on the way) and define the business logic in workflow and activity functions. There is virtually no learning curve because we use the native constructs of the programming language, rather than a complicated framework.
2. The Temporal runtime makes all the heavy lifting involved in executing the workflows. It receives workflow execution requests, schedules them, persist their state and communicates with all the relevant worker services.
Top that with a wonderful administrative panel that gives you full visibility on workflows and their state, as well as management functions such as the ability to terminate a workflow.
Workflows Activities Signals and Timers
The core concepts of Temporal are Workflows and Activities.
The Workflow represents the business process we want to model. We could have a workflow to model orders in an online shop.
The Activities represents the logical steps we need to perform in the Workflow. Such as reserving a product from the inventory or booking logistics.
Another two important features of Temporal are signals and timers. A signal is a message sent to the workflow which can be used to change its state and control its execution flow.
Timers are self-explanatory. Rather than waiting for a condition we wait for an amount of time. It could be 1 second, 1 hour, 1 day or even 1 year.
---------------------------
Code samples available on GitHub: github.com/marcolenzo/tempora...
🤓 There's more on my website! (articles, scripts, code snippets, etc...): 🤓
marcolenzo.eu/
🤝 Connect with me: 🤝
/ marcolenzo
👌Get a CodeCrafter Discount 👌
app.codecrafters.io/join?via=...
#microservices #saga #temporal
#java #designpatterns #softwaredevelopment
0:00 Introduction
0:14 What is Temporal?
2:11 Workflows and Activities
4:15 How Temporal Works
6:08 Signals and Timers

[@MarcoLenzo]

I love the idea of using native programming language constructs to define a business workflow. What are you going to use Temporal for?

[@giantboanerges206]

Best getting started with Temporal. Thanks

[@MarcoLenzo]

Thank you 🙏

[@tsurdilovic]

Hi, great video!! Couple small things: 4:05 - you can add that workflow executions in Temporal are not tied to a specific worker. If one of your worker processes crashes, your executions can be continued on a different worker. In this situation new new worker would replay your workflow code against the executions event history and discrepancies in commands generated would result on non-deterministic errors. In most cases users can still fix the issues and allow workflow execution to continue and complete. 5:36 "When this is done Temporal is going to schedule a replay of the workflow" - this is not really correct, workers have an in-memory cache of executions that it is processing. When activity worker responds back to service saying it has completed the activity, service would try to send this update to the workflow worker that has so far been processing the execution (sticky task queue). This is an optimization so that your workflow code does not have to be replayed on any update. If the workflow execution on this worker gets evicted from this worker cache before the new update (activity completion) comes in, then worker would need to fetch the event history from service and replay your workflow code against this event history to be able to continue execution. So in most cases updates to execution that are delivered to workers do not cause this internal worker replay. Looking forward to more of your videos on Temporal!

[@MarcoLenzo]

Thank you Tihomir! It is really appreciated, especially coming from you. I will surely do more videos about Temporal in the coming months. I am confident we will have more feedback to share from our personal experience with your solution. Regarding workflows replays, I wanted to focus on them because I noticed it is usually the most difficult concept to grasp. Once you get that, it becomes obvious why the workflow must be deterministic. However, we did notice the optimization you mentioned and I should have at least hinted at it. I take the chance to wish good luck to the Temporal team. We really love your product!

[@GavSaysPogMoThoin]

Great video! Very well broken down and explained. Thank you!

[@MarcoLenzo]

Glad you enjoyed it!

[@uri30000]

Please create a full blown tutorial with complext payment flows or data pipelines :)

[@MarcoLenzo]

Sure 😊 I'll do my best to plan such video

[@VaibhavPatil-rx7pc]

Excellent information thanks

[@MarcoLenzo]

Thank you 🙏

[@MrNikMic]

Interesting video however I do have a question. For companies who are heavily invested in a certain platform (eg. AWS), what are the advantages of using 'external' frameworks like Temporal over a more native solution like AWS? The same can be said for other functionality like queuing (SNS/SQS/EventBridge vs Kafka/RabbitMQ or DynamoDB vs a hosted version of MongoDB). I'm not sure whether this is a good topic for another video but it would be great to hear your thoughts about this topic :)

[@MarcoLenzo]

The nearest thing to Temporal in AWS would be Step Functions. They are both solutions to handle orchestration of complicated workflows in a distributed system but they do it in two completely different ways. AWS Step Functions uses a domain-specific language. It is also known not to be performant or cost effective. www.primevideotech.com/video-streaming/scaling-up-the-prime-video-audio-video-monitoring-service-and-reducing-costs-by-90 Temporal uses the programming language itself. The business logic is literally a durable function in your favorite language. No extra tooling needed. You are also relieved from defining and managing messages and broker topologies. You save a ton of time in development since a lot of code dealing with integration is not required at all. From an operational standpoint, Temporal can run in any cloud and it is open source. It is also present on the AWS marketplace. That said Temporal is not going to replace SNS or SQS but you can actually integrate them. I would still opt to do system wide event broadcasts with a message broker. Temporal is best suited for orchestration in my opinion. Whether you use Step Functions or Temporal, I think it boils down on which approach you prefer between DSL and programming language and/or performance and cost considerations.

[@ganesharulanantham2674]

Very good video clearly explained

[@MarcoLenzo]

Thank you for the feedback

[@sushilb7994]

Great video! One confusion I have is your description of the videos says it supports dotnet whereas Temporal documentation doesn’t say the same. Would you please clarify?

[@MarcoLenzo]

Sure! The .NET SDK has not a stable release yet but it's in active development. This is the link to the repository github.com/temporalio/sdk-dotnet and this is the link to the documentation dotnet.temporal.io/ Notice that the development guide is being written as well. It's just a matter of time and you can show your interest by joining their community or slack as well.

[@sushilb7994]

@@MarcoLenzo thanks for the details. Appreciate your prompt response!

[@mayori-engineering-hub]

more videos???

[@MarcoLenzo]

On Temporal?

[@Akash-rs6qv]

@@MarcoLenzo Yes Please!