Using sagas to maintain data consistency in a microservice architecture by Chris Richardson

  Рет қаралды 307,797

Devoxx

Devoxx

Күн бұрын

Пікірлер: 173
@benp7328
@benp7328 5 жыл бұрын
Very interesting talk, but set playback to 1.25x speed. It transforms this talk.
@michaeleaster1815
@michaeleaster1815 5 жыл бұрын
1.25x transforms all talks (!), usually for the better
@nallap6566
@nallap6566 5 жыл бұрын
Well tbh even 1.5x...sooner the better..when the speaker is too slllll000wwww..
@aldebaranh666
@aldebaranh666 5 жыл бұрын
1.75x no prob
@burstingfist
@burstingfist 5 жыл бұрын
Nah, 2x is the new 1.25x
@MrMikomi
@MrMikomi 4 жыл бұрын
2x is better.
@zhou7yuan
@zhou7yuan 4 жыл бұрын
Agenda [3:58] ACID is not an option [5:04] (Scenario: Customer has a credit limit) [5:13] Transactions in a monolithic architecture [5:58] Concurrent transaction for the same customer will be serialized [7:15] Microservices [9:00] Microservice architecture [10:30] Loose coupling = encapsulated data [11:13] 2PC is not an option [13:41] Overview of sagas [15:01] Use Sagas instead of 2PC [15:14] Create Order Saga [16:22] Rollback using compensating transactions [18:20] Saga: Every Ti has a Ci [19:08] Create Order Saga - rollback [20:30] Sagas complicate API design [22:03] Request initiate the saga. When to send back the response? Option #1: Send response when saga completes - Response specifies the outcome - Reduced availability Option #2: Send response immediately after creating the saga (recommended) - Improved availability - Response does not specify the outcome. Client must poll or be notified Revised Create Order API [24:41] createOrder() returns id of newly created order NOT fully validated getOrder(id) called periodically by client to get outcome of validation Minimal impact on UI [25:22] UI hides asynchronous API from the user Saga will usually appear instantaneous ( UI displays "processing" popup Server can push notification to UI Sagas complicate the business logic [26:33] Coordinating sagas [28:33] How to sequence the saga transactions? [28:41] After the completion of transaction Ti "something" must decide what step to execute next Success: which T(i+1) - branching Failure: C(i-1) Option#1: Choreography-based coordination (distributed) [29:45] Option#2: Orchestration-based coordination (centralized) [30:16] Saga orchestrators are state machines [30:48] Implicit vs. explicit orchestrator [31:47] Event-based, implicit orchestrator [32:38] Explicit orchestration [34:05] Create Order Saga (code sample) [35:12] Initializing the saga [37:20] Handling a reply [37:58] Customer Service - command handling [38:28] Transactional messaging [39:18] About Saga orchestrator ⇔ participant communication [39:22] Messaging channels [40:52] Create Order Saga messaging [41:31] Messaging must be transactional [41:59] 2PC still isn't an option [43:06] Use database table as a message queue [43:22] Publishing message using polling [44:33] Transaction log tailing [46:17] About transaction log tailing [46:57] MySQL master-slave replication protocol DynamoDB table streams Summary [48:13]
@enkaypeter9313
@enkaypeter9313 3 жыл бұрын
Indeed, not all heroes wear capes.
@Suamere
@Suamere 5 жыл бұрын
I incorporated a company in 2007 who's goal was simply to proclaim the wonderfulness of MicroServices. Since (and before) then, I have never seen such a flawless talk related to MicroServices. Every other "concern" about distributed systems, Service-Oriented Architecture, Statelessness, Events, Administration, Security, and so forth have all been so mundane. And frankly, all of those other concerns can be mocked out in fairly generic frameworks. This talk touches on the actual MEAT of Software Development which is different for every customer and is the actual fun challenge. It may be stuff that experienced MicroService people know, but it has never been spoken so eloquently. I appreciate this talk very much.
@thechargeblade
@thechargeblade 3 жыл бұрын
Exactly, a lot of talk just cover from the perspective of technology, sure Microservice is technically a better way to design a system, but when it comes to implementations data is number one priority.
@vishalsheth1888
@vishalsheth1888 4 жыл бұрын
Great talk, one thing I realized after this talk is that 2PC is an option.
@morespinach9832
@morespinach9832 9 ай бұрын
As are stored procedures from 30 years ago.
@anildatt8103
@anildatt8103 4 жыл бұрын
Old wine in new bottle. I have been doing this design pattern of State Machine Orchestrator in my SOA EDA , with compensating transactions to rollback what services had done till the point of failure. The failure event is either broadcast to all the services or to the Orchestrator so it can co-ordinate the cancellation. As I listened to the talk it was like going through my design document. Message table to message broker has a drawback when the load is high. As it is DB spefic to pull data out to push on message broker it introduces a bottleneck in the DB. Ideal for this is using a distrbuted cache DB or like he mentioned the replication and use the replicated data to message broker. Excellent talk. It touched all the touchpoints Newbies to Micorservices without much experience may think this is something rocketscience, but it as been there for long in a different avatar.
@morespinach9832
@morespinach9832 9 ай бұрын
What orchestration engines do you recommend? Stuff like camunda?
@GarriAndOkroSoup
@GarriAndOkroSoup 4 жыл бұрын
Had already been implementing this principle in some of my designs before learning that it had a name . what a saga :D
@alexknn15
@alexknn15 4 жыл бұрын
GarriAndOkroSoup same thing, implemented basic idea like 7 years ago but with 2pc inside each step of saga to save data and send to the next message queue:)
@skipdigit
@skipdigit 3 жыл бұрын
Agreed. Been building these types of solutions like 10 years ago. Now it has a name.
@רונילוי-ג8ב
@רונילוי-ג8ב 3 жыл бұрын
I readed alot about Saga design pattern and I developed two years a system than implement Saga design pattern and I still learned new things from this video. Thank you.
@morespinach9832
@morespinach9832 9 ай бұрын
How did you do things differently from a stored procedure?
@landkasupada
@landkasupada 4 жыл бұрын
This is a real talk on real use cases on real problems
@mohammadbarbast6524
@mohammadbarbast6524 2 жыл бұрын
excellent, I have looking for something like this for days. really thanks to Chris Richardson
@alexknn15
@alexknn15 4 жыл бұрын
Great talk, like that you explained very important part as processing data and sending event atomically. Other resources usually do not explain this important part. But I want to mention one moment, as usually messaging systems guarantee “at least once” delivery, we have to think about idempotency. And if we implement it, we don’t need complicated atomic save and send mechanism anymore, we now can save data and then try to send message to the next queue and ack/commit current queue. For example If we have server failure after saving to DB and before sending to next queue and acking current, then the other instance will get and process current message (here idempotency come into play as we already saved data to DB and another instance will do the same processing cycle with the message - save to db and send to next queue)
@avinandanbanerjee9568
@avinandanbanerjee9568 2 жыл бұрын
won't the other instance not send to the other queue, as it fails when saving to db? so we have at most once.
@brainoverflow98
@brainoverflow98 4 жыл бұрын
I feel like "Saga Orchestration" is completely against the logic behind the microservices which is building independent components and teams. In this scenario a team has to maintain the orchestration point of two services which also has to be aware of the business logic of both services. I know sometimes orchestration is inevitable but I think in this situation it's better to use choreography.
@8Trails50
@8Trails50 4 жыл бұрын
I agree. It seems like if you need to pull this out, something has gone wrong.
@АндрейЩербинко-ш5н
@АндрейЩербинко-ш5н 3 жыл бұрын
In fact saga orchestrator became distributed transaction manager
@willd1mindmind639
@willd1mindmind639 3 жыл бұрын
The example he is using is a bad one for the principle he is illustrating. Financial processing has always been the epitome of transaction processing in computing. And in modern web commerce you want the customer to know in real time when a payment fails. In that case, the payment service or credit service is a completely separate and independent application, often provided by a third party outside of the organization and there is no coupling directly between that service and the order service. The orchestration is done from the web client so that it can display the response from the payment service to the customer directly. You don't want that to be stuck in some asynchronous message loop after the customer has already placed the order.
@ramonennik2536
@ramonennik2536 11 ай бұрын
Well I do not agree, actually all the payment gateways I had to implement in the past were always working with an async backend call to update the payment status so unless that message arriverd earlier in the backend then you couldn't tell if the payment was really successful on the thanks page
@jsignuy
@jsignuy 6 жыл бұрын
I'd only add some minute or two about idempotency in some key moments of the process. Despite of that, this is an excellent talk.
@kevinkkirimii
@kevinkkirimii 7 жыл бұрын
Great speaker , articulate and simplifies the concept
@chang8106
@chang8106 4 жыл бұрын
Actually I'm readying that book. very informative book!
@greneroom
@greneroom 3 жыл бұрын
Very interesting talk! One question I had near the end of the talk - it seems like using the DB’s change log as the mechanism to publish to the message broken has a large drawback: it tightly couples the internal DB scheme to the API a service exposes to the Saga (which defeats one purpose of microservices, since you want to avoid leaking your DB). I didn’t quite follow why having each individual service coordinate a 2PC between its own DB and a message broker was problematic. This seems much less bad than using distributed TXNs to implement the entire Saga, since the 2PC is an implementation detail of each service. The only single point of failure is the message broker, and I think this is true regardless of your approach?
@talivanov93
@talivanov93 4 жыл бұрын
Great Speaker! Knows how to simplify ideas.
@BrahmaAcharya
@BrahmaAcharya 6 жыл бұрын
Great talk! I hope Sagas don't manifest themselves as ServiceBus. Whereas Sagas are some state machine implementation, people would abuse them until it becomes another ball of mud. I would rather prefer microservices to be along the following lines: - Define your bounded context well so that they are relatively independent and will have consistent state within the context - If you must have other services decide a consistent state (those shouldn't be lot if you have followed the above), rely on synchronous calls to other services. With next generation protocols like gRPC performance impact is negligible - Leverage service mesh framework like Istio for coordination
@hydtechietalks3607
@hydtechietalks3607 4 жыл бұрын
Thank you sir, great insights!!
@sidekick3rida
@sidekick3rida 3 жыл бұрын
His microservices talks are the most informative and concrete out of anything else I've watched. Definitely getting the book! There's also a lot of information on his website ➠ microservices.io
@hamedhatami2012
@hamedhatami2012 Жыл бұрын
It's been really informative demonstration about how the Saga works entirely, when it comes to publish a message to the message broker (queue) in order to make them transactional , we can design them to get ACK back then consider it as done thus give it a go
@donotreportmebro
@donotreportmebro Жыл бұрын
13:08 correct me if I'm wrong but that transaction example with a single database doesn't work unless you have specified a serialized isolation level (one cpu core to execute all transactions). Otherwise it suffers from write skew even with Snapshot Isolation transaction isolation level which is by default I guess on many relational DB systems. You still might have two transactions that would confirm that there is enough credit at the same time, and at the same time execute insertion of two new orders.
@jamiro85it
@jamiro85it 7 жыл бұрын
really pragmatic talk. thank you!
@99ProfessorGroup
@99ProfessorGroup 4 жыл бұрын
With 2x speed, it's a very knowledge 25 minutes talk!
@krishnapatni
@krishnapatni 4 жыл бұрын
one alternative approach to the deleting the messages in the message table is : 1. mark the processed messages as processed = 1 2. select only those messages for which processed = 0 and order them according to the timestamp
@benhook1013
@benhook1013 4 жыл бұрын
Would still have the issue where transactions that start in one order may get committed to the table in a different order (i.e. timestamp of 2nd transaction may be before timestamp of 2nd transaction's message), which is a problem if you want to continue processing them in the original order (which is why he mentioned the incrementing numbers).
@absolutejam
@absolutejam Жыл бұрын
Really enjoyed the talk, but imagine if they these talks used a richer language like F#, Rust, Scala, etc that have enums and traits instead of all the class based fluff ❤
@sharauro
@sharauro 5 жыл бұрын
Great talk. What I understood from this talk is microservice consideration and implementation have to be chosen very carefully. Otherwise a perfectly running system that maintains ACID can be runied in a very short period. I guess microservices are good when a system with low transaction volume can be refactored to be cloud native.
@khaledalothman4314
@khaledalothman4314 5 жыл бұрын
Microservices are good when the granularity is big enough to avoid ACID properties across services are not needed, yet the microservices architectural benefits can still be realized at that higher granularity.
@alexright7549
@alexright7549 3 жыл бұрын
Hi, i understand that 2PC is out of the scope between two microservices, but inside a single microservices you can use it. For example when you have to write a record on DB and send a message to a broker to start another micorservice.
@Adi-yi6qq
@Adi-yi6qq 11 ай бұрын
I am extremely confused by this orchestration based saga approach. Doesn't this present with the same problems that we set out to solve in the first place? For example, what if the saga has 5 steps and for some reason, after making the API call for the 3rd step (say to the CustomerService) and waiting for the API response, the orchestrator service goes down. How will the Saga Orchestrator after restarting know where to start from? It can at best start and try to perform 3rd step again but the 3rd step might have already been processed by CustomerService which means we might be trying to perform same action twice.
@JohnMcclaned
@JohnMcclaned 9 ай бұрын
The orchestrator is supposed to be durable and consistent. Meaning you could kill the orchestrator process and start it again and it should be able to load it's last committed state (from a database) and continue. You would use idempotent endpoints on each service the orchestrator calls to prevent double processing on the rare occasion.
@volkerreichel2285
@volkerreichel2285 7 жыл бұрын
Very good explanation of the concepts.
@tkousek1
@tkousek1 6 жыл бұрын
Great talk. Learned a lot. Question, at 29:50, isn't it a bad idea to use choreography under any circumstance since it no longer loosely couples microservices (which is one of the main benefits of a microservices architecture)? I know he mentions the following centralized approach which is better. But what I'm curious about is, "shouldnt one avoid (at all costs) doing choreography and not even say that is a viable option in the first place?"
@kevinkkirimii
@kevinkkirimii 5 жыл бұрын
There are situations where choreography works better than orchestration especially for systems that produce events that need not to be executed sequentially.
@opentrail
@opentrail 4 жыл бұрын
Good presentation and still this is just showing one aspect chipping off the iceberg. Moving from a monolith to distributed remote services really introduces a ton of complexity and impacts latency. Done badly, and most of them will be, companies move from one ball of mud to another. Its when something goes wrong that is the killer... and you have to scramble around for a magnet to find the needle in a haystack. There are still lots of companies that haven't matured in terms of building and packaging services for a monolith, let alone scaling that to a distributed service architecture which isn't any easier. Lots of opportunities for IT staff in the coming years. I still think that modelling vertical slices based on "entities" is a mistake in presentations. Customer or even order, for example, means completely different things to different roles in an organisation.
@BipinOli90
@BipinOli90 2 жыл бұрын
Excellent talk. Learned a lot. Thanks
@cschandragiri
@cschandragiri 6 жыл бұрын
Very well explained! Thanks!
@Twonee-122
@Twonee-122 4 жыл бұрын
This was great 👍 . Very easy to follow.
@cchinaka
@cchinaka 3 ай бұрын
Refreshingly great tech talk. Thank you
@stanleychan2099
@stanleychan2099 Жыл бұрын
surprisingly interesting talk and could really relate. great learning reference
@sigvamo
@sigvamo 3 жыл бұрын
These are very old and good known problems of any distributed system. Main point is when you go from monolith to microservoces you go from single instance to distributed with all of its problems and complications.
@ahmaddeveloper7595
@ahmaddeveloper7595 3 жыл бұрын
great talk, got a lot of useful information
@RomanRoschin
@RomanRoschin 6 жыл бұрын
Started as a good talk, then slightly became messy after the 'theory of sagas' and when all problems were 'interesting' (but without solution), then, when it came to 'practical' part, everything turned into a complete mess with reinventing message brokers on top of specific features of databases. Anyway, we will probably get another 'open-source' framework :)
@thatpaulschofield
@thatpaulschofield 5 жыл бұрын
The Saga and Outbox implementation remind me a lot of NServiceBus.
@rambo4014
@rambo4014 2 жыл бұрын
Chris the way you are referring to implicit orchestration through events, I am confused then what is choreography?
@austecon6818
@austecon6818 3 жыл бұрын
Great talk. Really useful
@marvinalone
@marvinalone 3 жыл бұрын
the money transfer case, I think this kind of function should be designed as in one microservice and done inside a single DB transaction. If there is no option to do so, then probably an intermediate cash pool (fake, intermediate account) is needed
@andyliu1210
@andyliu1210 Жыл бұрын
That's why in most bank accounts, there is a "reserved" state before it really goes off someone's account.😊
@MikaelUmaN
@MikaelUmaN 6 жыл бұрын
Not sure I get it. How is this not just a distributed transaction where you've added the concept of compensating actions.
@დავითჩხატარაშვილი
@დავითჩხატარაშვილი 5 жыл бұрын
nice talk by Satya Nadella
@vikrantsai7
@vikrantsai7 2 жыл бұрын
why are we trying to do this ? to make systems less complex , to have a smaller focused development teams ? to reduce time to production ? I feel we are making systems more complex , we are adding many more smaller databases , we are re-inventing the transaction mechanism such a waste of time , please do a video where you really need to have microservices and where not, its become a fashion to bounce the terms microservices , REST, K8S
5 жыл бұрын
Good and clear presentation. Nice talk. I am surprised about using tables for recording messages. If you are using Saga coordinator then it could be the one verifying that an operation was successful and generate message for the next operation.
@austecon6818
@austecon6818 3 жыл бұрын
But the sending of the message needs to also be atomic... usually at-least-once-delivery + idempotent consumer is the golden rule there... Can only be achieved (simply) by committing pending messages to local db in a local atomic tx with the new state change... then retrying to send the message until the consumer ACKs.
@putinscat1208
@putinscat1208 7 жыл бұрын
That opening was nuts!
@deathmachine808
@deathmachine808 6 жыл бұрын
No it wasn't bro.
@ismile47
@ismile47 3 жыл бұрын
Excellent, it took last 15mints to understand 1hour repeat and see. Its 4 years before, now i want to see what are latest changes or implications update in saga, any suggestions?? Please replay to comment
@deathmachine808
@deathmachine808 6 жыл бұрын
Thx for the upload, good stuff.
@anhtruong-t2f
@anhtruong-t2f Жыл бұрын
Thanks for sharing
@J-Ind24
@J-Ind24 2 жыл бұрын
This is dope 😍
@florianwicher
@florianwicher Жыл бұрын
Great talk! One question though: If you use a DB table to buffer the messages to be sent to the message broker, haven't you just moved the problem of exactly-once-delivery? What if the mechanism that dispatches the messages from your table to the broker crashes/rolls back?
@Autumn4_love_your_
@Autumn4_love_your_ Жыл бұрын
In this case we first publish message to broker and after that we delete raw with that message. In case when we go down between - we sent message twice but with same idempotent token that must be in table. Its called transactional outbox google it if you want to learn more
@zimpoooooo
@zimpoooooo 5 ай бұрын
What if there is an error when processing from the monitored db transaction log? Or if the monitoring process restarts? Will it skip everything that was logged while down? Everywhere else there is at-least-once guarantees, but I don't see it here?
@arajalali
@arajalali 7 ай бұрын
Interesting progression from "Monolith has this problem X" then "Microservices solved the issue X" but then we have problem Y and then "We solve problem Y by taking away some of the good parts of Microservices", which ends up in losing the benefits of both worlds of Monolith and Microservices.
@vinodprabha1
@vinodprabha1 3 жыл бұрын
Thank you very much
@MbgChat
@MbgChat 10 ай бұрын
Please don't roll your own workflow. See Durable Task Framework and Azure Durable Functions.
@alejandroagua5813
@alejandroagua5813 4 жыл бұрын
Everyone says this is a great talk. Honestly, I didn't see anything new. Little bits from this concept and that concept cobbled together. Talk is cheap. I want to see a working code.
@brite320
@brite320 4 жыл бұрын
We all do. As a software engineer we need to understand the concept and create something unique. It is also perceived by the speaker that in the room, there are alot of engineer using different languages hence a working code is out of scope
@riansyahtohamba8215
@riansyahtohamba8215 2 жыл бұрын
10:50 each services has its own databases. 12:30 how to maintain data consistency for each databases?
@ivantapia9966
@ivantapia9966 3 жыл бұрын
Thanks buddy :D
@rambo4014
@rambo4014 2 жыл бұрын
Choreography looks much better to me and more loosely coupled
@pothiq
@pothiq 7 жыл бұрын
Nicely explained, Thank you.
@郭亮-l3g
@郭亮-l3g 6 жыл бұрын
Thanks!!!
@aravindreddy6699
@aravindreddy6699 6 жыл бұрын
How we rollback if compensation transaction also fails
@AntonKa-yn4mb
@AntonKa-yn4mb 5 жыл бұрын
And i have not heard how to solve the problem of transaction isoletion when we have concurrent transactions
@manikandanr3128
@manikandanr3128 5 жыл бұрын
Compensation transactions should be idempotent and need to retry it unless its succeeded.
@aroundthisprettyplanet
@aroundthisprettyplanet 4 жыл бұрын
retry,, or simply ignore
@chang8106
@chang8106 4 жыл бұрын
Compensation transactions , where in the monolithic approach a Orchestration Saga should aware of the failures when executing each entries, and execute compensate transactions if seeing failures in the steps.
@7th_CAV_Trooper
@7th_CAV_Trooper 2 жыл бұрын
how awesome would it be to get an architectural/code review from Chris Richardson?
@vivekach1
@vivekach1 7 жыл бұрын
Awesome.. really good. Thank you :)
@moijafcor
@moijafcor 6 жыл бұрын
A masterclass! Thank you.
@elijahlucian
@elijahlucian 3 жыл бұрын
@11:00 why would somebody make multiple databases for this? seems utterly crazy is this because one would naively be following the microservices dogma?
@tyrt400z
@tyrt400z 3 жыл бұрын
My reasoning would be the following: 1. The database will become very large very quickly, depending on the number of services. 2. Let's say you have some services that only run in certain situations, then you still have all their data cluttering up the database, even though they are not even being used at certain times. 3. Latency might occur if you have like 10-20 microservices all querying the same database at once. Furthermore, if your company becomes successful the code will keep growing (e.i. more services will be created) and at some point, the database will be a gigantic bottleneck of the whole system. There are more reasons, like the lack of data encapsulating you get when having just one big database. For instance, let's say you have many different organizations as costumers of your system, you might want to run different instances of microservice, each for each organization so it is separated. In the case of one big database (where you will lack separation) then you might end up in a situation where a developer writes code that fetches stuff from one organization but sends it to the wrong organization, which is no bueno.
@NiravPatel1989
@NiravPatel1989 2 жыл бұрын
For the last part, "Transactional messaging", why can't we utilize out of the box solutions like Kafka instead of using database commit log tailing?
@gygabytes
@gygabytes 2 жыл бұрын
you could use the ack mechanism of kafka to only commit to the database when the message gets appended to the topic, but you will be coupling kafka (ie, kafka needs to be available) so that your system can function, which in theory it doesn't need it.. the event is for others, not for your system. so, if you use a local transaction log then your service will work as long as the database is available.. other component/service will send the messages/events to kafka
@alexanderbarvels7403
@alexanderbarvels7403 5 жыл бұрын
Do I have any advantage of using a message broker when using a message table unless I want to fan out my messages?
@dmitrynutels9340
@dmitrynutels9340 4 жыл бұрын
Distributed monolith FTW...
@philadams9254
@philadams9254 6 жыл бұрын
Why not just have a separate "Transaction Service" that tracks the steps and whether they fail or not?
@deathmachine808
@deathmachine808 6 жыл бұрын
lol then it's surely just back to being a distributed transaction - it has a single co-ordinator?
@philadams9254
@philadams9254 6 жыл бұрын
Hmm, yeah, maybe. Can't remember what I was thinking when I wrote that comment to be honest! 5 months in to using microservices and it's still hurting my brain!
@redsquarem3369
@redsquarem3369 6 жыл бұрын
Thats essentially what it is, the saga/process manager governs n events from multiple services
@USONOFAV
@USONOFAV 4 жыл бұрын
why not use an aggregate microservice for transactional operration?
@vinitsunita
@vinitsunita 4 жыл бұрын
AFAIK It is kind of antipattern to use database table as a message queue
@SuperSam4y0u
@SuperSam4y0u 5 жыл бұрын
Can somebody explain to me why is it not a good idea to send msg inside a txn? if a msg was not transmitted then isnt it perfectly ok to rollback the transaction? Wont tailing the commit log of db get us the same behavior?
@xinwang8938
@xinwang8938 5 жыл бұрын
U mean "inside a local transaction"? We should avoid to do this like : Local Transaction => start Txn ==> send msg ==> waiting for response ( ALL reserved ressource for this transaction are suspended ) ===>OK => end Txn ===>KO => rollback
@jfordgaming9615
@jfordgaming9615 2 жыл бұрын
I'm using gateway api as a middleman for my microservices
@pm71241
@pm71241 3 жыл бұрын
40:50 ... Hmm... "assuming that the message broker has durable at-least-once delivery". Yes... assuming that. - and that it's up. I'm not sure I buy that async ReST doesn't work just because you can make the problem go away by introducing a magical component. If you have to run this yourself, is it easier to have high availability on the message-broker than on your service? why? ... and why can't the client just have retry logic a part of the saga state machine?
@ronnie5966
@ronnie5966 3 жыл бұрын
You could have a Saga Service that handles all sagas?
@educostadev
@educostadev 5 жыл бұрын
Read more and get code samples at microservices.io/patterns/data/saga.html
@NavneetVermalivefree
@NavneetVermalivefree 5 жыл бұрын
The Saga seems to me like an AWS step Functions? Is that correct?
@kevinkkirimii
@kevinkkirimii 5 жыл бұрын
Yes
@igorgulco6608
@igorgulco6608 6 жыл бұрын
CreateOrderSaga? Really? Reinventing distributed transaction manager under a new name. The only difference is the sequential rollback invocation. Some silly stuff.
@roceb5009
@roceb5009 6 жыл бұрын
And now when the Customer service changes how they do their credit check, now that's a code change for the order service. Hey, but we could fix that by putting all the code in the same repository, so the teams could change each other's code! This could really turn into a whole app architecture paradigm, I'll call it "macroservices"
@kotobotov
@kotobotov 6 жыл бұрын
i hate whan on every case someone create new abstraction or even worse new framework, instead reuse existing one
@AntonKa-yn4mb
@AntonKa-yn4mb 5 жыл бұрын
This agenda was about bla bla bla
@dmitribodiu1347
@dmitribodiu1347 5 жыл бұрын
@@roceb5009 Nice catch! Using Saga as a central coordinator is a nightmare. It could be used inside bounded context, if some long running policy needed, but choreography is much preferred rather than have a central Saga place with Lots of dependencies...
@iirekm
@iirekm 3 жыл бұрын
Unfortunately the approach presented here is not perfect when comes to scalability: Order.state == PENDING is basically a database write lock in disguise.
@roshankumarmutha7389
@roshankumarmutha7389 3 жыл бұрын
It's better to understand the current
@danishrockz1
@danishrockz1 Жыл бұрын
how can I get that PPT ??
@maximecaron3133
@maximecaron3133 2 жыл бұрын
for peuple watching in 2023. don’t use saga! use a database that support multi/shard transaction!
@mariaguinsburg5092
@mariaguinsburg5092 3 жыл бұрын
Had to set it to 1.5 speed make it more consumable
@Mahorir
@Mahorir 3 жыл бұрын
At 8:37 , not monolith . It’s rather many monoliths
@allmhuran
@allmhuran 6 жыл бұрын
If only there was some way for a database to expose a consistent interface layer while still allowing the "private data" (ie, actual tables) to be modified "under the covers". Oh wait, there is. Views. Stored procedures. Functions. You do not need an application level API layer to provide data model independence. Views/procs/functions *are an API layer*.
@benjaminrood1648
@benjaminrood1648 5 жыл бұрын
Indeed. _You'll have to take the objects from my cold, dead hands_
@fsociety2871
@fsociety2871 4 жыл бұрын
This is somewhat anti pattern because if you call views, sps or functions, your service is now coupled with those APIs sitting on the same database of another service.
@allmhuran
@allmhuran 4 жыл бұрын
​@@fsociety2871 It's not an anti pattern, it's encapsulation 101. If you have a chain of "System A interacts with system B", it should do it through a public interface. If system B is a modern RDBMS then that interface can be made using views, procedures and functions. If you have a chain of "System C interacts with system B, and system B interacts with system A", then C is interacting with the stable API provided by B, and B is interacting with a stable API provided by A. Note that the actual technology implementation of each system is totally irrelevant. Any of A, or B, or C, might be a webAPI, or a database, or a file system, or a network. It doesn't matter. All that matters is that you don't interact directly to the internal, "private implementation" of that system. I think what you're describing is a situation where the system boundaries are not what I've described here. For example, we might say that "System A" is composed of *both* a database, *and* some kind of API layer. In this case, the database isn't a separate system, it's part of the implementation of system A. But again, the fact that the techology used for part of system A is "a database" is completely irrelevant to consumers. System A is already composed of lots of different elements (classes). Some of those classes expose public interfaces. Some of those interfaces are consumed by other systems. It could be that part of the public interface of system A happens to live in the database. To the caller it's totally irrelevant. As the designers of System A, we might decide to make the entire database private, and *only* have a public interface exposed at the dll layer, or webAPI layer. That's our call as the designers of System A. But no architectural principle is violated if we choose not to do that, as long as we are still encapsulating private data.
@fsociety2871
@fsociety2871 4 жыл бұрын
@@allmhuran But in the example you previously mentioned, a database object (view for example) to expose as an API let say of service A, if that will be used by service B which has it's own database then service B is now tightly coupled with service A's database.
@allmhuran
@allmhuran 4 жыл бұрын
@@fsociety2871 If that's the definition of "tightly coupled", then anything which depends on anything is "tightly coupled". You're basically saying "If A calls B, then A is tightly coupled to B". It doesn't matter whether B is a database, a webAPI, a network, a file system, an input device, or anything else, because the underlying technology is not relevant in a discussion about coupling. I think you want to say that the database is "private implementation" simply by virtue of the fact that it is a database, and a webAPI is a public interface simply by virtue of the fact that it is a webAPI, but there's no logical reason why the particular technology choice is relevant to the discussion. They're all just systems, and they can all have public APIs. In this example, it sounds like you want to say that if service B interacts with an API provided by application A's serivce, then this is somehow "better" Than interacting with an API provided by application A's database. But why? If interacting with application A's database is bad, then interacting with its service is also bad, so interacting with it at all is bad.
@songoku4271
@songoku4271 2 жыл бұрын
Why not use TC/C ?
@PietroYT
@PietroYT 5 жыл бұрын
talk starts at 15:16
@truongvo7466
@truongvo7466 5 жыл бұрын
Intro music name, plz ...
@alexanderbarvels7403
@alexanderbarvels7403 5 жыл бұрын
Daft Punk. Harder better faster stronger
@ram62836
@ram62836 3 жыл бұрын
10mins and still didn't found saga.
@DhrumilShahDOTin
@DhrumilShahDOTin 4 жыл бұрын
1.5 work for me .. :)
@DiamantineRakib
@DiamantineRakib 3 жыл бұрын
pro tips 1.25x sounds normal.
@joelmamedov404
@joelmamedov404 4 жыл бұрын
Will not work. The compensating transaction can fail also. In addition, when you commit (order created) then other processes can make decisions based on this fact. Those processes will perform some other business processes (transactions). Then , you decided to reverse order . But, the train left the station already. How will you rewind all other transactions?
@MaxPicAxe
@MaxPicAxe 3 жыл бұрын
Microservices isn't an architecture. Decoupling is. Microservices is nothing new. Decoupling has been around forever. (This is nothing related to the video, just felt like saying it.).
@jaydipdevkar4216
@jaydipdevkar4216 4 жыл бұрын
genius :-)
@chessmaster856
@chessmaster856 Жыл бұрын
This does mot work where ot maaters. It will eventially work where nobody cares.
@risebyliftingothers
@risebyliftingothers 3 жыл бұрын
I stopped watching when he says use database as a temporary message queue. WTF
@easyappscompany
@easyappscompany 5 жыл бұрын
(y) msilva
@122mlb
@122mlb 4 жыл бұрын
david blaine's father?
Microservices with Databases can be challenging...
20:52
Software Developer Diaries
Рет қаралды 97 М.
Yay😃 Let's make a Cute Handbag for me 👜 #diycrafts #shorts
00:33
LearnToon - Learn & Play
Рет қаралды 117 МЛН
СКОЛЬКО ПАЛЬЦЕВ ТУТ?
00:16
Masomka
Рет қаралды 3,5 МЛН
From Small To Giant 0%🍫 VS 100%🍫 #katebrush #shorts #gummy
00:19
Microservices are Technical Debt
31:59
NeetCodeIO
Рет қаралды 651 М.
Saga Pattern | Distributed Transactions | Microservices
17:18
Developing microservices with aggregates - Chris Richardson
1:09:50
SpringDeveloper
Рет қаралды 277 М.
Design Microservice Architectures the Right Way
48:30
InfoQ
Рет қаралды 716 М.
Authentication as a Microservice
50:26
Oracle Developers
Рет қаралды 217 М.
Yay😃 Let's make a Cute Handbag for me 👜 #diycrafts #shorts
00:33
LearnToon - Learn & Play
Рет қаралды 117 МЛН