Not Just Events: Developing Asynchronous Microservices • Chris Richardson • GOTO 2019

This presentation was recorded at GOTO Chicago 2019. #GOTOcon #GOTOchgo
gotochgo.com
Chris Richardson - Creator of the original CloudFoundry.com and author of 'POJOs in Action' and 'Microservices Patterns'
ABSTRACT
The microservice architecture functionally decomposes an application into a set of services. Each service has its own private database that’s only accessible indirectly through the services API. Consequently, implementing queries and transactions that span multiple services is challenging.
In this presentation, you will learn how to solve these distributed data management challenges using asynchronous messaging. Chris will share with you how to implement transactions using sagas, which are sequences of local transactions. You will learn how to coordinate sagas using either events or command messages. Chris will also explore how to implement queries using Command Query Responsibility Segregation (CQRS), which [...]
Download slides and read the full abstract here:
gotochgo.com/2019/sessions/717
RECOMMENDED BOOKS
Sam Newman • Monolith to Microservices • amzn.to/2Nml96E
Sam Newman • Building Microservices • amzn.to/3dMPbOs
Ronnie Mitra & Irakli Nadareishvili • Microservices: Up and Running• amzn.to/3c4HmmL Mitra, Nadareishvili, McLarty & Amundsen • Microservice Architecture • amzn.to/3fVNAb0
Chris Richardson • Microservices Patterns • amzn.to/2SOnQ7h
Adam Bellemare • Building Event-Driven Microservices • amzn.to/3yoa7TZ
Dave Farley • Continuous Delivery Pipelines • amzn.to/3hjiE51
/ gotochgo
/ goto-
/ gotoconference
#Microservices #CQRS #Events #APIs #AsynchronousMicroservices
Looking for a unique learning experience?
Attend the next GOTO Conference near you! Get your ticket at gotocon.com
SUBSCRIBE TO OUR CHANNEL - new videos posted almost daily.
kzbin.info...

Very comprehensive and clarifying. Coming from OOP, it helps me to understand sagas like this: - Decentralized Saga == Observer Pattern - Centralized Saga == Mediator Pattern

[@jleonardolemos]

One of the best presentations about distributed data, finally a very comprehensive CQRS explanation

[@fstaheli]

Great presentation! I've gone back and reviewed it a couple of times and learned new things each time I view it.

[@puravida77700]

Such a great talk. Thank you Chris Richardson, I learned alot.

[@kumarabhishek7877]

Chris, you are a great speaker, Awesome video with wonderful explanations, brings loads of clarity on mistakes that we have been doing.

[@sandeepsachan3729]

Excellent presentation and detailing about each concept. I love the most as one talk covers mostly design and technical design.

[@MuthuKumar-qi2um]

This was really a good presentation. Next, I've got to find more on failure scenarios in Saga & on creating, handling compensation processes and events.

[@funglee6030]

This is great talk! I think we used have DTO or VM but haven't relate it to microservices and CQRS which is a great use!

[@RiwenX]

Great presentation, it helped me a great deal. I think I'm beginning to understand why Temporal is such an amazing framework, too.

[@shenth27]

Excellent talk. He knows how to talk in a way people understand unlike some techies who just try to show off..

[@venquetgmail]

Good presentation close to enterprise system reality who worried about transaction management with MicroServices. Thanks.

[@lavpathak]

Excellent Session

[@pragashonlink]

Very comprehensive presentation on Messaging and architecting Microservices. Certainly answered some of my questions and verifies certain decisions I have made

[@DeepalJayasekara]

Excellent talk!!

[@desinenilokesh]

Wonderful session ,gained some great insights about Transaction management in microservice world. But one question I have is like in CQRS for Get/View purposes Chris mentioned we will have one more no sql schema like mongo something with same data. This will include whole lot of complexity in maintaining/deployment/cost perceptive right. Any other alternatives or suggestion to solve those complextities?

[@CarloL525]

I learned a lot. Great talk!!

[@AchillesGeng]

Great talk, very impressive, thanks

[@BryonLape]

A Postgres mention. w00t!

[@mikestock1848]

Awesome talk, very useful

[@andreid7655]

thanks. very useful!

[@jeffcrow1481]

Zeebe/Camunda for transactions., CouchDB for read model and db for mscs.

[@Om3gaz3r0]

This is a great talk and really useful for building things at scale. Now before everyone starts building systems like this, it's probably worth defining how far an ACID / SQL database can get someone before they need to implement this shiny microservice architecture.

[@XibaXela]

You can still use ACID / SQL databases in a microservice architecture, they just shouldn't span into different services.

[@XibaXela]

In terms of monolithic applications you need to weigh the pros and cons. - If scalability isn't going to be an issue, like you have few transactions, then the added complexity of microservices may not be worth it. - If you're developing a simple application. Microservices are needlessly complex. - If you want to launch quickly. Its better to get your proof of concept out there and validate your business idea (and make money on it). You can always iterate to microservices later. A good way to do this is with the API gateway Pattern which sits in between the consumer and your services. Which in a nutshell is where you route requests to the new microservice as you add them and route everything else to your old monolith

[@deepusasidharan2012]

Super good

[@WesleyChristelis]

Juval Lowy (iDesign and Righting Software) advocates decomposing by volatility, Chris here mentions functional decomposition. Which to choose ?

[@giovani5586]

Oh nice, I've just found out I've been doing the Transactional Outbox pattern. :P

[@drhdev]

When he was talking about semantic locking he said he had another talk that goes into it in detail 14:30. Where is that video?

[@Oswee]

I came up with a solution where i use websockets to update UI when projections are updated. But it requires some "gost" styling to give a user an indication that particular entity is still in update process.

[@briandopson5373]

FYI I tried to buy the book but got an error saying the coupon code has expired.

[@PietroYT]

I didn't get this 42:16. Does he saying that the event store is responsible for publishing the events into the message broker (kafka) once the event is stored?

[@pm71241]

Hmm... The argument around 17:00->18:00. One has to ask: How can the order service fail? If it is unreachable of course... but if it just has to acknowledge that req. and add it to it's own internal pending state, ... and then when complete, do an async callback to the service making the request. Then the req. to the order service is basically just a handoff of responsibility. Injecting a message broker is the same ... and it has the exactly the same problem: What if the message broker is unreachable.?? I not sure I see the argument for a message broker here. PS: And the model presented at 27:38 could just as well have been done with sync REST invocations. The customer service simply put "credit reserved" to the order resource.

[@DodaGarcia]

Microservices come with soooo many complications but I have a hard time imagining going back to having everything under a single monolithic system, where I can easily fall into the temptation of coupling So it's a bit sad that the reason to stick with microservices has nothing to do with something the pattern actually offers but with how it forces us to save us from ourselves

[@Qrzychu92]

If you think that microservices are not worth it, it means they are not worth it in your case :) and 95% of companies and products, don't need microservices.

[@mirekrusin]

You can force decoupling by putting code in separate repository as separate library/package. Also world is not divided into hot-dogs and not-hot-dogs (microservices and monoliths) - there is whole spectrum of modularity, dependency control, services width, runtime control (ie. supervisor/actor trees) etc. etc. As a side note please don't forget to prefix the whole complexity of this presentation with prelude "You can just do single line sql query or... ".

[@romanovfedor]

example on API Composition Pattern @31:06 makes little sense: why API Gateway would call Customer Service if customerId is already known as input param in findOrdersForCustomers(customerId)?

[@Klayhamn]

in case it needs some data from the customer service, like - maybe the user-preferences determine things about which orders should be hidden (e.g. maybe the customer chose to not display orders that are of the category "18+" (this is an example that came to mind but surely there are better ones...), or what currency they should be displayed in

[@PTM1008]

What's the semantic lock... Or is it semantic logs? I didn't get that part ... It's at 12:08

[@sergesemenov7856]

FYI: eventuate.io is for Java and the talk is great! dasync.io solves same problems for .NET in a language-oriented way instead of providing a framework.

[@ZeeshanAdil007]

hey, the link doesn't work. can you update with new one?

[@javadborhani7838]

at first, I thought he is Vision(Paul Bettany), why the color of his face is so red :) I got it. it's a reflection issue of back scene light which it didn't rendered properly by OpenGL fragment shader

[@vincentsels]

Is that other talk he mentions also available anywhere? Can't find it immediately: kzbin.info/www/bejne/oaqxfWqveNuEh8U

[@Oswee]

Solution to transactions is Saga pattern. As he states @11:00. Then what is Process Managers for?

[@redsquarem3369]

A saga in my mind has no state whereas a process manager is state machine based

[@rogercollins239]

These problems are easily solved with DDD: well-defined Bounded Contexts (customers shouldn't have knowledge of orders) and Aggregates (sagas)

[@BradleyWeston92]

Can't find anything on "symantec lock" for sagas, can anyone follow up?

[@gdargdar91]

semantic lock

[@gleniosp]

It's actually "Semantic Lock". For example, Chris posted an article here -> www.er.crichardson.com/post/antipatterns/2019/07/09/developing-sagas-part-1.html

[@kahnfatman]

Oh sagas! Like the Star War trilogy😮

[@christianwulf8224]

Mh, I do not understand why a workflow engine is a wrong approach. I think, Chris is wrong here. First, Saga is definitely complicated! It is not easy. Count the required concepts and the corresponding implementation parts that in addition needs to be correct: transactional outbox, unique message id, de-duplicating of incoming messages/calls, compensation actions, reverse order of compensation invocations, retries of remote calls, dead letter queue, resume execution, visualization of a saga execution for debugging. Second, take the workflow engine as implementation detail und hide it behind the Saga Dsl of, e.g., the eventuate tram library. Then, the workflow engine is not too much for such a 'small' problem. Instead, it is an ideal solution. It supports all of the concepts from above and requires minimal implementation and configuration effort from developers.

[@7th_CAV_Trooper]

This is a great talk except for the bit "I know we all want to use Kafka" - nope. We definitely don't. Seriously though, this talk helped me solidify my vocabulary around this complex topic.

[@dave4148]

I really don't get the obsession with avoiding synchronous communications, when this whole time a customer is sitting at a loading screen, or waiting for a confirmation email. I don't see the same advice given when talking about sending requests to a DB over a network, or to an event queue over a network, or to third party APIs over a network, etc. This is just something consultants say, knowing it will take a ton of billing time to implement.

[@jeffreymealo5215]

If you're looking to stream changes from your database, be it PostgreSQL, MySQL or MongoDB check out Lapidus: github.com/JarvusInnovations/lapidus

[@rimbik1]

You are going to Dubai Desert Safari and while in the middle of desert, u r seeking water bottle. - creating a order then reserving the money is same Case. go with buttle at first place. Sir

[@ArvindMishra2222]

1.5x

[@shawapratim]

1.75x

[@maksimsergeevich5939]

blm

[@Tony-dp1rl]

Wow, only 0.01% of developers will ever find this useful ... and most of those probably would have been better off with a different approach too. LOL.

[@antosha2224]

can i ask you of example of better approach please?

[@stevanhartig5573]

I think 0.01% is an accurate number if you include wordpress and js developers.

[@Gottii92]

i didnt know samuel l jackson was a dev

[@Gottii92]

apache starts *stares motherfuckerly*

[@timhosking]

Can't believe this CQRS rubbish presented here. Especially the example used. We solved this as an industry decades ago with the data warehouse.

[@Klayhamn]

Data warehouse loading still has latency in 2021 under normal conditions and unless you specifically construct something that *streams* a replication of your transnational database into your analytical database. So, for sufficiently complex data or for big enough data --- it's a trade-off between fast queries that show older state, or slow queries that show current state in any case CQRS (in his presentation) demonstrates how to solve the transnational READS in a natural application flow that is split into micro-services, NOT JUST analytical aggregations over masses of data

[@opelfahrer91]

You just didn't get it

[@djgreyjoy1495]

Why does the business logic of a business transaction "belong to the services" and not to the business transaction?