Design Microservice Architectures the Right Way

[@zhou7yuan]

A personal story (change API URL) [0:13] How does this happen? [0:38] Great Architecture [1:10] Not So Great Architecture [1:55] Design Microservice Architectures the Right Way [2:29] About Me [2:45] Let's Start With a few Misconceptions [3:37] M#1 teams to choose language [3:40] (expensive) M#2 code generation is evil [4:32] (just a tech) M#3 event log must be the source of truth [4:55] (ok to record service own record) M#4

[@commonsense9438]

holy shit

[@xskrish]

you're a god

[@blackerhawk1508]

This is awesome

[@FromNothingComesNothing]

Of course it's not a KZbin comment section without heroes who put timestamp for sections

[@YouToYoub]

This the best presentation I have watched my entire professional life of 22 years. And the best of the 85 videos I have watched on microservices.

[@mbryzek]

Thank you! Really appreciate your note

[@jonnytheponny5753]

Thanks for your comment. I was only 3 min into the presentation and about to leave, as I already had seen so many Microservice presentations. But I thought first lets look at the comments. And you are so right, thanks very much!

[@jamesmcmurtry5351]

@@mbryzek I have to second eJay's comment. This answered so many decision questions on utilizing microservices along with event driven architecture in examples of real world success. There are too many talks with buzzwords and not enough practical information on implementation. A great question you answered was using individual databases per microservice that is subscribing/publishing to an event. The "source of truth" narrative has been a bit ambiguous with regards to the rules of using other DBs.

[@mbryzek]

@@jamesmcmurtry5351 Thanks James - Really appreciate your comment!

[@bobchemist]

@@mbryzek Great talk. A good followup would be on the optimal microservice network for various use cases.

[@dhiresh5980]

One of the things I learnt in this video: Code generation can be used to set strict standards across the systems

[@biplobmanna]

I know this comment is 4 years too late, but the sheer volume of things I've learnt from this talk tops all the videos/tuts I've watched regarding microservices without debate.

[@akinniyiakinyemi2737]

I've watched a lot of videos on microservices. This is by far the most sensible, brilliant and cohesive explanation of all! Should be made into a textbook tbh

[@jaikantchandrakumaran3423]

I loved this talk, not because it is the gold standard but because Michael knows what he is speaking and has taken the efforts to setup the standardisation in every step of the process.

[@Mia12946]

this guy either was really prepared or a savant with his incredible train of thought

[@musicandoutdoors]

The way I view this talk is as a representation of the “perfect workflow” for microservices. In my small startup, most of it is “good to know” but I can see adopting things more and more as we grow. Particularly helpful for me was understanding the journaling event publishing aspect. The code generation stuff is mind blowing - I would not have thought to automate that much. I wonder how long it took to set that stuff up before they even shipped any product. Yes, now they probably develop lightning fast, but this would be a huge distraction from finding product market fit.

[@mbryzek]

most of our code gen started with a 2-4 hour "hack" that was sufficient to solve the use case at hand; then we built on that to add features over time. We did this as a startup - definitely doable!

[@volshebnikozz]

I haven't used it, but you can take a look into Atomist: docs.atomist.com/ they're doing something similar

[@Hex72]

@Robin Chan I was thinking the same. What if the generated code needs to change? Are all your manual changes and implementations lost?

[@hibas1235]

@@Hex72 I would say there should be no manual editing in the generated code. When I build a code generator it always produces interfaces or abstract classes where actual manual work is required. Everything else like Parsers etc. Should not be touched by anyone except the code generator itself.

[@Hex72]

@@hibas1235 That would make the most sense, yeah.

[@devrkd]

In the entire presentation most of time, it not slides with some block diagrams but real code examples. I really enjoy that. Creating/ investing into such discipline to standardise stuff, you got my respect man.And indeed the code was really beautiful.

[@1173mrwan]

Their DevOps game is on point

[@KAUSHIKAKALI]

This is the most practical presentation on the quote "Talk is cheap. Show me the code". Finally found something practical otherwise those theoretical rhetoric.

[@wenhoujx]

i watch this video again every year, and every time i learn something new. such a treasure trove. Would love to work with Michael someday, he seems an awesome engineer and an emphatic learder.

[@attilaviniczai7215]

Great software design material! It not only covers microservices, but also the devops operations associated to deliver quality software.

[@damejelyas]

these people can rebuild the internet if it goes down

[@KelvinMeeks]

Excellent talk - in particular, note the advice @24:25 - adopting a microservices strategy requires Continuous Deployment

[@dmoscrop]

Wow, we've basically built this exact thing over the last few years. It's nice to see this kind of validation.

[@thesobercoder]

My life feels like a lie now when I think about the fact that I know how to write microservices. Absolute gold standard!

[@SatyanarayanaBolenedi]

I would say, these are Gold Standards of MicroService Architecture From code generation till deployment. Every stage is well designed!! Thanks a lot Michael Bryzek, you made my day!!

[@mbryzek]

Thanks!

[@positivetech]

really amazed by - "6 lines of yaml" great effort by folks who automated the default setup.

[@GamerMinecraftivity]

It's funny, I keep coming back to this video after studying up on microservices and understanding more and more of why these are good design principles. Great video

[@garygreen8699]

Amazing architecture story. One of the best tech presentations I've ever seen.

[@bahadiryagan]

Great talk. Most tech talks nowadays are passing time with things like "show me your hands, who is using kafka". But on this one, I actually had to pause a few times to digest what's been said.

[@y.5107]

Same for me. He seems to be a clever guy. Especially when he corrected the guy asking a question about feature development. CDeployment vs CDelivery are two different things. This little examples shows that he knows what he is talking about.

[@peterbodifee]

This was an excellent talk (thank you Michael) and he is the living example of how to execute on the principle "if you need to do something more than once, you need to automate." Automation is so much more predictable then humans ;-)

[@dhanyajayachandra8645]

I could like this a million times over. The best presentation on Microservices.

[@ChidambaramVelayudham]

Really a great presentation. Michael is talking what he has done in his real life. I was surprised he is not using Kafka for streaming and adopted a different approach. Thank you a lot!

[@bobchemist]

Kafka is way more efficient and fast, but can be fussy in production. Kinesis is easier to manage on a global scale because it is so well supported by AWS.

[@nsisodiya]

One of the greatest presentation I saw in the tech world. Really a lot to learn from Michael. I have hugely invested my decade in frontend world and now when I am digging in backend worlds, I see, I can invest next decade in it.

[@giulioambrogi5413]

So much content in such a short time, great presentation.

[@andrew5222]

I hope everyone understands how brilliant this is

[@RichardBuckerCodes]

There are several things wrong with the second half of the presentation: - how did they decompose the GRPC messages to be analogous to the API with it's personal DBs? - what happens when one microservice is dependent on data in a different API or event? - if there is some reporting that needs all the DBs then how is that performed? - if a transaction requires more than one API and there is an upgrade/deploy mid transaction how is repeatability supported (see monkey patching and erlang live code replacement) ... good enough for now.

[@elnur0047]

have you found answers to those questions? those are exact things I was wondering

[@RaMz00z]

- Not sure I understand, but if I do, everything is described in the API schemas, even the form of the objects used in the APIs - Local copies. The rule of thumb if I've understood is that *all* the data you need is in your own db. If you need to add new data for a new service, and you don't own that data, then you subscribe to the events from the owner, and you duplicate that data into your own db. - Events. If every single service emits upsert events, then you can have a reporting system that consume them all and aggregate them in a big db (at least I'd do it that way) - There is no transaction in micro services. First, the probability of that happening is (and needs) to be extremly low, with parallel and hot deployment. Then if it does there is two situations for me : - The event that failed can be replayed because no break in interface was made : he talked about that, you can just replay it, automatically or not I didn't understand that part. I'd say automatically. - The event that failed *cannot* be replayed because of a breaking change in the interface : then you'll need some way to create the new version of all the failed events in the system, and then I don't really have an answer because he didn"t... Note that breaking changes in event is the biggest challenge (by far) of an Event Driven System, so I guess it must be tough for them as well :/ I hope I helped and that my anwers were correct :)

[@hansteam]

Wow! Thank you so much. This talk is going to drive a lot of decisions in my upcoming microservice architecture.

[@anandbgrowthcoach]

One of the most amazing talks about Microservices Architecture - You guys are geniuses and thanks for sharing all that knowledge!

[@KaranPratapSingh]

This is one of the best talks on microservices

[@benwho1549]

Realworld Microservice example, amazing!

[@guibirow]

First of all, Nice presentation, thanks for sharing. IMHO, The services he described are plain CRUD operations, this is why this approach works well. I've done similar solution in the past where I would design the database schema first and generate the rest of the service, it worked like a charm for CRUD services. DDD systems on the other hand are much more complex, will be much more complex to write business logic just using DSL or schemas like he demoed. The code generation would help a lot though.

[@gonkula]

Thanks, perhaps would you mind nudging us in the direction of what to look for when solving similar issues in DDD systems?

[@guibirow]

@@gonkula DDD systems are designed around business rules, which most of the times are checks and validations happening before an operation is applied to an entity, these are generally designed as workflows with multiple stages that are updated gradually on each user/system interaction. I've seen solutions in the past trying to auto generate code based on business rules, but the amount of code generates and the complexity for maintenance doesn't worth the investment.

[@guibirow]

If you are just planning to do DDD, a good example I can share is this: docs.microsoft.com/en-us/dotnet/architecture/microservices/microservice-ddd-cqrs-patterns/net-core-microservice-domain-model

[@blarvinius]

Rewatching for 4th time. Will continue. 👍👍👍

[@driziiD]

i'm really impressed by that rds dev cli i want one

[@Alij47]

Great presentation. Very informative and great design lessons. It also represents great team collaborations and good culture at Flow. I wish more companies could adopt such innovative ways of working.

[@foxford4981]

Great presentation, but i have to admit, reading your comment that its great, i started to think its a very good guideline to quite specific practices when developing a microservice system. Not as an microservice architecture as a whole. Not complaining for sure, i learnt a lot from it, but it seemed like it focues on the micro elements of it XD Not the whole.

[@sanjaig]

Fantastic!!! Packed with info and best practices! Thank you!

[@BarZiony]

Awesome talk! Thanks Michael. Can you perhaps elaborate a bit on the different main microservices that you have? What does the users microservice is in charge of for example? How do you ensure data consistency between multiple services that each save their own local copy of the same data (a user for example)?

[@sushantnp]

I came here to learn microservices. He is teaching me how to code, and I never felt dumber.

[@I_dont_want_an_id]

May this encourage your faith facebook.com/esther.chester77/videos/1029971633787995/ kzbin.info/www/bejne/o3rbfJ9vgL6qhLc m.facebook.com/story.php?story_fbid=2582272895211292&id=490097534428849 m.kzbin.info/www/bejne/e3aVpH1jjK6ZqtU kzbin.info/www/bejne/h6iVmaqZpMiJrK8 m.facebook.com/story.php?story_fbid=777934336005747&id=100013675634259

[@vekzdran]

So... onboarding a new developer to all of that custom built stuff takes time to pick up and is very specific for their build / devops case. Its a fun talk and I like his delivery style and self-confidence in it. I am amazed they built stuff on their own and did not use existing stuff. Also those services are super thin from what I got the idea but then yet they have databases. Being unexperienced in this much code gen I can not say it is that worth it but again, hey they deliver and auto gen those e2e tests. Cool talk especially as I never did java or play professionally.

[@stack.1]

I'd probably watch this several times over

[@danaadalaide5648]

Lots of useful information here, i've always been into defining metadata around data structures - i had written the same kind of thing about 7 years ago where classes were automatically generated from the schema, which was defined in the database with standard types for text, email, textareas etc. but what i find most interesting in your talk is grpc; its able to not only generate data structure templates but also has client and server libraries for creating a service out of the box.

[@cas1652]

I love the way this guy approaches developing: schema first, strong types, end to end tests, strong tooling, expressive language (scala). All of it together creates a virtuos cycle where each iteration carries you upward. I do wonder about schema changes and the event bus though. What happens when Service A puts a changed User onto the queue when the consumers haven't been yet updated. I also wonder about transactionality because at my job we sure seem spend a lot of time trying to tack on stuff that looks a lot like commit / rollback to the event bus.

[@darglo]

I am blown away! Great video! Thank you!

[@geoffreyhibon2651]

Great moment to watch this presentation :)

[@go_better]

Very cool presentation! Lots of practical tips to use.

[@vishalsharma-bp9zu]

This talk was absolute best and absolute gold. Thank you.

[@fyrofux]

I might be wrong (and also a bit inexperienced) but this talk seems highly focused on Designing a Microservice Architecture **for the cloud**. Most of it seems to be great advice but still requires some deep though on "how do I adapt this to my own environments?" for example their Delta project would not work without a cloud environment (unless?).

[@sagechris4742]

crack is wack

[@RaMz00z]

Cloud is just a deployment detail. It could work *exactly* the same with private cloud or even Virtual Machines. Especially since he mentioned docker images. It is after all a standardized way of starting an app. So deploying it to the cloud, private cloud, VM, or your desktop is just a matter of wich command to run, that's all.

[@MrNorthCat]

Great presentation! Must to view for all practitioners!

[@santoshdl]

absolutely love this. thank your for posting this. Keep doing good work and keep sharing.

[@Alperic27]

... but maybe the developers themselves are generated clones...

[@awright18]

I think I'm going to have to have to borrow heavily from these ideas. This information is pure gold. Thank you so much! Just what I was looking for.

[@mbryzek]

thanks!

[@yahorsinkevich4451]

omg dev cli is an AWSOME idea, thanks for that :)

[@Euquila]

agreed

[@mbryzek]

thanks!

[@Laggie74]

Excellent. But I think this just highlights why you shouldn't build a microservice architecture if you don't need it. Especially if you are doing a small to medium size project, with few developer resources. Until the day when you have some standardized tools that allows you to pump out services/components in your tech stack, that just works out of the box.

[@zHqqrdz]

This is 100% true. Please, to all devs scrolling through comments trying to find reviews of smaller companies who tried using microservices : don't. Just build a monolith, make it clean, make it work. Have tests. Have a clean workflow. Once you're there and you start to feel the need to extract a part of your app into another project, extract just that part, nothing more, nothing less. It takes a crazy amount of work to succeed with microservices on the long term, and it also requires a great deal of skill. Let's face it, all developers aren't equally competent, and most have more experience working with monoliths (for now). So just write good monolith code, ship on time, and you'll still be doing better than most.

[@williamvankeulen1307]

I love this talk. So useful and so practical.

[@atulverma7783]

very well presented especially focus on tooling

[@JoaoKunha]

impressively simple, well done

[@rahulgoti3864]

Thank You

[@granand]

Hey experts, Please advise where to start and what software are needed to practice some projects from start to end with intention to know to design APIs, Test APIs please. Can I accomplish this with single laptop I have ?

Nice talk, thanks! Even if it's old, I have a question: how do you/they automatically create git tags. I guess they are following semantic versioning or something similar. How do you/they know automatically the version to increment?

[@barefeg]

It could be done by standardized commits. Given that most things are standardized I wouldn’t be surprised if that’s the case.

[@averasko]

the approach is limited to the situation when you actually CAN devote some time (like 9-12 months) to bring up the infra and the start coding aggressively. in most cases, you cannot.

[@KashifMinhaj]

Rightly said. It takes years to get there. In most cases you're better off managing stuff on your own. Automation comes with its own headache.

[@scotwilcox1771]

Manual work vs automated work is still just work. It's just stuff that needs to get done. Either you spend the time to make the computer do the work for you, or you're stuck doing it forever. The real benefit to automation is consistency across teams. That's the main takeaway from this talk: the only way to have consistency is with automated tooling. If you don't have very many developers and can't spare the time to maintain that automated tooling, because it does take maintenance, then it's not worth it to you. The larger your team and product, the more important the tooling gets.

[@WayneMunro]

Code must be generated from a well defined model of the in context of the service. Almost anything can be derived from the model especially SOLID code. The question is how best to define a validated conceptual model up front in a data platform and language agnostic way that conforms to some first order predicate logic that proves the correctness of the facts expressed in the domain.

[@kappaj01]

Awesome presentation sir! Well done.

[@mbryzek]

thanks!

[@ajitkulkarni7020]

Brilliant one @mbryzek. Absolutely loved watching it. Lot to learn from this. BTW, anyone here knows a similar implementation already available for Golang/java?

[@giovannimarazzi8737]

what are the best resources for learning software architecture?

[@JGB_Wentworth]

Wow, who ever would have guessed that Topher Grace would have gone on to be so successful in the software industry after That 70s Show?!

[@antonlogunov1936]

Agree, The Right Way. Isn't that much and difficult. Microservices rule!

[@sjogzor]

45:08 Haha! Love this part. How much time, you ask? A lot! And a lot of anxiety too :D

[@andreylebedenko1260]

Microservices are required to extend the DB business logic in some way that can not be easily implemented in the DB language.

[@chessmaster856]

How are complex queries implemented in microservices?

[@khatuntsovmikhail6223]

How you deal with states and stubs in integration tests: you create user and you validate that user been created? web stub should be simple but state make it complex...

[@raarky]

great presentation. Thank you

[@mbryzek]

thanks!

[@djkim24601]

Man, this guy is a monster.

[@fakeapplestore4710]

Great speaker and even better content.

[@NastyaSmirnova]

Great talk! thanks

[@rajeshkishore7171]

Excellent

[@aloussase]

What is the API documentation standard that he is using?

[@bephrem]

beautiful talk

[@bastiengerard433]

Fantastic video, really inspiring!

[@joaopedrobourbon7483]

Brilliant talk, very informative

[@0xccd]

What a great talk!!

[@VishalGupta9]

I did not understand his point about creating indexes at 22:54 . He mentions that they would catch the developer from trying to create an index before they do, but what is the advantage and why should we appreciate whatever he is trying to say. If someone can clarify, please?

[@vectorhacker-r2]

What he means is that the developer shouldn't have to think about something like an index on a column, when they could have just worked on better tools so that the dev can focus on more important things, mainly the application itself. His point is that developers shouldn't have to waste time on trivial tasks or tasks that could have been solved with proper automation and code generation.

[@sagechris4742]

Gupta, plz shut up and go cold call ppl. GUPTA!

[@mbryzek]

Great question and thanks for asking. What I have seen happen all too often is a situation where the "application becomes slow" and then is "fixed" by creating an index. This ideally should never happen and be instead part of the development process. By using code generation that provides methods or parameters based on the fields that are indexed, you can significantly reduce the probability of this error occurring because the indexes will be there from the start.

[@hommeoursporc5874]

This is SO interesting !

[@crogersdev]

If we have the API, why favor the events? He must be talking about internal clients and not public ones, right?

[@gravisan]

What is the fastest way to deploy 'Galactic the all knowing user service provider'?

[@vishalsh1624]

Love this talk! Loads of insights

[@DitaASubrata]

wrote tests for production and ran in cron job. omg never thought of that.

[@nisimjoseph]

WOW! Amazing talk.

[@mbryzek]

thanks!

[@smakarevich]

Cool guy. Great speech!

[@johnboy14]

Great presentation but doing this in established companies with layers of beaucracy is incredibly hard. If your building a platform from scratch, then this is a nice way to go about it. The big takeaways here is the power of code generation and schema management. The way its described here makes so much sense. Continous delivery on the other hand only pays off if your testing is exceptional, else its just a quicker way to break things and lose money. The idea of pushing code to production on a Tuesday afternoon before you leave for the day would give me the jitters. That PR can definetely wait 😁

[@gaatutube]

Every service having its own database sounds good in theory ... will be a nightmare in practicality. DB's are not light weight components. They can be pretty heavy if you use stuff like Cassandra, MariaDB, ES etc. You do not want 20-30 instances of DBs running that extinguish all your RAM. A better alternative is for each MS to have its own isolated namespace in some common DB cluster. It's effectively equal to each service having its own DB without running multiple DB instances.

[@Daxten]

He's talking about Databases, not Database Instances I would say. He just says that only 1 Service should Read/Write to its namespace (for example Database, or Filestorage). Like he said in the beginning, for him as a software engineer he doesn't care about the specifics of the hosting (are these on the same instance?), that's for others to decide :)

[@gaatutube]

Thanks for clarifying. If he meant that ... good. I probably misunderstood his view.

[@DerkJanKarrenbeld]

It is actually a database instance he is talking about, but that is not the point he is making. He is saying: make sure that all reads and writes are only through the microservice so that all the contracts as defined by the schemas are always applied and checked. Yes you can have a database instance that has multiple databases and yes you can put the separation on the table level but the idea here is that only one service can read or write through a direct connection. So a row is owned by a service, a table definition is owned by a service.

[@EduardoRFS]

actually RAM is cheap today, and most of services gonna use really little data. So having a DB instance for each service isn't crazy as it sounds, it's not cheap of course. But If you want to be cheap, just use namespacing

[@reheitube]

Is the way to go man, without having an isolated storage it will be a mess in microservices... true story.

[@OwenWilliamsRobotics]

What are you using for your schema definitions? Your format looks similar to Apache Avro but it's not identical. Also what code generating/templating engine are you using?

[@Bryanerayner]

@Chris ter Beke You can host apibuilder in AWS it seems....

[@spagon]

bravo

[@tomwright9904]

Are there any open source tools to work with these APIs?

[@sergeiromanoff]

1.5 weeks - we all know this story. It never ends with your initial implementation

[@kennethcarvalho3684]

Regarding Misconception No 3...are you saying that to create a resource in database you set up an event via a microservice and then that event was processed by another microservice and you were waiting for the response, which naturally you found not feasible. After that you got the solution that you directly saved the resource in your database via your microserver and also put in an event stating that this resource got saved via your microservice. Is my understanding right?

[@zHqqrdz]

Yes

[@AjayKumar-fd9mv]

Great

[@amkng82]

Wow, isn't that super expensive? A t2.small for every microservice?!

[@saturnusstudios]

Dude its $0.026 / hour lmao

@@saturnusstudios that's $1900 per month if you only have a single instance of each service, given that the application is composed of 100 services. That's not necessarily cheap.

[@mr.s7767]

@ If there are 100 micro services, then it's not a fun project or a small project. $2000 per month is considerably cheap for that size of a company.

[@tobiowolawi5044]

i'm sure they use caching somewhere at some point.... but he never mentioned it