Event-Driven Microservices, the Sense, the Non-sense and a Way Forward • Allard Buijze • GOTO 2019

This presentation was recorded at GOTO Amsterdam 2019. #GOTOcon #GOTOams
gotoams.nl
Allard Buijze - Creator of Axon Framework
ABSTRACT
Microservices, and especially their Event-Driven variants, are at the very peak of the hypecycle and, according to some, on their way down. Meanwhile, a large number of success stories and failures have been shared about this architectural style. How do we ensure that we don't throw away the baby with the bath water and end up re-inventing the same concepts again a decade [...]
Download slides and read the full abstract here:
gotoams.nl/2019/sessions/862/...
RECOMMENDED BOOKS
Sam Newman • Monolith to Microservices • amzn.to/2Nml96E
Sam Newman • Building Microservices • amzn.to/3dMPbOs
Ronnie Mitra & Irakli Nadareishvili • Microservices: Up and Running• amzn.to/3c4HmmL Mitra, Nadareishvili, McLarty & Amundsen • Microservice Architecture • amzn.to/3fVNAb0
Chris Richardson • Microservices Patterns • amzn.to/2SOnQ7h
Adam Bellemare • Building Event-Driven Microservices • amzn.to/3yoa7TZ
Dave Farley • Continuous Delivery Pipelines • amzn.to/3hjiE51
/ gotoamst
/ goto-
/ gotoconferences
#Microservices #SoftwareArchitecture #EventStreaming #EventDriven
Looking for a unique learning experience?
Attend the next GOTO Conference near you! Get your ticket at gotocon.com
SUBSCRIBE TO OUR CHANNEL - new videos posted almost daily.
kzbin.info...

[@SerpentsHiss]

None of the cons or pitfalls of Event Sourcing are laid out in this presentation (and there are many, especially as your architecture grows), and anyone trying to follow this video would do themselves a favour and read up on these, before beginning a very extensive and potentially wasteful exercise.

[@chmart]

This talk is somewhat weird tbh. First events are explained which makes this the 1000th talk to do so. Then the talk is about event sourcing without really explaining what it is at first. How all of this fits into microservices in the first place is not discussed it seems. Imo it's a talk with some random bits and pieces of everything without a common thread or conclusion and not having a consistent picture of the expected basic knowledge of a target audience in mind

[@arvi8843]

Thanks. Do you have video recommendations related to microservices? 😅

[@humbled_pleb]

It seems heavily "inspired" from a Martin Fowler goto talk on event based architecture

[@slotrans17]

If you view the second half of the talk in light of the first half, it's clear that Event Sourcing and CQRS are seriously advanced architectural tools that you probably should not use until you have mastered the earlier steps of evolution... or perhaps not use at all. The talk would be more coherent overall if it consistently emphasized the negatives of these fancy architectures instead of devolving into Event Sourcing + CQRS advocacy. That kind of advocacy is a failure mode endemic to talks delivered by consultants. In particular the dismissal of storage costs and implementation complexity with respect to event sourcing is way off base. Storing events can consume 2-3 *orders of magnitude* more storage than storing state. This is not trivial! And he dismisses implementation complexity by talking about implementation *difficulty*, which is not the same. Complexity is about the braiding together of different parts of the system, which is a major problem here, and which we cannot wave away by saying "you'll get used to it".

[@LusidDreaming]

If storage costs are your main concern, how much data do you have? Data storage costs are almost always trivial compared to the value of the data you're storing (especially considering the value of an event log over simply storing the current state). I think most of the complexity people attribute to event sourcing is actually about distributed systems and cluster based scaling. For example, you don't have to use Kafka as a message broker. You can scale to a fairly high degree using a relational database and a polling mechanism. I've built both CRUD applications and event sourced applications and, although event sourcing takes more upfront work, the event sourced applications are far easier to maintain over time. The one thing I will concede is that functional programming can be a difficult paradigm shift for people who have worked exclusively in the OOP world. But it's honestly easier to understand (imo of course) how wrapping a function inside a function works compared to the decorator pattern in OOP (and most of the common patterns are simpler to write and understand in a functional style). But everyone is different so I understand other people may disagree. But after building systems in both styles, I find event sourcing to be my preferred style.

[@TheEnfernuz]

I've found this presentation to be useful and entertaining at the same time. Good job and thank you, Allard!

[@tyrotoxin]

No like, no dislike. 1. The knowledge is 2 years stale for me - have been designing services with queries, commands, events, and CQRS. 2. I liked the critics of choreography. The real example is GitHub demo project called 'eShopOnContainers' (by Microsoft). Yuck! 3. Nouns are not bad. Do Event Storming to define functional aspects of your app, put them in boundaries and pick nouns. 4. Don't call other architects stupid. The strong language may sound funny, but also can be offensive. 5. The problem of deploying multiple services at the same time lies in the contract, but not the absence of modularity. Introducing a new feature sometimes require changes to multiple services. This is normal. However, if you break a service contract and don't provide backward compatibility, then yes, update and deploy dependencies. Completely isolated from each other services is an illusion.

[@karpuzye]

temel bilgileri uzatarak ve karisik bir sekilde anlatarak doldurulmus bir sunum. Konu ile ilgili temel bilgisi olmayanlara tavsiye etmem. Iyice kafalari karisir.

[@strangnet]

The anti-corruption layer that transforms the events made no sense at all.

[@allmhuran]

If system B receives an event notification from system A, and then has to query system A in order to complete its process, you've lost your scaling performance advantage (because the functional operation of system B still puts read load on system A), and you've lost your availability (system B cannot complete its function if system A is down), and you've lost a small degree of your decoupling (system B at the very least needs to know that system A exists, and what its capabilities are). Also, the concept of transmitting domain events is a little smelly to begin with, because surely system A and system B are working with different bounded contexts. So whose domain do you use when raising this domain event? The most obvious issue with event sourcing is never mentioned in event sourcing talks: read performance. If I have to calculate the current state based on history, then that is a much larger space of data to query, and there's potentially a lot of logic that needs to be run in order to get the result. Sure, you can take snapshots, but that's just a mitigation of the problem unless you create a new snapshot for every change. And sure, CQRS says that is perfectly legitimate - separate the read model from the write model. But if you do this, your event store is nothing but a backup. Your concurrency issues return (because you have to write data into the "current state" read model for every event), so there's no longer any write speed benefit. And what about all the enterprises running systems they bought "off the shelf"? If you bought a system which uses a relational database back end, you need to populate it. You can't just tell the vendor that you're moving to event sourcing and expect them to completely rewrite their data access layer just for you, using your persistence model instead of theirs.

[@Belliger1991]

You are somewhat right. But remember the start. The consumer knows the producer. No longer the produce knows all consumers. This is what is solved with this aproach. Events today also have a data payload since its not "i just did x" but " i just did x with the result y" therefore helping other services save computation (by paying with storage) but relieves A from the read Load. Or what we often use is simpel caching (since microservices should handle their own data) snd partialy invalidate/update the cache based on the events. What i read in your comment ist mostly that you may splitt your microservices to eager and could benefit from recombining those high read intensity services (because of the super high coppling in the data) But event sourcing in my problem space allows me to scale super fast and grow the storage more dynamically since most data is longterm i dont need it fast on hand and therefore can afford to keep much in memory and numbercrunch the specific events in the few cases needed. Additionaly the backporting of new features is worth its weight in gold. So basically not everything is a nail ;) EDIT. Your of the shelf product problem is easy too. Just integrate the api in a minimal service handling the events you whant/need. Not everything needs to be fully event sourced. You just need a way to translate it into events and back. But IF you realy need a vendor software to integrate that hard into your system you realy realy need to ask yourself WHY. (... What do we realy gain? ... Are we relying in our deep core on a not relyable vendor? .... Are we pushing event sourcing to far out where apis should be?)

[@allmhuran]

​@@Belliger1991 The consumer doesn't necessarily know the producer, and in fact should not, since one of the desired advantages of message orientation is to remove the direct coupling between systems. You can certainly include more data in the payload, but that's an ECST model, not an event notification model. And the problem with domain events stands: whose domain? I am currently building a pub-sub ECST message architecture for distributed systems using solace pubsub+, and we use a "common data model" shared data structure, which precludes "domain events". Instead messages are really just transmission of change data to be interpreted by each subscriber as applicable. This eliminates most of the issues I described above, at the cost of having to define a CDM.

[@Belliger1991]

@@allmhuran glad to hear someone knows his stuff :) You are right, it is an ECST model if you add the payload. But since data has to be transfered you eiter know all consumers and push, know the producer and regularly pull, know the producer and pull on event or use the "independent" ECST model approach :) i guess its a pick your poisen ^^, since ECST, if you don't version those and watch out for backwords compatibility, can bite hard in your data consistency and trustyness of data availability down stream too.

[@allmhuran]

@@Belliger1991 In our design both the producers and the consumers only know about the broker. Producers push to the broker on one or more topics (topics are defined by the common data model), and other systems subscribe to those CDM topics. Solace uses a push-to-subscriber model, so subscribers don't need to poll for data. The data structure of our messages are transmitted as protobuf binary, which is a good format when it comes to versioning messages. The protobuf.net serialisation library will handle events as long as the attributes are not actually removed from the message (they would simply be deprecated and only actually removed after a long time). Additional attributes can be added, and consumers or producers making use of an earlier version of the model that does not include those attributes will simply ignore them during serialisation/deserialisation. In this model the complexity arises when translating to and from CDM data structures inside the anti corruption layers. Translating-to is actually harder, since a change to a single producer entity may not equate to a change to a single CDM entity. It could be n-to-1 or 1-to-n, but for this reason I specifically design the CDM entities to be narrow, such that the relationship is typically 1-to-1 or 1-to-n from producer to CDM. This is much much easier to handle than n-to-1, because if you have n-to-1 then producing a CDM change is much more complicated - it becomes an "or" relationshp across publisher entities: "If A changed or B changed or C changed then construct the CDM entity out of the bit that changed plus the bit that didn't change, if any". That's pretty hard to do. Much easier to say "If A changed then B (and C and D...) changed".

[@jukkanikki3395]

I didn't get much wiser. Too broad, marketing like.

[@FrankKrasicki]

Modeling ideas using iconic notation is indispensable. To begin a talk mocking the idea of modeling systems or solutions using iconic notation and then explaining yourself using that same notation is just plain silly. This is a talk full of platitudes and popular put-downs that add no real value to the discussion. In fact the discussion, such as it is, rambles around the architectural territory pointlessly. As a George Harrison song quips, "If you don't know where you're going, any road will take you there" (how's that for one upmanship?). I could not identify Maslow's Syndrome - there is however a Jonah Complex Syndrome associated with Maslow that fits perfectly in a presentation that is littered with random bits of pseudo-wisdom.

[@pengdu7751]

a geek explaining to the world in a way the feels cool himself. lots of fluff.