Monolith vs Microservices

Monolith vs Microservices - which should you build? As ever in software development, each has its benefits and its downsides. What is a monolithic architecture in this sense, and what are microservices?
One useful perspective on this is what is the scope of change that you need to evaluate before release, and how easy is it to make, and evaluate those changes. To put that another way, if we think about this from the perspective of version control, should you store your changes in a single mono-repo or in multiple-repos?
In this episode, Software Engineering expert Dave Farley looks at this problem from the perspective of Continuous Delivery and explores some of the trade-offs, and while he’s doing that he rules-out a very common, but wasteful approach to SW design and architecture.
_____________________________________________________
📧 FREE MICROSERVICES 'HOW TO GUIDE' 📧
Sign up for the CD mail list and get Dave’s free guide on How to Get Started with Microservices
➡️ www.subscribepage.com/microse...
-------------------------------------------------------------------------------------
🚨 My "TDD & BDD: DESIGN THROUGH TESTING" Course is Available NOW! 🚨
Learn to write great tests, and how to use those tests to improve the design of your software: with step-by-step guidance and demos by Dave Farley, and practical exercises for you to learn TDD and BDD. Available to start now!
ENROL HERE 👉 bit.ly/3JB5smY
_____________________________________________________
📚 BOOKS:
📖 Dave’s NEW BOOK "Modern Software Engineering" is now available here ➡️ amzn.to/3DwdwT3
In this book, Dave brings together his ideas and proven techniques to describe a durable, coherent and foundational approach to effective software development, for programmers, managers and technical leads, at all levels of experience.
📖 "Continuous Delivery Pipelines" by Dave Farley
paperback ➡️ amzn.to/3gIULlA
ebook version ➡️ leanpub.com/cd-pipelines
📖 The original, award-winning "Continuous Delivery" book by Dave Farley and Jez Humble ➡️ amzn.to/2WxRYmx
NOTE: If you click on one of the Amazon Affiliate links and buy the book, Continuous Delivery Ltd. will get a small fee for the recommendation with NO increase in cost to you.
-------------------------------------------------------------------------------------
CHANNEL SPONSORS:
Equal Experts is a product software development consultancy with a network of over 1,000 experienced technology consultants globally. They increase the pace of innovation by using modern software engineering practices that embrace Continuous Delivery, Security, and Operability from the outset ➡️ bit.ly/3ASy8n0
Octopus are the makers of Octopus Deploy the single place for your team to manage releases, automate deployments, and automate the runbooks that keep your software operating. ➡️ octopus.com/
SpecFlow Behavior Driven Development for .NET SpecFlow helps teams bind automation to feature files and share the resulting examples as Living Documentation across the team and stakeholders. ➡️ go.specflow.org/dave_farley

[@neildutoit5177]

Yea when I got into software engineering "monolith" was a synonym for "badly designed". I thought I had to do things microservices. I thought there was no choice. At this point I'm trying to reduce the number of services I have on all my projects, and all new projects I start are monolith. Networking is hard, expensive, and even when it works, extremely slow. And (at least in Python) API frameworks are terrible. The versioning problems discussed here have been especially bad for me because I do document processing pipelines so each step depends completely on the service before it and what version it was. I've spent most of my time over the last few years fighting to try and get APIs to do what I want them to. Such a waste of time. Not nearly worth the effort. So much documentation these days just assumes that you're running microservices and you're running clusters and I'm starting to think that companies are doing this on purpose to get you to buy more servers. Honestly I've never needed multiple databases running in a cluster. And I've never needed to run more than one copy of my apps. You can say oh well I've only worked on small projects which is true but vertical scaling is so underrated a well written application on a big server can handle millions of requests just fine.

[@IsThisAmerica1]

Glad I'm not the only one who thinks API frameworks in python are terrible. Just curious - what have been your favorites (regardless of language)?

[@xcuu]

FastAPI is not bad. Probably not the best one across all languages, but at least it’s good enough

[@chpsilva]

You are not alone

[@rafaelrosa3841]

yes, completely right, and MORE THAN THIS, still horizontally can be achieved, you just need to run more one instance of your monolith on a load balanced environment.

[@tiagodagostini]

As everything in live the middle ground is usually the best place.

[@sdb584]

My last client was so happy that I chose a monolithic approach to keep everything simple.

[@PavelHenkin]

It's rare that I want to upvote a video twice, but here we are. Thank you for elucidating the issue. Too many teams don't take that step back, but just go with the loudest voice's misunderstandings.

[@viewerone]

Mono repos can have independently deployable modules inside of it. They’re not always coupled to each other outside of commit history. Tools like Azure DevOps allows directories in a mono repo to be treated as if they are a separate repo.

Hi, I've google it with no success... by any chance, do you have that documentation at hand? Thank.s

[@SebastianSipos]

^ this, folks. use a modular architecture.

[@vleliv11]

Definitely what I was thinking about most of the video

[@vleliv11]

@Dilton McGowan II What do you mean? None of what he proposes has anything to do with c# or windows... I have seen the monorepo idea pulled off easily with nodejs and openshift, it was way easier to manage than multiple repos.

[@Glotov444]

I really like how you leveled up your editing. Great job, and great content, as always Thank you!

[@superfluidity]

I think the term monorepo may have caused some confusion - Dave is using it simply to mean "one VCS repository that holds the entire system", but I think what a lot of people understand by the term is more like "one VCS repository that holds multiple separately deployable parts, generally containing all or most of the software for an an organization, and often with tooling that detects which parts are changed and only re-deploys the changed parts"

[@taraskuzyk8985]

This is a question I struggled with for the last week, and there was almost nothing on this online. Thank you so much for this.

[@ContinuousDelivery]

Glad it was helpful!

[@psyaviah]

We exactly decided on doing this, as we suffered from a "dumb-one" approach as we thought it would be better... Yet we stumbled on that testing roadblock and dependency roadblock of shared code... It just so happens you made a video about it as well today. But hereby I can fully agree, please don't split too much. It will hurt you on the long term.

[@a__random__person]

THANK YOU for making this video! Hahaha I've seen so many companies do this, and recently convinced a team to revert back to monolith as they were starting to go down this road. I previously would refer to it as "micro-features" but coupled modules makes more sense haha.

[@dlabor1965]

I really appreciate you reminding me to hit the like button every time. Makes my life a lot easier not having to think about it all the time. Luckily I don't have to worry about the subscribe button and bell because that was done a long time ago. It is also very interesting to keep hearing from your sponsors and I always enjoy remembering the priceless knowledge and experience you share in your books. Hopefully a lot of people will use your online course, I can't, but that's another story. Keep it up, Dave!

[@ContinuousDelivery]

Thank you for your support!

[@randomgeocacher]

A company I worked at consolidated ~3 tightly coupled repositories into one. For about one year productivity was high and the devs were happy. Then someone decided to invent an architecture and that each architecture box into a module. A dev asked me to review a change (as experienced subject matter expert, wasn’t part of the devs). “Why is it in 8 gits?” “It’s because of the new architecture, the change spans 4 gits and then it is also needs a port to the customer’s mirror” “okay it looks okay-ish but it’s weird that the format conversion in one direction in module A, but in the other direction in module B. Decide where the conversion belongs and handle both sides in one module”. (Month go by, dev shows up in other errand) “hey we talked about X some time ago, how did it go?” “I f*ing gave up. It is impossible to get merge approval on a 4 git change. Fixed for customer only and did not merge upstream.” That’s an impressive way to tank productivity of a company. I never understood the wisdom of implementing a repository restructuring their most experienced devs described as bad (but in much more profane wording), if the ones who actually understands code says you are shooting yourselves in the foot… but eh, companies unable to understand who to listen to. My current responsibilities span a a bunch of trivial automation scripts where we’ve already run into dependency issues even if everything is just trivial scripts, no “real software”. But so far I’ve just identified our few stupid couplings and de-coupled the so virtuellt nothing spans more more than one or at most two gits. Do spend some time wondering if we should move mono-repo’s, but there also is something nice with a collection tiny gits where all history is just about that script. But sometimes I worry that my ant-army of tiny gits will become an issue in the future, hindering growth.

[@jimbrown6422]

This is the best discussion of repos for microservices vs monoliths. Great job!!!!! I am designing a global class app from scratch for the Green House Gas programs around the world. I have chosen microservices in their own repos with their own pipelines as you describe. I had considered grouping microservices but you easily showed me why that is not a good idea. Keep up the great work!!!!

[@ContinuousDelivery]

Glad it was helpful!

[@stephenpaul7499]

I love this channel! My CTO's motivation was not so much about the microservices being independently deployable, but more about scalability. I often feel that this decision was premature and wonder whether we could have somehow correlated specific operations with a spike in memory, and then after exhausting our optimization options, extract dedicated microservices piecemeal. We also now have a 'kitchen sink' microservice which is responsible for 80% of the work. Developers are ambitious, with a Nostradamus complex. 9 times out of 10, our predictions about the 'axis of change' are wrong. We're not all working on the next Uber, and even if we are, we will then have the budget to design the system appropriately. I also feel like we've spent a lot of money (on developer salaries) in order to save a comparatively tiny amount of money on our server costs.

[@brianbatchelor6744]

An excellent presentation of monolithic architecture.

[@chulucninh2083]

Thank you Dave. When I asked for your comments on mono-repo vs multi-repo in your video, I thought you ignored it. But instead you release a whole informative video for the topic, WOW!

[@ContinuousDelivery]

😉 Thanks for the suggestion.

[@seiofecco]

We use the multi repo approach. We chose it to help us be sure that there are no hidden or overlooked dependencies between services. Once a service artifact is deployed to production, we save a copy of the artifact. Our return to safety is to simply re-deploy the old version.

[@mesopotamianmessenger5830]

I reached to similar conclusion after practicing the dumb option for a while. This video came too late for me but it is good that it is up now - it can save someone from making their life difficult for no reason.

[@defeqel6537]

Yup, definitely been there, done that. There were good reasons, but not good enough to do so at that stage of development (if ever).

[@topperthehorse]

This is bang on. Great video, thanks.

[@romeli1941]

It's almost an instinct to separate a system into modules that aren't deployable separately. I see people do this constantly and everywhere. Thank you for calling out the real deciding factor - the deployability.

[@marikselazemaj3428]

I approve this conclusion!

[@superfluidity]

For anyone with coupled modules in separate git repositories, I suggest looking up the `--allow-unrelated` option of `git merge`.

[@fede77]

Great video and one of the BEST t-shirts ever!!

[@ContinuousDelivery]

I've worked out a deal with the t-shirt providers for a discount for CD viewers - bit.ly/3FVX3tP *Remember to use the discount code: ContinuousDelivery

[@enderan27]

I had a pipline with several microserviceses that were used independently and together. Such that sometimes one, few or all microservices deployed together. It was impossible to agree on the peace of development for each microservice. So we follow a microservice repo approach with an additional project. The project was a set of Integration test that tested all the combined features of the services. This project was triggered only by a change on the major or minor version (subversion). Every time this project run successfully builder a new "platform" version; creating a "release". So you could deploy a version of the platform which was a set of versions of the individual microservices. This version was kept by tagging the repos for every version. The version of the repos where generated by Swagger version. The swagger version was generated in several manners according depending on the respective team. My repos generated the version automatically ba detecting changes from one swagger file to the next. If there were no changes, was a patch version, if a change on parameters minor, if new endopoints major.

[@SEOng-gs7lj]

Just wanted to share that I'm using PHP's package manager in a microservices way, basically all I do is create packages and they may or may not be used by the numerous projects. While it made version control harder initially, I see the benefit of fixing bugs only once. Of course, without perfect understanding of the problem initially, the boundaries of the packages had to be adjusted over time. Overall, speed is slower, but each step forward has a higher impact

[@madmanX1314]

Very cool topic and also a great video. I need to share it with my coworkers who think splitting up the codebase of a monolith into separate repos is a good idea :-)

[@ContinuousDelivery]

Please do!

[@madmanX1314]

@Dilton McGowan II Why?

[@pythonBlender7]

I just recently moved my team to a monorepo. There were growing pains but we made it work and now our systems are much more synced up than before. Also after we've settled down, version control is much simpler.

[@vk3fbab]

Worked at a company with over 500 repos. Pattern was one repo for code to build AWS. One repo for some of the code. Another repo for the code. It could have been in one project. CDK from Amazon can deploy the infrastructure and code. But we had 3 repos. Across the organisation it was hard to find which repo was responsible for deploying a stack that was already in prod because there was no naming convention and no AWS resource tagging convention. Hence why I've always preferred fewer repos over too many.

[@willvincentparrone3339]

Good talk!

[@theuncle3961]

Thanks!

[@________w]

in my experience, anything which calls itself a "monorepo" is actually talking about what you refer to as "coupled modules", but which happen to share a same repo. Which is to say: I've never seen a monorepo whose entire state can always be deployed. Often there are situations of "module A can be deployed, but module B can't be deployed until there's a transitional deploy and a migration", ie: there's a required order of deployment, and so the supposed benefit of keeping everything in one place is lost. Sure, it can all be deployed into an empty environment at any point, and it can be tested at any point, but production state matters, and so long as the speed of light remains a constant, no deploy will be fully atomic. There will always be a need to handle transitional periods, and so everything logically needs to break down into something resembling microservices eventually anyway. Monolithic repositories aren't an aberration, but I think we agree that the important thing is to divide your repositories along the lines of how they are deployed. If "Module A" needs to be deployed separately from "Module B", then they should be in separate repos. And if Module A sometimes needs to co-exist with multiple versions of Module B, then they should be in separate repos.

[@dovahnok]

Storing incompatible modules in a monorepo does not make sense to me. As you said, you loose the benefits. But I think the missing piece here is not to give up on a synchronized timeline but to properly represent the transitional period in the monorepo timeline like this: Commit1: Change API of A but maintain backward compatibility Deploy Commit2: Migrate B to new API of A Deploy Commit3: Remove backward compatibility in API of A Deploy That way you keep deployability at all times and you are implicitly storing the "deployment order" in the monorepo timeline.

[@________w]

@@dovahnok right, that workflow makes zero sense to me, though. It's saying "don't commit that work, yet! We need to wait for a deploy!" Which over-couples development and deployment. Nobody should ever hold off on committing due to a deploy concern.

[@dovahnok]

@@________w I get your hesitance with coupling those things. However, I think that continuous delivery is actually about allowing the "keep production stable" problem to backflow a little bit into the realm of app development and repo versioning and to solve this more upstream than downstream. As explained in the video, you rather wanna use something like static typing to detect if APIs are incompatible (upstream) than deploying them and then finding out they are incompatible (downstream). If you want to commit something that is not intended to be used yet, you can hide it behind a feature toggle or just comment out the part that activates it. I think the whole approach of a monorepo with CD challenges the concept of what a git commit actually represents. It does not represent an API version of a single component but rather a version of the fully integrated system that is deployable and compatible at all times. That means that one commit can also contain a component API in a transitional state with partially deactivated code.

[@lunify2814]

@@________w I must confess I fail to see where the dependency between dev/deployment lies in that workflow. Aren't runtime config feature flags designed for this? as dovahnok said, "A" can deploy both versions of the API, then disable the backwards compatible code without a redeploy after the whole system switches to the new version via a flag. removing the dead code/flag is not a dependency issue in this case.

[@________w]

@@lunify2814 feature flags cover only a very tiny and trivial subset of deployment state

[@hypenage2415]

Dave, after further research into a monothilic repo, I would agree with you. However, there does seem to be some logistical issues with a mono repo. Google for example back in 2015 had 84TB of files in their mono repo. No other team operates with code that big, but a repo size above 20GB seems like a lot to me. Do you personally have any tools that you use that would allow developers to pull code for specific projects with in the mono repo apposed to the entire repo? Further so, doesnt a mono repo increase the time to build, and in that slowing down CD/CI? It seems great but what tools does one use to start building a mono repo?

[@ContinuousDelivery]

Mostly what I mean is not necessarily one repostory for all your systems, but one repository per deployable unit of software. You can parallelise the deployment. For the bigger systems that I worked on, we'd usually have a distributed, service-based, architecture and deployment would be carried out with each part being deployed in parallel with the others, but coordinated so that the release was between the previous version of the whole distributed system to the new version of the whole distributed system. Actually, the size of the repo doesn't really matter much unless it causes problems - you certainly don't want to be downloading 84Tb a day! But if your tools only work on deltas, then knowing that all 84Tb of stuff DEFINITIVELY works together is a useful think to know.

[@JonathanMERCIER1]

Thanks, Dave, for your video. I have a disambiguation question. About, the concept of repos discussed here, it is at which level ? i) at microservice level (i.e. as a docker compose or others pods) in such case you refer to backend and frontend repos; ii) at ecosystem level (i.e. the test environment with all services and API gateway); iii) those two levels. To me, I see an advantage to put microservices into separate VCS repos it is the CI when you commit you don't want to wait several minutes for running tests from others services. I see that microservices owns contracts with their public API which need to be respected and based on this contract you apply a BDD approach or others. While at ecosystem level you can have a repository with a deployment recipe of all services and test communication part of all services and to do a global end-to-end test.

[@davefarley77]

Good point, the "repo" I was discssing here is source control. Your GIT VCS for example. From this I expect to be able to build, test and run a deployable unit of software. A microservice or a monolith.

[@mateuszszczecinski8241]

How do you approach breakdown of the system on frontend apps (SPA, mobile) and backend apps? Do you threat them as separately deployable. Are there any advantages of keeping them in monorepo if they are created in different technologies? Should they have separate acceptance test suits, or should we test frontend-backend as whole system and some test run through ui, some through api?

[@superfluidity]

Generally no, better to treat them as one app in one repo and deploy them together. Simplest way is just to make the backend responsible for serving the code of the frontend to the browser, but you can also have a deployment script that updates the backend and the frontend on different servers as long as it always does both very close together. You still may need to account for a little bit of interaction of different versions because the browser can hold an old version of the front end in memory while the backend is updated. But you can limit the time this happens for, and depending how reliable you need that to be you may be able to get away with ignoring that issue - it also exists if you have something as simple as an HTML form together with backend code for handling form submissions.

[@bankoleogundero9446]

Love the slogan on the shirt. That solution never gets old (whether physical, i.e. turn the switch off and back on again, or software, i.e. kill the process and run a new instance😂)

[@ContinuousDelivery]

I've worked out a deal with the t-shirt providers for a discount for CD viewers - bit.ly/3FVX3tP *Remember to use the discount code: ContinuousDelivery

[@osisdie]

I think that's why macroservice comes to help re-define deployable unit against microservice from too smaller granularity piece of application to modularized services and thus you work, communicate and test with dependent services unless all your service could be truly run independently then it makes sense to separate them into repos all with its own version.

[@chat-1978]

In my opinion, the key factor is the database or similar concepts. If the schema is shared by multiple services, then you have a monolith which usually requires one deployment for all services including the schema of the e.g. the database. You can be smarter but usually monoliths are also connected with the lack of properly definition of versioning and contracts within the services of the application if they exist. There is no need after all. On the other hand, most orgs end up creating a destributed monolith when doing microservices because they don't understand them.

[@_shulhan]

There is another cost when using multiple repos in small team: context switching. Switching between repo A and B to C consume times, especially if you need to run test, and then create PR for each of them.

[@Mozartenhimer]

Here's a case for a "coupled module". You have seperate projects that are not related at all in business purpose, but you need a library to share between them. In this case, we version that we manage the dependency in both repos. One can lead, staying on the current main, and one updates the version deliberately.

[@superfluidity]

Is that the same as when you have just one repo but you use a build tool to download an off the shelf library (e.g. Doctrine / Hibernate / ActiveRecord SQLAlchemy ). I think that's not really the same thing as the coupled modules that Dave's talking about here. The library is never deployed by itself, it's just made available for other projects to include. The project that includes it will specify which version of the library it includes either within the repo or at latest as part of the deployable artifact that gets tested. There's no need for the two separate projects to always have the exact same version of the library.

[@tiagodagostini]

The greatest advantage of having independent systems is that you can test and evaluate (not only feature test, but business test) a new approach for something more easily. Obviously the limit is that you cannot have the tool (the architecture in this case) cause more effort than the job itself.

[@SylwesterKogowski]

A good video, gives some nice ideas :)

[@stefanarnautu3032]

I've been following your channel for a few months now and I'm a big fan of your content :). I understand your (and many other's) point about not using microservices just because, and I agree with it, and I also think I understand the problem of using multiple repositories (testing, and integration of the whole system becomes very difficult). In my experience, unfortunately, every time I worked in a company that uses "monorepos" or "monoliths", that translates to "disorganized, tightly coupled code", or "spaghetti code". The repository itself is generally gigabytes large, and to build it takes about an hour and it has to pull in hundreds of gigabytes (hundreds! not exaggerating) of dependencies. Although the pipeline is unitary, as in you can test and build all the code at once (integration and system testing is easier), bringing any changes to the pipeline is a nightmare. Most companies that use "monorepos" use it because of legacy reasons and don't know what auto formatting tools are or static analysers. Imagine how difficult it is to then get hundreds of developers (best case scenario a handful of tech leads) to agree on a common formatting rule and then common static analysis checks. And then actually implementing those changes in Jenkins or whatever pipeline tool you're using. And most monorepos have multiple languages! All the problems mentioned above, multiply them by the number of languages your monorepo has. Then you get to the point of actually building your repo and testing your code. Setting a common development environment is practically impossible. That means every developer has to spend hours (if not days) to set up and install their libraries and tools according to some wiki page someone wrote 5 years ago and hope that eventually the build will work more or less like their colleagues. And once in a while, somebody has a weird build failure with a clean fresh repo that nobody else has. Good luck figuring out what the difference in environments is. (In my experience, using docker containers to standardize the development environment is a good solution, but that works with small repos, not very realistic with monoliths) * How do you share development environment settings with monorepos? (libraries required for building and checks etc) * How do you improve the pipeline for a monorepo in a timely and realistic manner when you have hundreds of developers working on a repository? * If you have to just make a small change to the code, how do you only pull and build that single part of the code? (I was reading about how Google does it, but I have not seen any company actually use toolchains similar to theirs efficiently, and more information about those tools would be nice) I appreciate it if you read this far and I apologize for the wall of text :) The downsides of monorepos have been frustrating me much more than the downsides of polyrepos and if we're insisting to use monorepos, I'd appreciate if you'd make a video about the downsides of monorepos and possible solutions. Thank you, keep up the good work Dave :)

[@StefanSchade721229]

As so often, I disagree with Dave's conclusions while I find his presentation very valuable. In my opinion his statement that everything that has to be tested together before deployment should stay in one repo is way to strict. As he points out correctly, the downside of distributed repos is, that integration testing accross multiple repos is more difficult - this is not even a technical but an organizational challenge. However there are several problems. He does not debate the advantages of distributed repos nor does he mention, that there acctually are solutions to the challenges they pose. Another problem is, that in my work field (banking) it is still quite common to test the whole system landscape in unison before going live. (I know Dave is not going to like that, but that is how it is) - unthinkable to put a service in production without even testing on the system level. Even proper "microservice" systems are thus released in unison, not independently. I currently work on the renewal of a monolithic legacy codebase (1M lines, 20+ yrs old, 20 devs) and we are pulling it apart into several independent components. Think of a configuration tool, a market data tool, a calculation tool and a presentation tool - of course we have to have automatic integration tests - despite Dave's straw man these have a place to go, they have their own repos. This approach allows to divide and conquer - we can renew the application one piece at a time and we can split the team into scrum teams who do not have the mental burden to understand every aspect of our application. So we break Dave's rules by not sharing a repo, but I do not see what's wrong with that. And no, I would not suggest a microservice approach - we have neither the team size nor the need for that. PS: Looking forward to reading the book since I find Dave's thoughts valuable even if I disagree

[@Pifagorass]

1M lines project with 20 Devs with 20+ years old code ... I remember 6 people creating almost everything you are talking about in 2 years from 0 providing 7-40 micros sequencing/finishing transaction of events (including market data). Legacy maintenance can be very demanding and more expensive than rewrite from scratch.

[@StefanSchade721229]

@@Pifagorass Haha... This was my estimate to begin with too... BUT! With a legacy system comes a legacy team and you need a lot of convincing to change things. The next problem is, that the issue is not purely technical or not even foremost. As a matter of fact people do not know their own product anymore and you need careful reverse engineering in some cases to come up with the business requirements. The next issue is that managers do not like a big investment on the spot, but want to renew out of the maintenance budget without additional cost... Not completely sure about Microservices either since the team lacks relevant experience and there are no crazy scaling requirements. Perhaps rather a few modularized components.... But apart from this, the point is: Dave's rule "no separation of repos when there is no independent deployment" seems to strict. What do you think? Btw: of course we have automatic integration tests just refers to the reworked components

[@Pifagorass]

@@StefanSchade721229 completely understand constraints in your environment. The microkernel architecture is the way to go for the agency trading environment. I'd go for a more granular approach that worked nicely in multiple places - separate repos for libs (shared with backwards compatibility and deprecation approach) and repo per deployable. Pure Microservices are not a good fit specific area.

[@StefanSchade721229]

@@Pifagorass Thanks for the microkernel hint. I did not know this term and will do some research

[@manuelgurrola]

What do you think about Microservices as a solution for Software Modules that are used in many different (but similar) projects? A sort of implementation for a Platform.

[@ContinuousDelivery]

Can be a good idea, but they need to be independently deployable, and that means independently testable. That is changes to these services, don't force change on the users of the service. I have a recent video that talks about that, not just from a microservices perspective, but from a platform view: kzbin.info/www/bejne/lauraId_jcide9U

[@cflowlastname548]

There are many more options that just micro-services vs monoliths with coupled modules, one example is monolith with decoupled modules, you basically get all the pros of micro-services and monoliths without any of their drawbacks (if your implementation of them is good enough).

[@chulucninh2083]

My team maintain services that are independenly deployable, but each services contains many small services in its internal structure. We tend to think about our services as abstract class (the api/interface of the service) and implementation details (the internal services that actually do the job). This way of design allows us to split the team while still make sense of the services.

[@ContinuousDelivery]

For me it is all about scope of evaluation. If your mini-services, from which you construct "independently deployable services" are all in one repo, and all evaluated in the same pipeline prior to release, then yes, I think that is the most efficient way to organise things. If each of your mini-services is in a separate repo, and then you test them all together when composed as an "independently deployable service" or even worse "your mini services are used in multiple 'independently deployable service" then IMO you have an inefficient distributed monolith, either at the level of each of your "independently deployable services" or in the second, worst, case at the level of your whole system which is coupled through the re-use of the mini-services. if your mini services are in separate repos, the question is, what do you have to test after changing a mini service?

[@nilssjogren443]

I think the real benefits of monorepo are not well mentioned here. It’s really about separating the concern of version control from “ci”, deployment, and projects in general. What typically happens when you have a microservice architecture is that you have hundreds of repos that contain the individual services and then a bunch of shared repos for things like IaC, and some other shared resources. This is a nightmare for most small/medium companies. In practice there are many cases where one developer over a short timeframe touches many different “projects” for one single task/feature. It might be something like being able to change a users email in the front end and adding support for it in a backend api; or adding a s3 bucket and writing something to it. Being able to do those changes (sometimes atomically) in the same repo has a huge advantage in simplicity and clarity. Another benefit is having full snapshots of the production state for debugging. The main downside with monorepo is that most tools developers use are not built for it and if the repo grows too much you are forced to use/learn something like Bazel. In the end you choose your architecture to scale with the nr of developers, not the complexity of the problem. imo it’s more about having autonomy and ownership within your team so a good nr of repos to have is about the same as the nr of teams in the company ( this assumes that the teams have high autonomy and focuses of a specific problem area/domain).

[@miletacekovic]

Hundreds of repos and monorepo are most probably two equally bad choices in this case. If you follow the rule: 1 module = 1 team (3-7 people) = 1 Repo = 1 CI, you should be OK. If you project is really that big (> 1000 people) than 100-200 repos is fine. 20 people and 100 repos is a disaster, as well as 100 people and 1 repo.

[@Ratstail91]

I might have a problem with my website-driven game. I originally wrote it as a monolith, and it was unmanagable for a number of reasons. So I restarted the project using microservices, and it was wonderful. However, thinking about it now, there's a lot of coupling between the services i.e. almost every service depends on JWT authentication from a single service, and some processes bounce between different services; the in-game shop might need to check the quantities of items that the player owns, so it contacts the inventory service... Basically, the open source engine is fairly good (except the JWT problem), while the closed-source game has dependence issues. Edit: I don't know what to do about this, and there might not be a solution at all.

[@superfluidity]

I guess it depends why the original monolith was unmanageable. Maybe the monolith needed stricter separation of layers / modules within it to make it easier to understand and change. You can maintain this separation by hand, using language features to restrict visibility of elements to code in other parts of the system, or by using automatic software architecture checking tools.

[@brahmcdude685]

is there an audio version of the book?

[@ContinuousDelivery]

sorry, only kindle/e-reader or paperback, at the moment.

[@LiviuGelea]

I was hoping I wouldn't hear about static typing as an advantage of monorepos since that implies writing every single component in the same programming language, encourages coupling between components and is no better than a distributed monolith in that you still have to test everything together to make sure there is such a thing as a "working state". Monorepos? sure..I can agree we can store all microservices in same.repo for that versioning advantage

[@paneerlovr]

One comment, the title of this video seems to discuss monolithic system design vs microservices system design --- that's not what this video is about. It's about how to use a microservices system design in source control so that it has the benefit of source control rollback, while at the same time being testable with actual testing of calls to microservices and how they fit into the whole system.

[@BjarkeBruun]

From a systems administrators view then your complaints are minimal and very subjective in how the business you are developing for is run/running and the cost of your development time is worth more than your inconvience when approving code changes irregardless of it being in 1 or 8 repos. (If it is in 8 repos then you should perhaps re-consider your feature and check if it should be in a new microservice by it self). If you have to use a week to get a feature in 4-8 repos approved then it is a very small cost compared to the downtime you have to test, change approve and merge to the main branch used for prodution builds. With monoliths you have to constantly consider if your change/feature is implementing the same functionality already present in the code and you have to re-test the entire monolith if your change changes 4-8 different areas of the monolith like you have to with 4-8 microservice repos. Testing a monoliths is a much larger and more time consuming task than testing a microservice. Time is money and your time as a developer to do a change approval is minimal compared to the potential downtime of the monolith not working in production as you have to re-deploy it all - and with schema changes it can be hours of downtime which is a lot more costly for the company than your approval time to checkout and verify a few changes in a few repos that does not cost any business downtime. With microservices, in general, you have smaller databases so a new feature or schema change in one microservice does not potentially break the entire monolith requiring huge code base changes "all the time" as there is not shared code that uses the same schema or is bound to the old schema - only the microservice. As a developer you should be abstraced from what version runs in production as the operations team should have a handle on that and the binary (+ config files and databases etc.) to do a rollback/restore to a working state, if you as a developer screw up. Hence it is 2 different domains and a developers change approval process is an absolute minior and trivial cost for a company vs actually running the code that breaks causing downtime. As a sysadmin I'm in direct opposition to the view in the post as the developer change approval process is a tiny overall cost vs the development time used in monoliths and testing done before change approavel vs microservices. Microservices can be a direct copy of an existing version that implements new features and run as a new service in the infrastructure and keep the old working microservice as well. Also work on features in microservices are easier to deploy at test with new features and a new microservice with a new feature can be setup to run along all the other services without breaking anything and be put to use over time as other applications start to utilize it vs the monolith where it is a "here you go" deployment that can break everything.

[@FlorianHolzner]

Monorepo is the dream. It‘s what we‘re currently working towards. All our code for one customer in one place. That‘s even our goal while sticking to our fucked up architecture (we‘re stuck mid transition from monolith to microservice). It‘s not pretty. :/

[@2k10clarky]

I have a concept of the awesome-lith with is a monolith that can be deployed to multiple environments by a CI pipeline, has a full test suite that runs on build and uses modern frameworks. Usually paired with a document store and/or RDS which can be deployed in a known state per environment with something like flyway. This is great for the early stages of a project when things are changing fast, once something becomes more established I can then look at logical separation into services but I'm a firm believer that doing this up front is wasted effort. The complications from the microservice paradigm far outweight the benefit until you reach critical mass. That critical mass is quite high in my opinion.

[@AllanKobelansky]

Where to start. First. It doesn’t matter what camp you are in, both offer the opportunity to create a really badly designed system. Second. If you are relying on IDEs or tools to keep you on the straight and narrow with good software engineering practices, perhaps software engineering isn’t for you. Third. And I see this all the time. There have been many innovations that have happened in the hardware world that need to be mapped and adopted to the software world. Wireless protocols, such as Ant+, were designed to enable interoperability of components made by different manufacturers. By using inband signalling , it is possible to query connected systems to get the version number of the hardware/software running upstream or downstream, and act accordingly. Then you have processes that collect and aggregate these messages enabling you to know what versions of software / hardware make up your system. You should support continuous testing of services by injecting test messages with expected results. If anomalies are detected, you raise an alarm. Finally, if keeping track of the services and versions that make up your system is challenging, then perhaps you should consider another vocation. If you choose to not keep track of this information, that’s a choice. A bad choice. But a choice nonetheless.

[@gozumetaklanlar9274]

At company where I work, I think we don't use any of this and all of it. We don't use git, we developed out own version control system and we record every and any changes made to any source code file. All of our software is splitted into modules, few of them are main modules and software wont work without them, rest of modules can be added or removed without any problem. When we write new code or add new features we dont change any old code, so we have backward compatibility. Also all code is writen whit this in mind, so if we add new feature, modules that require old version of it wont brake. In our VCS we have bot that controls unused old features and reports them for future development or removal from source code.

[@superfluidity]

I don't know what your business is, so perhaps its justified, but that all sounds amazingly inefficient. Does all that work really give you an advantage over using git or another off the shelf VCS and editing old code instead of always adding to it?

[@gozumetaklanlar9274]

@@superfluidity It may be overkill, but when you are editing something there is big chance that someone will ruin something. Then extra security justify an overkill, little bit expanded open close principle of oop :)

[@nwoDekaTsyawlA]

What if some of the components of system A are reused by system B, and only some of them are tightly coupled? Should we have one gigantic repo for both systems? At least one downside of this is that there are going to be many groups of people interacting with the repository, and that development can be slower than having one or few groups touching a couple of repositories. Of course the points in the video still stand, but I think only the upsides of mono repo have been discussed.

[@ContinuousDelivery]

Yes, in that case the most efficient way to organise things is to have a single (not necessarily gigantic) repo for both services. As I said, the real question is what do you have to test before changes anywhere, in either of these services, are safe to release? The real answer given your scenario is both services. For microservices it is generally preferable to duplicate behaviour than share code between services, to retain their independence, and keep them de-coupled, unless the code that they are sharing is another, decoupled, microservice. The downsides of monorepo are that you need to work harder to get results fast enough, and that it can result in poor or inexperienced teams to ignore coupling within the repo (this is true of microservices too but they suffer more when it happens). Even in a mono repo you still want good, modular design.

[@nwoDekaTsyawlA]

@@ContinuousDelivery Thanks for thoughtful answer. On this particular case it will be too difficult to merge the repositories, so we are probably stuck with the current solution. Sadly the ugly scenario "this feature in repoA requires this other pull request in repoB to work" has happened more than once already. At least I will take into consideration to avoid creating even more repositories when adding features/services.

[@LupusMichaelis]

And what about a master repo to bind them all? A master repo responsible for managing components in submodules and manage the CI+CD pipeline from there? Ok, I have to admit, git submodule is tricky and painful (or I'm stupid), so I failed my first attempt at doing this and now I'm looking with less contempt at monorepo to host microservices.

[@fdolsky]

After seeing all of these scenarios, I cannot agree more. Btw. it seems there is some confusion in comments here. Using monorepo doesn't mean you cannot use separately deployable pieces. It doesn't mean you cannot deploy microservices into k8s. It just mean you keep all the related source code in single repo. For each piece (microservice) you are then building dedicated image and running it as pod in k8s. Getting back to running monolithic app is very bad idea from my perspective.

[@ContinuousDelivery]

💯

[@miletacekovic]

I think that the line between 'True Microservices' and 'Coupled Modules' is not that sharp as presented in this video. Even in 'True Microservices' sometimes, probably very rare in established projects, you come to the situation when it is more practical (read much simpler, much cheaper, much faster...) to break compatibility of the interface than to keep it at all cost. In that cases what we should do? Insist on backward and forward compatibility and spend a lot of resources to achieve it, or say, ok, this special time we will break the compatibility between A and B starting from A6 which need B5+ to work and we synchronize deployments for A6 and B5. We test A6 only with the new, compatible contract of B5. I think this approach is valid if it is not happening too often. There is no disadvantage in testing as we will do standard contract testing, albeit only with he new compatible contract. Only disadvantage is the need for tracking breakages of API compatibilities between modules and their versions, and synchronizing deployments of modules with broken API dependencies. If this is not happening too often and is happening in a controlled way, and if we manage to keep APIs compatibility most of the time with reasonable spending of resources (but not blindly in 100% of the cases), then it is OK.

[@ContinuousDelivery]

Sure, and you can do that with contract testing, which is what I usually recommend, but I still think that the real test is "can you release your service without testing it with others first?" if not it is not microservices, it is coupled-modules with all the costs associated with that.

[@rodrigobeltran7482]

Where did you study dave?

[@rodrigobeltran7482]

and what? :v

[@SIBUK]

Can you not just build a monolithic app and then, when it becomes so popular that a single server can't handle it and is getting bogged down, abstract out parts of the app that are particularly resource intensive and make a separate service out of them and put them on a separate server?

[@UllrichPraetz]

I don't get the statement clearly. Is it: - Organize multiple Microservices using multiple repositories in version control - "multi repo": one per microservice - Organize a Monolithic service using single repository in version control - "monorepo": containing all modules If this is the case, I also would prefer using a "monorepo" for multiple Microservices. Development is much more friction- and painless in this case. I would strive to ensure that each microservice can by deployed individually.

[@fdolsky]

Yep, my opinion - single repo with microservices (each "module" containing Containerfile) is the way. The most important thing, you will deploy all together as single release, but as a microservices. Repo containing folders: Backend (src, Dockerfile, helm chart), Frontend (src, Dockerfile, helm chart), Shared (src, Dockerfile, helm chart) CI pipeline in main branch: build, deploy all microservices (even though only frontend will change for example) to dev environment, run integration test -> continue deploying to other environments

[@FreddericUnpenstein]

Please include links to your t-shirts.

[@ContinuousDelivery]

We have an announcement on that soon 🤫

[@ContinuousDelivery]

I've worked out a deal with the t-shirt providers for a discount for CD viewers - bit.ly/3FVX3tP *Remember to use the discount code: ContinuousDelivery

[@TheCameltotem]

Help me understand how something can be completely independent. If my Microservice A needs a response from Microservice B and I change that output, then it will break. Thats a dependency. And what is the point of a microservice thats either not having any input or output. How can there truly be independent applications?

[@turn1210]

Because you will have well defined interfaces that allow for flexibility, so as long as your changes do not break the conditions of the interface, neither service will notice any change in the other

[@superfluidity]

What happens if your service needs to call a third party service, like something from the payment processor Stripe, or something like AWS. You can test your changes with Stripe before you deploy, but Stripe is almost certainly not going to test their changes with your app. And even if you test your app with Stripe you can't be sure that Stripe won't deploy changes while your app is in production. So it's the same thing. There's always some coupling, but you can still deploy changes without testing everything together - you need to make the coupling loose, and mostly avoid making breaking changes. E.g. instead of renaming a field in a response, give it an alias and send both - follow the expand-contract pattern. Breaking changes can still be made but clients need to be warned well in advance so they can stop relying on the aspect of the service that's going to go away before it happens.

[@TheCameltotem]

@@turn1210 Yeah so a contract break ie changing the actual interface would not be seen as coupling issue. I guess what im saying is that every application has to have some form of communication or else they are completely standalone. So there always has to be some form of coupling. Thanks!

[@turn1210]

@@superfluidity true, but the coupling is to external sources by design, and in effect the abstractions between internal services will provide them with protection from this change.

[@turn1210]

@@TheCameltotem I know what you mean, there is always a necessary coupling to message transport. The project I’m working on now minimises but doesn’t achieve 100% decoupling. Basically there is one shared ‘services’ library that allows transport mechanisms, logging and a few other pieces to be abstracted by interface, so it’s coupled to that one library, but aside from that, the services operate completely independently of each other.

[@robin8561]

Hi Dave, first of all thanks for your valuable contributions, definitely the best channel for software developers :D However, this is the first video where I disagree. At least, if I understand the statement correctly: if you don't use microservices, use Mono-Repo. Thinking this idea further, without microservices you end up with a corporate repository. We have customers that actually works like this, and... believe me... this is hell! To pick up on your example, if A, B, C and D only go into one product, I can live with the Mono-Repo, at least up to a certain point ;) But where is the world that simple? Let's imagine we have product P1, which consists of the binary artifacts of the builds of subproducts A, B, and C. Another product P2 consists of A, B and D. However, C and D have absolutely nothing to do with each other and are completely independent, they are never used together. Why would I put these subproducts into a common repository. I wouldn't want to burden a developer working on P1 to deal with with D. And let's think about this on a realistic scale. We have hundreds of subproducts that can be combined in various combinations to form the products P1, P2, P3,.... . Millions of lines of code, all in one repository? Where most of the modules have absolutely nothing to do with each other with no need to be tested or deployed together? Let's bring versions into play: P1 should be released with A.1, B.1, C.2. P2, on the other hand, is to be released with A.1, B.2, D.1. B.2 is currently being worked on, but B.2 is incompatible with C.2. At this point, at the latest, the Mono-Repo no longer helps us to test and deploy things together. In the Mono-Repo, we might have to branch now. And as you said and i totally agree, branching is not a good idea. In fact, my impression is that Multi-Repo and trunk-based developement are a perfect match. In addition, MultiRepo has further advantages. The separation into different repositories forces to respect architectural boundaries. If A depends on IB but not on B (I = interface, and yes, IB can also be a subproduct and live in it's own repository), and I don't have the source code of B at all at hand, I stay mentally in A's world, and, very importantly, do not choose shortcuts past the architectural boundaries (even good programmers do this on bad days :) And of course the repositories are not that gigantic. That means I can find my way around more quickly. A new developer in the team, who initially works on A, does not have to dig through gigantic mountains of code with modules that are completely irrelevant to him for specific task.

[@ContinuousDelivery]

I think that you start with a false alternative, it is not "microservice or corporate repo", it is "align your repo boundaries with deployable things". So mostly I'd have a repo per system, and the test for "deployable things" is "do I need to test this with anything else before I release it?". If not you are good, if you do, then you are dealing with more complexity than you need to. In my experience coupling is a MUCH bigger problem than lack of re-use. Of course we want to architect systems so that there are cannonical versions of some behaviours, I am a big fan of service oriented design, key to the strategy of microservices is to reduce coupling, and as soon as you share code you increase coupling. This means that it is always a dance. There is no single simple answer, and my experience for the past few years is that virtually every client company that I have worked with are struggling to cope with too much coupling between things that they think of as microservices, when they are really coupled-modules. I don't advise mono repos for everything, but I at the moment I see many more people struggling because of multi-repos than because of mono repos. It is more complex than either one of these solutions which is why I like the "independently deployable unit" as a guide. That doesn't rule-out sharing code, but when you do, you protect those boundaries where you share in some of the ways that I mentioned.

[@echobucket]

I really think we need a better tool than git that knows how to handle different "sub repos" in a better way than git's submodules/subtree etc work now.

[@juanpisukan]

Сергей Доренко похорошел

[@marawanlotfy1477]

this question on your t-shirt have you tried turning it off and on again? the answer is no i let the supervisor to do it for me man

[@ContinuousDelivery]

😂😂😂 If you're interested, I've worked out a deal with the t-shirt providers for a discount for CD viewers - bit.ly/3FVX3tP *Remember to use the discount code: ContinuousDelivery

[@X39]

The biggest problem with git repositories is that submodules suck sooooooo much Like literally, there is nothing worse than having submoduls in git They are not properly supported by any ide, they suck regarding support of git (the countless times I had to fix the mistakes of colleagues due to ide mistakes are really an understatement) and worst of all, submodules having submodules are pretty much not supported, because sharing them properly is not an option Mono repos are great to manage those things... But horrible in every other aspect But submodules are OK for contracts, I guess?

[@colinmaharaj]

I'll stick with monolithic code, thats it. I don't even use git

[@redwrath5]

Is this coupled modules thing really common? How can people possibily fall into that trap that often? My company was trending towards that tbh, but most of the tech leads fixed the course before it got 'really' started.

[@ContinuousDelivery]

It’s the most common approach that I see in most of my clients. May be a self-selecting group I suppose.

[@michaelrstover]

I think people fool themselves into thinking their modules aren't coupled. The problem is, coupled vs uncounted isn't really a binary proposition. There is more or less coupled. Even if most changes are ok, if your services work together at all, its possible to make changes in one that break the other.

[@superfluidity]

@@michaelrstover Right. If you run a website your code is coupled to the code in every user's browser. But generally you wouldn't try to control when users upgrade their browsers - although that happens with intranet sites - so you know not to couple too tightly to the specifics of any browser version, and browser vendors are good at making backwards compatible changes.

[@michaelrstover]

@@superfluidity They are now. They did not used to be. That has taken decades to hammer out, and it's still not perfect. Now imagine trying to convince yourself you've made some new system of microservices that have that level of independence.

[@aram5642]

This is always confusing and annoying that everyone has seems to have their own understanding of the terms. I can't really understand this lecture without knowing what the author means by monolith vs monorepo for that matter.

[@ContinuousDelivery]

Monorepo means one repo, multi repo means many. You are right to say one or many for what. What the video attempts to explain is that it should be for the scope of evaluation necessary to release a change into production - a single deployable unit of software. The advice in the video is to organise your repos based on the releasability of your software. You should be able to release the contents of a repo without testing it with anything else first.

[@hstrinzel]

WAAY too complex for my liking.

[@ralfdekleine]

Better rename to ‘Why microservices suck and monoliths rule!’.

[@ContinuousDelivery]

I don't think "microservices suck" but they are a lot more complicated than monoliths, so if you can't build and manage a monolith well, you aren't ready for microservcies. The problem is that people think microservices are easier, when they are not. Microservices are the most scalable approach to SW dev, but that comes at a cost.