Docker Compose with .NET 8, PostgreSQL, and Redis (step by step)

Get the source code for this video for FREE → the-dotnet-weekly.ck.page/doc...
☄️ Master the Modular Monolith Architecture: bit.ly/3SXlzSt
📌 Accelerate your Clean Architecture skills: bit.ly/3PupkOJ
🚀 Support me on Patreon to access the source code: / milanjovanovic
Docker is a platform designed to help developers build, share, and run container applications. A container is a standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing. Docker Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application's services. You can easily containerize your .NET applications with a Dockerfile. Visual Studio can help you by scaffolding a Compose YAML file you can further customize. I'll show you how to start with Docker Compose in this video.
Check out my courses: bit.ly/3PupkOJ
5 Awesome C# Refactoring Tips
www.milanjovanovic.tech/blog/...
Join my weekly .NET newsletter:
www.milanjovanovic.tech
Read my Blog here:
www.milanjovanovic.tech/blog
Chapters
0:00 Adding the Docker-Compose project
1:15 Reviewing the Dockerfile
2:14 Reviewing the Docker Compose YAML file
3:08 Adding PostgreSQL using Docker compose
5:15 Connecting to PostgreSQL running inside Docker
6:33 Running with Docker Compose from Visual Studio

[@MilanJovanovicTech]

Get the source code for this video for FREE → the-dotnet-weekly.ck.page/docker-compose P.S. How often do you use Docker and/or Docker Compose in your projects?

[@markovertus3909]

In every project, I love it. However, it's still an issue for real development. I would prefer to have a real time build and compile on every file save + debug if the app is running inside docker-compose. When I saw the title of your video, was low key hoping to see that, as it's still eludes me on .NET.

[@Its_Kaleb_]

Been messing around with docker a bit more recently, particularly for using dev containers in VS Code. Made for some interesting, config-as-code dev environments using compose to wire up dependencies too. Have you done much with dev containers?

[@MilanJovanovicTech]

@@markovertus3909 I recently learned about 'Docker Compose Watch' - worth looking into

[@MilanJovanovicTech]

@@Its_Kaleb_ No, but I've played around with Testconatiners for my integration and E2E tests

[@markovertus3909]

@@MilanJovanovicTech yea, Ive played with it as well. I dont like that you need to mount your dlls for it to have any effect. But other then that, I agree with you, the support for docker and compose especially in Visual Studio is pretty insane.

[@igorbarros9620]

I'm going to make myself a cup of coffee and get ready for some knowledge from the Wizard Milan himself. \m/

[@MilanJovanovicTech]

Hope I delivered, then 😁

[@igorbarros9620]

@@MilanJovanovicTech You sure did! Hey, man. Any chance I can a discount on your course? I live in Brazil, so for me to pay, it's like 5x times the current price (1 dollar is like 4.98 reais). I have been watching your videos since last december, and you have been helping a lot on my work! And thank you for that :))

[@MilanJovanovicTech]

@@igorbarros9620 Check the course page, there should be a purple banner on top with a discont code (for Brazil)

[@akeemaweda1716]

Thanks Milan for this great and straight-to-the-point video. Stay awesome 🎉

[@MilanJovanovicTech]

Always!

[@itssalmanvlogs_]

I always learn something new watching your videos, Thank you.

[@MilanJovanovicTech]

Happy to help! :)

[@DaminGamerMC]

Great video, just what i needed for my job

[@MilanJovanovicTech]

Perfect!

[@stochasticity86]

Amazing and to the point. Love it!

[@MilanJovanovicTech]

Glad you liked it!

[@ertackosucu]

Excellent as usual. Thank you.

[@MilanJovanovicTech]

Glad you enjoyed it

[@feinirl]

great video my guy :3 thanks for pca v2 btw you're insane

[@MilanJovanovicTech]

You're welcome. I'd appreciate any feedback on V2 😁

[@shuvo9131]

Thanks for awesome explanation. This is to the point

[@MilanJovanovicTech]

You're very welcome :)

[@UberEverywhereSKRT]

Milan your videos are the best. I been following you for a minute now and ur amazing. Would be nice if u could do a video on how to setup a full .net development flow including dev, stage, prod env with docker , how to handle env variables during local vs prod (in cicd for example). I get confused on where the env variables (and secret) should be stored in local dev vs prod (where and when they are fetched from, etc). Ty in advance!

[@MilanJovanovicTech]

Great suggestion!

[@rafaspimenta]

​@@MilanJovanovicTech Go for it please :P

[@PatricSjoeoe]

Really nice and easy explained! =)

[@MilanJovanovicTech]

Glad you liked it!

[@paolosartiano1454]

very thanks! fantastic your video!

[@MilanJovanovicTech]

Glad it helped!

[@juanlopeztello7315]

Excelent. It would be nice a video about setting up a development environment using devcontainers in vscode

[@MilanJovanovicTech]

Great suggestion!

[@giorgospanagopoulos3993]

Excellent video. If you also showed how to connect to the data source via postgres & redis client UIs it would have been a perfect one!

[@MilanJovanovicTech]

May make a YT short for that

[@kj2w]

This video is awesome. I shows a great use case for using Docker with an already working system. Correct me if I'm wrong but didn't you also do a video where integration tests can leverage a docker container?

[@MilanJovanovicTech]

Yes: kzbin.info/www/bejne/qpuYi3aqq8yLgbs And publishing a functional testing video soon

[@andersjuul8310]

Love it ❤

[@MilanJovanovicTech]

Awesome, glad to hear that! 😁

[@antondoit]

Great as always

[@MilanJovanovicTech]

Glad you think so!

[@mohamedalaa8212]

Another fantastic video, as always! Keep up the great work! 💥 I'm also curious about the process by which Docker Compose build the productapi image and assigning it the dev tag.

[@MilanJovanovicTech]

That's a subject for a different video 😁

[@Liumbert]

oh man u r awesome, u won a new subscriber

[@MilanJovanovicTech]

Welcome aboard!

[@Fikusiklol]

Cool stuff, Milan Just a couple of things: 1) you want to depend_on a database in compose file to be created first 2) you can specify environments in compose for appSettings file using ConnectionString__Database | Redis, cause those might be different for production/dev

[@MilanJovanovicTech]

2) Which is why I'm using appsettings.Development.json, which shouldn't be used in production anyhow

[@Slyzor1]

What a great video, thank you very much!

[@MilanJovanovicTech]

You're very welcome!

[@hamitkarahan2973]

Hey Milan thanks for great explanation. I'd like to ask you that what is the difference using docker compose over just debugging our project from visual studio development environment? And do developers usually use docker compose in their local machines for debugging the project, or they using it in environments(Development, Staging or Production) also?

[@MilanJovanovicTech]

Just debugging won't start the containers you need

[@onosnow6500]

Hi Milan, thanks for the amazing video explaining the docker-compose. Just a very quick question, if I am using Amazon RDS, how to specify the volume (04:35)

[@MilanJovanovicTech]

Isn't it a cloud service?

[@Corvin_]

Really cool video, could you please clarify when or why you would use this? I can never really find scenarios on my own where docker makes my life easier. Does it make local development easier compared to setting up a localdb? Can you for example connect a db from the cloud for the production environment?

[@MilanJovanovicTech]

Local development would be the main use case. Everyone in your team has the same environment.

[@Corvin_]

@@MilanJovanovicTech Alright cool. I guess it is a tiny bit simpler than applying migrations for localdb on everyone's pc. But not by a ton.

[@bojned]

Nice and practical... I don't spend as much time coding these days (at least as not as much I would like), and videos like these keeps my knowledge fresh and up to date.

[@MilanJovanovicTech]

Glad it was helpful!

[@TrungTran-le2ht]

Thanks for the content. I have a question What is the benefit when using DbContext directly compare to Repository/UoW pattern ?

[@MilanJovanovicTech]

Testability mostly, and code reuse (less duplication)

[@selmangulmez6476]

Hey Milan, Cache GetAsync doesnt have overload callback function which extension package gives that chance for us? Thanks for informations.

[@MilanJovanovicTech]

It's this: github.com/mgravell/DistributedCacheDemo/blob/main/DistributedCacheExtensions.cs

[@UnseenScofield]

Thanks @@MilanJovanovicTech

[@montanomariano]

Great video! What if you were using database-first code-first approach? How would you make sure you get a copy of your existing database?

[@MilanJovanovicTech]

This example is code-first. The migrations are applied when the app starts. But you can connect to the DB as any other database, and apply your changes.

[@vibes992000]

Thanks a lot milan. Any comparison video for Docker Compose versus .Net Aspire ?

[@MilanJovanovicTech]

Not yet, but that's a great idea

[@vibes992000]

@@MilanJovanovicTechThanks for considering. Expecting it to be a detailed video on pros and cons of both worlds and if possible multiple cases where each fits best. Thanks in advance.

[@amimridul4278]

Just awesome

[@MilanJovanovicTech]

Thanks!

[@enis7x]

When utilizing Docker, how do you create EF migrations? The environment variable I have placed in the Docker container for the database connection does not get read when I run EF commands. Except for doing EF migrations, the application functions as intended and the db-server connection is working fine.

[@MilanJovanovicTech]

Yeah, you'll have to use the actual connection strings for running migrations. You can specify it manually when running EF commands. learn.microsoft.com/en-us/ef/core/cli/powershell#update-database

[@enis7x]

@@MilanJovanovicTech That works when you are updating the database but not when you are trying to create migrations. I'm still trying to find a workaround for the problem but thanks for the help, at least it works with the update command.

[@enciphered7650]

Would you ever recommend caching result sets ?

[@MilanJovanovicTech]

Result sets of what?

[@enciphered7650]

A commonly loaded table. Let's say customer table.

[@winstochurgle9133]

I've been thinking why I can't establish a connection with Redis container. You need to add the appropriate network in the docker-compose.yml so that they act in the same context 🤓

[@MilanJovanovicTech]

Strange, I didn't need to define a network

[@usamabinahmed8012]

Hi Milan , Where we can get this code example code available ? I think that will help new bees to get more things out of the concepts .

[@MilanJovanovicTech]

I share all the source code over on Patreon

[@MilanJovanovicTech]

But I'll see if I can release this pro-bono on GitHub, I agree it would be helpful

[@TheOverBack]

So. Docker Compose is useful when you want to run the whole solution with their dependencies? But if I have 2 MVC apps, a background application with cron jobs, and a lot of broker consumers, they are dependent on db, Redis, rabbit, and mongoDB. And if I want to run one MVC website with consumers or another MVC app with jobs, do I have to create a lot of docker compose files to cover all combinations?

[@MilanJovanovicTech]

You can place all of them inside the same docker-compose setup

[@user-xm7sh3vw8o]

Can you explain more? Many of you don't understand. From the comments, it can be concluded that Redis is just a client? How to publish it online for use?

[@MilanJovanovicTech]

You run Redis like any other service

[@cleitoncarlospereira2006]

Hello Milan, how are you? When I run the application, the EF migration control table is created, however, the tables in the database are not created. Do I need to look at anything else?

[@MilanJovanovicTech]

Strange? The migrations are there? They're applied at startup - app.ApplyMigrations()

[@petiriesz1916]

Same for me, I'm using .net 8 maybe that caused the problem. I applied the migration via cli and it works.

[@lourenceplay9017]

How would you cache a ddd class that only has private constructor and Create() static method? There will be an error related to deserializing. I stuck on this error: System.NotSupportedException: Deserialization of types without a parameterless constructor, a singular parameterized constructor, or a parameterized constructor annotated with 'JsonConstructorAttribute' is not supported.

[@MilanJovanovicTech]

JsonConstructor - or use a different type to cache

[@LucianDunca]

The video is awesome so thank you for that and everything you are doing. However there are some points which are missing, or might be difficult for an average Jr-Mid developer to figure out by himself without accessing the source code... 1. Postgres setup and creating the database initial migration, with docker is hard to do this following your video; On short you need Postgres running in order to create your first migration 2. RedisCache part. I think you are relying on an extension that allows for the cache asside pattern, the Default GetAsync method in the IDistributedCache interface does only have the key and cancellation token as parameters. This might be on purpose to get people to want the source code :) Nontheless your vieos are great and an invaluable source of learning! Stay awesome!

[@MilanJovanovicTech]

It's more out of convenience - I already have videos on running EF migrations at startup and quite a few videos on caching, Redis, etc.

[@DaminGamerMC]

You can apply migrations with code like taht, but to add a migration you have to change the startup project which is kinda weird. Is there a better way?

[@MilanJovanovicTech]

"dotnet ef" command line tool instead of using the VS console

[@sauravbhatta5303]

I was working on the pragmatic course and facing the same configuration null error in docker run. Could be same server name issue. You should also add a video to run debugger in docker.

[@MilanJovanovicTech]

Did you manage to solve it?

[@sauravbhatta5303]

Checked it earlier. everything looks good and no problem locally but in Docker I am getting unhandled exception. ArgumentNullException: Value can’t be null. Parameter configuration. It’s at Infra.DI.AddInfrastructure(IConfiguration) Seems like not finding my appsetting.

[@sauravbhatta5303]

That error is gone by adding appsetting file to the project. But still my port does not show up in docker. Port 5001 is not visible. I can open swagger with the 5001 port. Any idea why it’s not picking up from the docker file or compose file.

[@minimalstory]

good!!!

[@MilanJovanovicTech]

Thanks!!

[@nathangrosvenor7661]

What would you say is better and faster to get to production out of .NET Aspire and Docker Compose

[@MilanJovanovicTech]

Aspire isn't meant for production workloads

[@nathangrosvenor7661]

@@MilanJovanovicTech Makes sense

[@timur2887]

should you clear the cache on updates or just update the cache?

[@MilanJovanovicTech]

Either one works fine. If you have the data ready, updating the cache makes sense.

[@_miranHorvat]

In a real world scenario your app is deployed to multiple on-prem instances. Each instance needs its own settings and when you spin up a new updated version it has to use the same settings. So, where do you put your settings?

[@MilanJovanovicTech]

You want all of them to use the same settings? Why not something like Azure App Configuration?

[@_miranHorvat]

@@MilanJovanovicTech No. Different settings and local resources on different locations. When you update i.e. install a new version, it has to use the same settings as the old one.

[@BRH90593]

Docker container does not run when Visual Studio is closed. What am I doing wrong

[@MilanJovanovicTech]

If you start it from VS - I'd expect it to also shut down with VS

[@jamesreynolds3167]

With the new container features in the build tools, you no longer need a docker file. You can configure it all within the csproj

[@MilanJovanovicTech]

That's a nice improvement. Maybe I'll make a YT short about it

[@MateStier-hd4nv]

What kind of postgres gui do you use?:)

[@MilanJovanovicTech]

Usually just DBeaver

[@flacky3690]

I followed the steps but then I removed the docker-compose file and trying to add it back again but whatever I do I get this error "Unable to find any supported orchestrators for this type of project" any ideas?

[@MilanJovanovicTech]

No idea, haven't seen that issue before

[@flacky3690]

@@MilanJovanovicTech Got it to work! Had to remove all the docker compose configuration and also go inside the project folder and remove all the docker related files that had been created. They didnt get removed automatically when I removed the docker compose project that's what caused the issue.

[@tienviet7442]

what is theme you are using in vs?

[@MilanJovanovicTech]

ReSharper

[@mad_t]

It's a pity MS doesn't support podman (yet?), and you need extra actions to make it work with some limitations :(

[@MilanJovanovicTech]

Yeah, that's annoying. I'll see if there's a nice solution for that, and maybe make another video.

[@artemivanov2141]

Why do you need build and then publish in dockerfile? I think build is redundant here, publish already has build inside

[@MilanJovanovicTech]

If it works, great

[@baxram97]

But what if I have migration files?

[@MilanJovanovicTech]

You can automate that part - like I'm doing here (for local development) and applying it via DbContext.Database.Migrate() Or create the migration scripts, and run them yourself

[@user-bz9ep9ep3x]

Please publish a video by adding docker support for your CQRS pattern project(Gatherly)

[@MilanJovanovicTech]

Just follow the same idea from here 😅

[@ivandrofly]

4:59 - Connection string dns-naming

[@MilanJovanovicTech]

Hmm?

[@lettuceturnipthebeets790]

just when Nana posted her Zero-to-Hero vid on Docker Compose 😅 looks suss jk jk it's fine to take inspiration from others)

[@MilanJovanovicTech]

Pretty cool, thanks for introducing me to a new tech channel on YT 😅 P.S. I can show you this was recorded December 12th last year. I typically have one-two months of videos scheduled ahead.

[@harkiratsingh4947]

Milan is right , I have been following him over 1 year now and I have seen docker before as well. Milan is Azure AD B2C in your plan as well for sometime ahead ?

[@capnbootyberry]

Hey, great video! Quick question regarding launch profiles. When I was just running my API using a Dockerfile, without docker-compose, I hade a `Docker` debug launch profile that I could set environment variables with when debugging. I know there's a Docker Compose launch profile now, but this doesn't seem to be exposed in the same manner as before. For instance, I can't go to the debug properties of my API project and add environment variables to this launch profile. Are you able to explain this for me?

[@MilanJovanovicTech]

You would add the environment variables in the docker-compose.yml file

[@ChandanDey-cw4zq]

Hi, I'm getting following error Npgsql.NpgsqlException: 'Failed to connect to 172.18.0.3:5432' public static class MigrationExtensions { public static void ApplyMigrations(this IApplicationBuilder app) { using IServiceScope scope = app.ApplicationServices.CreateScope(); using ApplicationDbContext dbContext = scope.ServiceProvider.GetRequiredService(); dbContext.Database.Migrate(); // error here } }

[@ChandanDey-cw4zq]

It's working. After second time execute

[@MilanJovanovicTech]

Probably DB did not start

[@capnbootyberry]

Can host machine environment variables be accessed by docker compose? Does something like `POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}` work when declaring a container's environment variables?

[@MilanJovanovicTech]

Yes, you can specify environment variables