Docker Compose with .NET 8, PostgreSQL, and Redis (step by step)

[@MilanJovanovicTech]

Get the source code for this video for FREE → the-dotnet-weekly.ck.page/docker-compose Want to master Clean Architecture? Go here: bit.ly/3PupkOJ Want to unlock Modular Monoliths? Go here: bit.ly/3SXlzSt

[@markovertus3909]

In every project, I love it. However, it's still an issue for real development. I would prefer to have a real time build and compile on every file save + debug if the app is running inside docker-compose. When I saw the title of your video, was low key hoping to see that, as it's still eludes me on .NET.

[@Its_Kaleb_]

Been messing around with docker a bit more recently, particularly for using dev containers in VS Code. Made for some interesting, config-as-code dev environments using compose to wire up dependencies too. Have you done much with dev containers?

[@MilanJovanovicTech]

@@markovertus3909 I recently learned about 'Docker Compose Watch' - worth looking into

[@MilanJovanovicTech]

@@Its_Kaleb_ No, but I've played around with Testconatiners for my integration and E2E tests

[@markovertus3909]

@@MilanJovanovicTech yea, Ive played with it as well. I dont like that you need to mount your dlls for it to have any effect. But other then that, I agree with you, the support for docker and compose especially in Visual Studio is pretty insane.

[@igorbarros9620]

I'm going to make myself a cup of coffee and get ready for some knowledge from the Wizard Milan himself. \m/

[@MilanJovanovicTech]

Hope I delivered, then 😁

[@igorbarros9620]

@@MilanJovanovicTech You sure did! Hey, man. Any chance I can a discount on your course? I live in Brazil, so for me to pay, it's like 5x times the current price (1 dollar is like 4.98 reais). I have been watching your videos since last december, and you have been helping a lot on my work! And thank you for that :))

[@MilanJovanovicTech]

@@igorbarros9620 Check the course page, there should be a purple banner on top with a discont code (for Brazil)

[@aweklin]

Thanks Milan for this great and straight-to-the-point video. Stay awesome 🎉

[@MilanJovanovicTech]

Always!

[@UberEverywhereSKRT]

Milan your videos are the best. I been following you for a minute now and ur amazing. Would be nice if u could do a video on how to setup a full .net development flow including dev, stage, prod env with docker , how to handle env variables during local vs prod (in cicd for example). I get confused on where the env variables (and secret) should be stored in local dev vs prod (where and when they are fetched from, etc). Ty in advance!

[@MilanJovanovicTech]

Great suggestion!

[@rafaspimenta]

​@@MilanJovanovicTech Go for it please :P

[@LucasLima-il4ky]

I do not have words to thank you Milan , sure i'm going to buy your courses!! THANK YOU FOR ALL KNOWLEDGE!!

[@MilanJovanovicTech]

My pleasure!

[@itssalmanvlogs_]

I always learn something new watching your videos, Thank you.

[@MilanJovanovicTech]

Happy to help! :)

[@Fikusiklol]

Cool stuff, Milan Just a couple of things: 1) you want to depend_on a database in compose file to be created first 2) you can specify environments in compose for appSettings file using ConnectionString__Database | Redis, cause those might be different for production/dev

[@MilanJovanovicTech]

2) Which is why I'm using appsettings.Development.json, which shouldn't be used in production anyhow

[@Mark-jk1jv]

Another excellent video Milan. These short video tutorials are excellent learning resources. Thanks.

[@MilanJovanovicTech]

Glad you like them!

[@bojned]

Nice and practical... I don't spend as much time coding these days (at least as not as much I would like), and videos like these keeps my knowledge fresh and up to date.

[@MilanJovanovicTech]

Glad it was helpful!

[@willgordon5737]

Bro you are so smart you must be working in a FAANG company thank you and congratulations for 100 k subscribers, you deserve a million!

[@MilanJovanovicTech]

Working for myself, even better 🤷‍♂️

[@andresbeltran5779]

Thank you, I was suffering for several days to achieve this and finally I did it

[@MilanJovanovicTech]

Awesome, good job 🏆

[@DaminGamerMC]

Great video, just what i needed for my job

[@MilanJovanovicTech]

Perfect!

[@sunzhang-d9v]

Can you explain more? Many of you don't understand. From the comments, it can be concluded that Redis is just a client? How to publish it online for use?

[@MilanJovanovicTech]

You run Redis like any other service

[@juanlopeztello7315]

Excelent. It would be nice a video about setting up a development environment using devcontainers in vscode

[@MilanJovanovicTech]

Great suggestion!

[@kj2w]

This video is awesome. I shows a great use case for using Docker with an already working system. Correct me if I'm wrong but didn't you also do a video where integration tests can leverage a docker container?

[@MilanJovanovicTech]

Yes: kzbin.info/www/bejne/qpuYi3aqq8yLgbs And publishing a functional testing video soon

[@distractednine_di]

Thank you very much! This video saves lives!

[@MilanJovanovicTech]

Glad it helped!

[@giorgospanagopoulos3993]

Excellent video. If you also showed how to connect to the data source via postgres & redis client UIs it would have been a perfect one!

[@MilanJovanovicTech]

May make a YT short for that

[@stochasticity86]

Amazing and to the point. Love it!

[@MilanJovanovicTech]

Glad you liked it!

[@shuvo9131]

Thanks for awesome explanation. This is to the point

[@MilanJovanovicTech]

You're very welcome :)

[@branislavstojkovic2040]

Great tutorial! Thank you very much Milan :D

[@MilanJovanovicTech]

My pleasure!

[@ForrestBrown]

Running migrations this way in development is definitely easy, but how would you recommend running them for production? Thanks for all of the great videos!

[@MilanJovanovicTech]

www.milanjovanovic.tech/blog/efcore-migrations-a-detailed-guide

[@harunaabaldeh2006]

I was going to ask the same question but luckily found this one already?

[@feinirl]

great video my guy :3 thanks for pca v2 btw you're insane

[@MilanJovanovicTech]

You're welcome. I'd appreciate any feedback on V2 😁

[@ertackosucu]

Excellent as usual. Thank you.

[@MilanJovanovicTech]

Glad you enjoyed it

[@selmangulmez6476]

Hey Milan, Cache GetAsync doesnt have overload callback function which extension package gives that chance for us? Thanks for informations.

[@MilanJovanovicTech]

It's this: github.com/mgravell/DistributedCacheDemo/blob/main/DistributedCacheExtensions.cs

[@UnseenScofield]

Thanks @@MilanJovanovicTech

[@montanomariano]

Great video! What if you were using database-first code-first approach? How would you make sure you get a copy of your existing database?

[@MilanJovanovicTech]

This example is code-first. The migrations are applied when the app starts. But you can connect to the DB as any other database, and apply your changes.

[@juancarlosalemanmayorga3093]

Thanks Milan!! Interesant, using the cache

[@MilanJovanovicTech]

Glad you liked it!

[@maevshadowsong]

what a nice speedrun lesson

[@MilanJovanovicTech]

Hey, thanks!

[@mohamedalaa8212]

Another fantastic video, as always! Keep up the great work! 💥 I'm also curious about the process by which Docker Compose build the productapi image and assigning it the dev tag.

[@MilanJovanovicTech]

That's a subject for a different video 😁

[@kodindoyannick5328]

Thanks for all of the great videos!

[@MilanJovanovicTech]

Just happy to help :)

[@shanabunny]

You explain things so well! Thank you

[@MilanJovanovicTech]

Glad it was helpful!

[@Corvin_]

Really cool video, could you please clarify when or why you would use this? I can never really find scenarios on my own where docker makes my life easier. Does it make local development easier compared to setting up a localdb? Can you for example connect a db from the cloud for the production environment?

[@MilanJovanovicTech]

Local development would be the main use case. Everyone in your team has the same environment.

[@Corvin_]

@@MilanJovanovicTech Alright cool. I guess it is a tiny bit simpler than applying migrations for localdb on everyone's pc. But not by a ton.

[@vibes992000]

Thanks a lot milan. Any comparison video for Docker Compose versus .Net Aspire ?

[@MilanJovanovicTech]

Not yet, but that's a great idea

[@vibes992000]

@@MilanJovanovicTechThanks for considering. Expecting it to be a detailed video on pros and cons of both worlds and if possible multiple cases where each fits best. Thanks in advance.

[@usamabinahmed8012]

Hi Milan , Where we can get this code example code available ? I think that will help new bees to get more things out of the concepts .

[@MilanJovanovicTech]

I share all the source code over on Patreon

[@MilanJovanovicTech]

But I'll see if I can release this pro-bono on GitHub, I agree it would be helpful

[@sauravbhatta5303]

I was working on the pragmatic course and facing the same configuration null error in docker run. Could be same server name issue. You should also add a video to run debugger in docker.

[@MilanJovanovicTech]

Did you manage to solve it?

[@sauravbhatta5303]

Checked it earlier. everything looks good and no problem locally but in Docker I am getting unhandled exception. ArgumentNullException: Value can’t be null. Parameter configuration. It’s at Infra.DI.AddInfrastructure(IConfiguration) Seems like not finding my appsetting.

[@sauravbhatta5303]

That error is gone by adding appsetting file to the project. But still my port does not show up in docker. Port 5001 is not visible. I can open swagger with the 5001 port. Any idea why it’s not picking up from the docker file or compose file.

[@timur_skov]

THANK YOU VERY MUCH!!!!! You really helped me.

[@MilanJovanovicTech]

You're welcome!

[@hamitkarahan2973]

Hey Milan thanks for great explanation. I'd like to ask you that what is the difference using docker compose over just debugging our project from visual studio development environment? And do developers usually use docker compose in their local machines for debugging the project, or they using it in environments(Development, Staging or Production) also?

[@MilanJovanovicTech]

Just debugging won't start the containers you need

[@timur2887]

What is the proper way to use contaniers for everything EXCEPT the web application? How to start postges and redis via docker-compose and api outside of it?

[@MilanJovanovicTech]

I think the best answer to that you'll find in the docs for each relevant service.

[@onosnow6500]

Hi Milan, thanks for the amazing video explaining the docker-compose. Just a very quick question, if I am using Amazon RDS, how to specify the volume (04:35)

[@MilanJovanovicTech]

Isn't it a cloud service?

[@Liumbert]

oh man u r awesome, u won a new subscriber

[@MilanJovanovicTech]

Welcome aboard!

[@enciphered7650]

Would you ever recommend caching result sets ?

[@MilanJovanovicTech]

Result sets of what?

[@enciphered7650]

A commonly loaded table. Let's say customer table.

[@LucianDunca]

The video is awesome so thank you for that and everything you are doing. However there are some points which are missing, or might be difficult for an average Jr-Mid developer to figure out by himself without accessing the source code... 1. Postgres setup and creating the database initial migration, with docker is hard to do this following your video; On short you need Postgres running in order to create your first migration 2. RedisCache part. I think you are relying on an extension that allows for the cache asside pattern, the Default GetAsync method in the IDistributedCache interface does only have the key and cancellation token as parameters. This might be on purpose to get people to want the source code :) Nontheless your vieos are great and an invaluable source of learning! Stay awesome!

[@MilanJovanovicTech]

It's more out of convenience - I already have videos on running EF migrations at startup and quite a few videos on caching, Redis, etc.

[@respectmathias]

How did you mount the volume locally, since there is a known issue with postgres and the windows file system e.g. "changing permissions of '/var/lib/postgresql/data': Operation not permitted", from what I've read you need to use a docker volume which is what I've done and works. But I would like to have it stored locally like you do.

[@MilanJovanovicTech]

I didn't do anything other than what's shown in the video.

[@PatricSjoeoe]

Really nice and easy explained! =)

[@MilanJovanovicTech]

Glad you liked it!

[@jamesreynolds3167]

With the new container features in the build tools, you no longer need a docker file. You can configure it all within the csproj

[@MilanJovanovicTech]

That's a nice improvement. Maybe I'll make a YT short about it

[@AbdusSalamAzad]

That is a great and simple explanation. Thanks @Milan

[@MilanJovanovicTech]

Glad it was helpful!

[@Tengri9090]

Docker container does not run when Visual Studio is closed. What am I doing wrong

[@MilanJovanovicTech]

If you start it from VS - I'd expect it to also shut down with VS

[@paolosartiano1454]

very thanks! fantastic your video!

[@MilanJovanovicTech]

Glad it helped!

[@antondoit]

Great as always

[@MilanJovanovicTech]

Glad you think so!

[@ricardk1443]

for me show the error volume sharing is not enabled when I tried to run by visual studio. I readed I should to enabled this in docker desktop but didnt have this option there

[@MilanJovanovicTech]

Try running it from the command line instead, maybe that'll give more info.

[@torrvic1156]

Thanks for another awesome video, dear Milan! But in my case I prefer using a secrets.json instead of appsettings.json. Is it possible to confer details of my connection string and other important configuration details somehow to my .NET WebApi container in compose.yaml file?

[@MilanJovanovicTech]

Yes, check this: docs.docker.com/compose/how-tos/use-secrets/

[@torrvic1156]

@@MilanJovanovicTech thank you so much Milan! Oh dear! This is exactly what I was looking for. Have no idea how did I managed to overlook it.

[@andersjuul8310]

Love it ❤

[@MilanJovanovicTech]

Awesome, glad to hear that! 😁

[@enis7x]

When utilizing Docker, how do you create EF migrations? The environment variable I have placed in the Docker container for the database connection does not get read when I run EF commands. Except for doing EF migrations, the application functions as intended and the db-server connection is working fine.

[@MilanJovanovicTech]

Yeah, you'll have to use the actual connection strings for running migrations. You can specify it manually when running EF commands. learn.microsoft.com/en-us/ef/core/cli/powershell#update-database

[@enis7x]

@@MilanJovanovicTech That works when you are updating the database but not when you are trying to create migrations. I'm still trying to find a workaround for the problem but thanks for the help, at least it works with the update command.

[@cleitoncarlospereira2006]

Hello Milan, how are you? When I run the application, the EF migration control table is created, however, the tables in the database are not created. Do I need to look at anything else?

[@MilanJovanovicTech]

Strange? The migrations are there? They're applied at startup - app.ApplyMigrations()

[@petiriesz1916]

Same for me, I'm using .net 8 maybe that caused the problem. I applied the migration via cli and it works.

[@mad_t]

It's a pity MS doesn't support podman (yet?), and you need extra actions to make it work with some limitations :(

[@MilanJovanovicTech]

Yeah, that's annoying. I'll see if there's a nice solution for that, and maybe make another video.

[@ahsansk84]

Your Api and database image tag is showing dev and latest respectively. But my all image tag is showing dev and also get an error. When I create an image using by Terminal, then tag shows latest. What is my problem.

[@MilanJovanovicTech]

I have no idea

[@TrungTran-le2ht]

Thanks for the content. I have a question What is the benefit when using DbContext directly compare to Repository/UoW pattern ?

[@MilanJovanovicTech]

Testability mostly, and code reuse (less duplication)

[@khushi0831]

Hi Milan, thanks for the video. but i have a postgres server and i need to specify the server name which is currently localhost. how or where do i need to specify that ? since it will be going to change on production server

[@MilanJovanovicTech]

Through the connection string? It should be the Host value

[@_miranHorvat]

In a real world scenario your app is deployed to multiple on-prem instances. Each instance needs its own settings and when you spin up a new updated version it has to use the same settings. So, where do you put your settings?

[@MilanJovanovicTech]

You want all of them to use the same settings? Why not something like Azure App Configuration?

[@_miranHorvat]

@@MilanJovanovicTech No. Different settings and local resources on different locations. When you update i.e. install a new version, it has to use the same settings as the old one.

[@timur2887]

should you clear the cache on updates or just update the cache?

[@MilanJovanovicTech]

Either one works fine. If you have the data ready, updating the cache makes sense.

[@lourenceplay9017]

How would you cache a ddd class that only has private constructor and Create() static method? There will be an error related to deserializing. I stuck on this error: System.NotSupportedException: Deserialization of types without a parameterless constructor, a singular parameterized constructor, or a parameterized constructor annotated with 'JsonConstructorAttribute' is not supported.

[@MilanJovanovicTech]

JsonConstructor - or use a different type to cache

[@Slyzor1]

What a great video, thank you very much!

[@MilanJovanovicTech]

You're very welcome!

[@muadgra3545]

Thank you, I have a question. The docker files are gone from the Docker Desktop when I close the visual studio. Do I need to run the program from the visual studio every time for it to be there?

[@MilanJovanovicTech]

The images are cached, the containers are wiped

[@baxram97]

But what if I have migration files?

[@MilanJovanovicTech]

You can automate that part - like I'm doing here (for local development) and applying it via DbContext.Database.Migrate() Or create the migration scripts, and run them yourself

[@timur2887]

when working with PostgreSQL containers you may use special docker-entrypoint-initdb.d volume to run initialization scripts inside newly created containers. They will be runned only once. You may use it to run any scripts on schema and/or import your data into database

[@DaminGamerMC]

You can apply migrations with code like taht, but to add a migration you have to change the startup project which is kinda weird. Is there a better way?

[@MilanJovanovicTech]

"dotnet ef" command line tool instead of using the VS console

[@TheOverBack]

So. Docker Compose is useful when you want to run the whole solution with their dependencies? But if I have 2 MVC apps, a background application with cron jobs, and a lot of broker consumers, they are dependent on db, Redis, rabbit, and mongoDB. And if I want to run one MVC website with consumers or another MVC app with jobs, do I have to create a lot of docker compose files to cover all combinations?

[@MilanJovanovicTech]

You can place all of them inside the same docker-compose setup

[@shahidwani6445]

Thanks milan sir

[@MilanJovanovicTech]

Sure thing

[@obasoroolakunleadeyemi]

Hello, I have a .NET app and migration has not be run but want to connect it with a postgres database using docker-compose

[@MilanJovanovicTech]

Hook into the container and create it manually

[@nathangrosvenor7661]

What would you say is better and faster to get to production out of .NET Aspire and Docker Compose

[@MilanJovanovicTech]

Aspire isn't meant for production workloads

[@nathangrosvenor7661]

@@MilanJovanovicTech Makes sense

[@baduy101]

Is there any chance that you can do same with SQL Server?

[@MilanJovanovicTech]

Just switch DB containers

[@nemanjaspasic1242]

Hello Milan how do you use swagger when you start docker since its not in development environment anymore? How did you solve this? In program.cs i changed settings for publish but i think it shoud not be done like that? Can you share with me your solution?

[@MilanJovanovicTech]

Start it from VS

[@hieunguyendanh5302]

video very good, do you up video connect dockercompose with sql server, please help me

[@MilanJovanovicTech]

Haven't made one

[@reneellup7561]

Hi Sir. Is it possible to use a remote Docker host (in Linux VPS) to run this?

[@MilanJovanovicTech]

Should be?

[@shaheenlive]

await cache.GetAsync doesnt have an overload which takes 4 args why ?

[@MilanJovanovicTech]

Check the docs

[@shaheenlive]

@@MilanJovanovicTech I did, with .Net9 , there is only two arguments, I followed a different video of your and created an extension method that basically does the same but do not take ct as an arg, for sample purposes it worked for me

[@ivandrofly]

4:59 - Connection string dns-naming

[@MilanJovanovicTech]

Hmm?

[@sanampakuwal]

I am getting this error: initdb: error: could not change permissions of directory "/var/lib/postgresql/data": Operation not permitted

[@MilanJovanovicTech]

How's that happening?

[@sanampakuwal]

@@MilanJovanovicTech nvm I found solution in some github thread, I need to exclude data from there like /postgresql/data is not working whereas postgresql/anything is working.

[@xanhxanh5097]

thanks great video

[@MilanJovanovicTech]

Any time! :)

[@MateStier-hd4nv]

What kind of postgres gui do you use?:)

[@MilanJovanovicTech]

Usually just DBeaver

[@andor7323]

I followed the steps but then I removed the docker-compose file and trying to add it back again but whatever I do I get this error "Unable to find any supported orchestrators for this type of project" any ideas?

[@MilanJovanovicTech]

No idea, haven't seen that issue before

[@andor7323]

@@MilanJovanovicTech Got it to work! Had to remove all the docker compose configuration and also go inside the project folder and remove all the docker related files that had been created. They didnt get removed automatically when I removed the docker compose project that's what caused the issue.

[@tienviet7442]

what is theme you are using in vs?

[@MilanJovanovicTech]

ReSharper

[@MalleshwarRao-m5v]

Please publish a video by adding docker support for your CQRS pattern project(Gatherly)

[@MilanJovanovicTech]

Just follow the same idea from here 😅

[@amimridul4278]

Just awesome

[@MilanJovanovicTech]

Thanks!

[@minimalstory]

good!!!

[@MilanJovanovicTech]

Thanks!!

[@lettuceturnipthebeets790]

just when Nana posted her Zero-to-Hero vid on Docker Compose 😅 looks suss jk jk it's fine to take inspiration from others)

[@MilanJovanovicTech]

Pretty cool, thanks for introducing me to a new tech channel on YT 😅 P.S. I can show you this was recorded December 12th last year. I typically have one-two months of videos scheduled ahead.

[@harkiratsingh4947]

Milan is right , I have been following him over 1 year now and I have seen docker before as well. Milan is Azure AD B2C in your plan as well for sometime ahead ?

[@pulsar1934]

This won't work requires https certificate

[@MilanJovanovicTech]

Run it from Visual Studio

[@ChandanDey-cw4zq]

Hi, I'm getting following error Npgsql.NpgsqlException: 'Failed to connect to 172.18.0.3:5432' public static class MigrationExtensions { public static void ApplyMigrations(this IApplicationBuilder app) { using IServiceScope scope = app.ApplicationServices.CreateScope(); using ApplicationDbContext dbContext = scope.ServiceProvider.GetRequiredService(); dbContext.Database.Migrate(); // error here } }

[@ChandanDey-cw4zq]

It's working. After second time execute

[@MilanJovanovicTech]

Probably DB did not start

[@timur2887]

you may healthscheck the postgres container and wait on statring you application in docker compose service dependency for service_healthy services: api: ... depends_on: server: condition: service_healthy server: ... healthcheck: test: ["CMD-SHELL", "pg_isready -d -U "] interval: 5s timeout: 5s retries: 5 start_period: 30s

[@miguelalves8209]

@@timur2887 Dude, thanks so much you and @MilanJovanovicTech, I was struggling for days with my project to work with migrations and orchestration! You guys are awesome! xD