Testcontainers have forever changed the way I write tests

  Рет қаралды 124,409

Dreams of Code

Dreams of Code

Күн бұрын

Пікірлер: 128
@yuu-kun3461
@yuu-kun3461 8 ай бұрын
Whenever you upload a video I am always impressed by just how advanced tools around programming have become. This is stark contrast with the legacy stuff I deal with daily.
@dreamsofcode
@dreamsofcode 8 ай бұрын
I am constantly amazed at how much things keep changing in software development. I often feel like it's a challenge keeping up with it all!
@Gregorius_
@Gregorius_ 8 ай бұрын
@@dreamsofcode it is! but thanks to your videos, it's slightly less challenging ;)
@savire.ergheiz
@savire.ergheiz 8 ай бұрын
They need to keep busy or will be kicked out by AI overlord 😂
@Tresla
@Tresla 8 ай бұрын
Small correction: At 5:20 you should use the "require" package instead of "assert". This is because you don't want to the test to continue running in this case. "require" will cause the test to immediately exit, whereas "assert" will just mark the test as failed, but continue executing the rest of the test, which might give you strange failure messages. Likewise at 6:59
@l_unchtime
@l_unchtime 8 ай бұрын
Just use the -failfast flag when running go test :) Besides, he's only using a single test so it doesn't really matter in this case. But you're correct that other tests would continue running without this.
@kirillgashkov
@kirillgashkov 8 ай бұрын
I love how informative, concise and useful these videos are. Thanks for your work!
@yudhiesh1997
@yudhiesh1997 8 ай бұрын
Instead of spinning up multiple test containers I just stick with one and make sure that I have setup and tear down code to keep the database state consistent across all tests. Spinning up multiple containers in parallel would put a lot of load on my CI/CD environment, so unless the tests are painfully slow I would just stick with sequential test runs.
@kowalkem
@kowalkem 8 ай бұрын
A good way of running tests in parallel against a SQL database is to use separate schemas. But unfortunately not all services have analogous capabilities.
@WarpingWombat
@WarpingWombat 8 ай бұрын
But isnt that the Point? You want your Tests to be isolated so you can Run them in parallel
@ImperiumLibertas
@ImperiumLibertas 8 ай бұрын
​@@WarpingWombat not necessarily. The purpose is to make test environments as reproducible and consistent as possible. That can still mean running tests synchronously. It all depends on your runner strategy.
@TehKarmalizer
@TehKarmalizer 8 ай бұрын
It really depends on the nature of your tests and whether any have side effects that could interfere with other tests.
@kaosce
@kaosce 7 ай бұрын
You can also use transactions rollbacked at the end of each test
@randomsde4390
@randomsde4390 8 ай бұрын
Watched this video and started to use testcontainers immediately in our CICD system. Really cool!
@dreamsofcode
@dreamsofcode 8 ай бұрын
This is amazing to hear 💜
@vijayramachandran3559
@vijayramachandran3559 7 ай бұрын
Fantastic advise! A friend in HR whom I respect greatly says that most problems have a large people component, it's never about only tech
@dreamsofcode
@dreamsofcode 7 ай бұрын
People / working in a team is always more challenging than code imho!
@catcatcatcatcatcatcatcatcatca
@catcatcatcatcatcatcatcatcatca 8 ай бұрын
If you are sponsored by docker, it would be cool to see more videos about using docker for testing and integration. I would especially like a video (or few) about how do you start a project from ”scratch”? You have a great ability to pick out the important bits and show only those in a clear manner. So while documentation and plenty of learning material exists, I think you really could add a valuable learning resource by doing a video in your style, from even more basic point of view. I’d encourage pitching the idea to Docker. Golang seems like a great language for this kind of content as tests are so integral for it. And the syntax is very easy. This would allow showcasing all of the code in the same video, which I feel would be essential for the consept. I’d love to see you answer simple questions through your usual straight-forward demonstrations, on an ”empty” project: How do you add and update dependencies with docker? How do you write the very first tests in test-driven project, and how do you expand from that? And lastly how do you go from a development image to a deployable, minimal image?
@dreamsofcode
@dreamsofcode 8 ай бұрын
This is a great idea that I can pitch to the team as well! I'm getting set up to do some streaming as well so this might also be something I can do then!
@danielcooke3243
@danielcooke3243 7 ай бұрын
really cool project, i always hate the boiler plate around setting up docker-compose tests, tearing stuff down properly, implementing health checks. this will make things much easier in future
@StrikevonNice
@StrikevonNice 8 ай бұрын
Will look into this, part of my problem is having x number of containers to start locally and manage that. Never liked mocking away a lot of things as your left with a lot of assumptions so this should help with that. At the very least should be good for CI/CD testing before it hits a testing/prod enviornemnt.
@romanzaiev
@romanzaiev 8 ай бұрын
Fun fact - can't be used with podman because test-container's implementation of DockerCompose relies on "compose ps --format=json" and it's still not implemented in podman. And nobody cares, unfortunately.
@ragectl
@ragectl 8 ай бұрын
"sponsored by Docker". They don't want more people migrating to Podman 😂
@WarpingWombat
@WarpingWombat 8 ай бұрын
Most of stuff Runs ob docker and docker aquired Testcontainers so there is No Point for them to make it portable with podman
@romanzaiev
@romanzaiev 8 ай бұрын
@@WarpingWombat there is an open PR in podman-compose about this missing feature since November and I believe it’s the podman’s issue (lack of compatibility)
@twocsies
@twocsies 8 ай бұрын
From the home page: "Want to give Podman or Rancher a try? Simply switch your local runtime for all your Testcontainers-powered dependencies. And with Testcontainers Cloud, you can even run them in the cloud on demand, while saving your local resources."
@DoCLov
@DoCLov 7 ай бұрын
One thing I find docker compose is good at is documenting what your environmental dependency in one place. It is good when you have to jump between many projects. I personally never faced with the issue of port binding usually because my tests run from another “test” container. All in all I agree there are merits to using the package there is also added complexity when orchestrated from another container by requiring to share a socket with the host
@cole.maxwell
@cole.maxwell 8 ай бұрын
Awsome! Thanks for linking that Git repos too!
@aredrih6723
@aredrih6723 8 ай бұрын
You can technically avoid the port conflict by not allocating port at all and using docker inspect to find the container ip (they all have one and the wiki page of docker inspect shows how to get it in an example). But having wrappers around popular services and automatic clean up does put test container apart.
@kiview42
@kiview42 8 ай бұрын
Conceptually, that is how we do it in Testcontainers :)
@coffee-is-power
@coffee-is-power 4 ай бұрын
The really cool part is that the core of testcontainers is just a http server that's running inside a docker container so you can make a library for interacting with it in any language and you don't have to implement the container management yourself
@linbynd
@linbynd 8 ай бұрын
Apart from the videos being extremely informative, I am amazed by how well the video is edited and put together? what software are you using to edit this amazing videos if I may ask
@dreamsofcode
@dreamsofcode 8 ай бұрын
Thank you! I use Davinci Resolve for about 90% of the work and then After Effects for any advanced animations!
@VinitKhandagle
@VinitKhandagle 8 ай бұрын
@@dreamsofcode simply amazing its a breeze to watch the subtle but eye catchy animations
@jedlechner3788
@jedlechner3788 8 ай бұрын
This is fire! Thanks for sharing. I can now do all of my personal development without deploying.
@PhilmannDark
@PhilmannDark 8 ай бұрын
If you don't want to pay the performance price for starting containers, try this approach: Add a test package to each service that you need. This test package contains all the test data that the service uses to determine that it works correctly (plus setup code, etc). Add a test dependency in your project to this test package and then use the test data to test your own code. If the service isn't under your control, you can write a small project which sends test requests to the service and verifies the responses. The requests and responses are then your test package which your project can consume. Every once in a while, you need to rerun this "collect test data" project but your tests will be several orders of magnitude faster.
@dreamsofcode
@dreamsofcode 8 ай бұрын
You can also start a container one off for all your tests in a package using the TestMain function in Go
@Peter1215
@Peter1215 8 ай бұрын
Excellent explanation and useful examples, thank you!
@Fonsecaj89
@Fonsecaj89 5 ай бұрын
test may become a nightmare when the deadline is close, this is a life saver!
@alandosman5002
@alandosman5002 8 ай бұрын
Nice video, So one note for tests, in the modules section you talked about parallel tests and stuff, but one thing, tests should be written independently, it should behave independent, it should make no problem for other tests, so you tests should be deterministic.
@goffkock
@goffkock 8 ай бұрын
Those wait parameters to postgres preconfigured container should really be included in the module though. One issue with it is for example spinning up Kafka on each test run, takes much more time than hosting it in docker manually
@dreamsofcode
@dreamsofcode 8 ай бұрын
You can still spin up a single instance for all your tests using test containers! You would do so in TestMain using Go
@joaowiciuk
@joaowiciuk 8 ай бұрын
Thanks for sharing, I'll consider using testingcontainers in the future
@RomualdBrunet
@RomualdBrunet 8 ай бұрын
This is great to run locally, unfortunately this is not usable when running inside a docker container run within a CI. I have yet to see some library that would integrate with both environments (use mysql container spawned by CI, but spawn a local container when ran locally)
@dreamsofcode
@dreamsofcode 8 ай бұрын
Github actions works inside of a docker container. So does gitlab which also uses them
@vikingthedude
@vikingthedude 8 ай бұрын
So you’ve covered test containers, now it’s time for dev containers!
@dreamsofcode
@dreamsofcode 8 ай бұрын
🫡
@Alex-lu4po
@Alex-lu4po 6 ай бұрын
Why do you say that docker compose has port collision and still have port collision locally?
@databasemadness
@databasemadness 8 ай бұрын
There is more value in this video than i handle!
@ruslan_yefimov
@ruslan_yefimov 8 ай бұрын
My team lead was amazed when I've made a deployment pipeline which applies all migrations with dbup, lol.. I'm a new guy with like a year of experience (THEY DO ALL THE DEPLOYMENT MANUALLY)
@tswdev
@tswdev 8 ай бұрын
Years ago I made some some functions to automatically control docker from tests. It was cool but not all too great. I need to have a look at this Testcontainers stuff (actually have the tab open for some time, just havent had the time). Thanks!
@GigiSher-td2zs
@GigiSher-td2zs 5 ай бұрын
Thank you for the nice video. Very helpful! I really want to try using localstack for unit testing my code calling s3, kinesis, etc. Are the docker images free to use in our unit tests? Any license issue we need to consider?
@SimGunther
@SimGunther 8 ай бұрын
0:38 Are we to assume the starting state of the accumulator is +/- infinity, int_min/max, or zero? If we're saying flakiness is a negative value when all is said and done, are we trying to get that number closer to zero? That's my assumption because there's no way that we can have a positive end result, meaning there's less flakiness than anti-flakiness (no idea what that means here). I get that it's supposed to be shorthand for a codey joke, but reduce in this notation means that you flatten a container into a singular dimension, which could mean either reducing a list of events with an initial state (acc) into a final state structure or flatten an N dimension container into a single value/N-1 dimension container.
@dreamsofcode
@dreamsofcode 8 ай бұрын
Bruh
@SimGunther
@SimGunther 8 ай бұрын
​@@dreamsofcodeFalling victim to Muphry's law, it turns out that the version in the video assumes the 0th element is the initial value and we can iterate through the rest of the array starting from the 1st element, but when the array is empty, we though an error. Lucky that there's at least one test, right? 😅
@dreamsofcode
@dreamsofcode 8 ай бұрын
@@SimGunther 🤣
@31redorange08
@31redorange08 8 ай бұрын
It fails either way, because it would try to subtract the flakiness of the 1st test from the 0th test (object). NaN.
@nijolas.wilson
@nijolas.wilson 6 ай бұрын
Why do I want to test Redis, S3, SQS, etc? If I can't be reasonably assured that some third party tool works then I probably shouldn't use it and pick something more reliable. Seems like a very expensive way to avoid unit testing? 🤔
@AndreaIppolitoIppo
@AndreaIppolitoIppo 4 ай бұрын
@@nijolas.wilson my thoughts exactly. TCs are starting to make their way in some of the projects I work on, but honestly I'm more of a unit test guy. If I need to spin up an actual Kafka container to ensure the messages I'm producing are actually delivered/consumed, I'm testing Kafka, not my code. Not to mention that a TC test can get quite complex quite fast, to the point you're eventually kinda writing a compose file but in Java (or whatever language you happen to be using), with env overrides and whatnot. Not my cup of tea.
@flwi
@flwi 8 ай бұрын
That's very handy. Thanks for the explanation!
@imbetterfyi
@imbetterfyi 8 ай бұрын
I'm curious how heavy the added overhead of spinning up so many container instances (esp. multiple database instances) can be considering some code bases have thousands of tests to be run + how much delay it introduces
@KamiKagutsuchi
@KamiKagutsuchi 8 ай бұрын
I unfortunately can't use testcontainers at my job because it requires you to be able to run docker-in-docker to run in the ci pipeline which is apparently considered a security risk and therefore not allowed FeelsBadMan
@dirty-kebab
@dirty-kebab 8 ай бұрын
Where can i find your theme and font? Im sure i probably already have it somewhere and have just lost it?
@SIMULATAN
@SIMULATAN 8 ай бұрын
Glad to see valkey getting so much love ❤
@kevinmarques9334
@kevinmarques9334 7 ай бұрын
But what if the container stops right after it shows the log message? I'm having this problem in a small application and I can't find any function that checks the health of the container even after it stared... :(
@dreamsofcode
@dreamsofcode 7 ай бұрын
You can use some other waiting protocols, such as an open port
@arielmolina6277
@arielmolina6277 7 ай бұрын
Awesome video, thanks for sharing !
8 ай бұрын
Thanks for the video. The wait forlog is the next flakey part.
@dreamsofcode
@dreamsofcode 8 ай бұрын
You can wait for ports to be open and other things as well. It's pretty useful
@skeplo12
@skeplo12 8 ай бұрын
May I ask what font you are using for coding?
@prashlovessamosa
@prashlovessamosa 8 ай бұрын
Can you please make one on Delve how to setup it and start debugging using it pleaase.
@intimidate123
@intimidate123 8 ай бұрын
you have a link to your dotfiles?
@JoeyJurjens
@JoeyJurjens 8 ай бұрын
Very cool, thanks for sharing!
@NostraDavid2
@NostraDavid2 8 ай бұрын
I like to live dangerously and mock it all (if I had my way - alas, my team also chose test-containers, now I test 10x as slow)
@NostraDavid2
@NostraDavid2 8 ай бұрын
Spinning up a single instance is fine, but not when you need Postgres, Hadoop, Kafka and whatelse all at the same time.
@ImperiumLibertas
@ImperiumLibertas 8 ай бұрын
These really shouldn't be used as part of the dev workflow. They're good for CI/CD.
@AnythingGodamnit
@AnythingGodamnit 8 ай бұрын
Isn't this just a distinction between unit tests and integration tests?
@swyxTV
@swyxTV 7 ай бұрын
How are these videos made? With the text and animations? Without a giant motion graphics budget?
@dreamsofcode
@dreamsofcode 7 ай бұрын
All me at the moment! Basically a lot of time and effort and learning how to do motion graphics. I've got a couple of videos coming out that are really pushing my abilities!
@debemdeboas
@debemdeboas 8 ай бұрын
what plugin are you using for that column line? love ot
@yonatanavidor8299
@yonatanavidor8299 4 ай бұрын
Love the small jabs at the recent moves redis made
@flo4604
@flo4604 8 ай бұрын
Geat video, maybe its time to get into tests :) Dragonfly would also be something you could take a look at instead of valkey
@covle9180
@covle9180 8 ай бұрын
I came for the cool tech, I stayed for the digs at redis
@faysoufox
@faysoufox 8 ай бұрын
Great video, thanks
@Argosh
@Argosh 6 ай бұрын
Listen, if they didn't want us to test in prod, why is all that juicy test data in the prod system? Have you thought about that? Have you? (Send help. Please.)
@seandougherty3022
@seandougherty3022 8 ай бұрын
another solid video. nice one.
@Malix_Labs
@Malix_Labs 8 ай бұрын
Great video However, 11:15 do not use testcontainers for development (there's devcontainers for that)
@Rundik
@Rundik 8 ай бұрын
I wrote my own testcontainers implementation a few years ago in a couple of hours. It was just a few hundreds lines of code.
@knucklecorn
@knucklecorn 8 ай бұрын
too bad it doesn't work with podman
@N0FPV
@N0FPV 8 ай бұрын
I don't want my docker containers to be "Typed"
@Fudmottin
@Fudmottin 8 ай бұрын
That's pretty cool! And wow you type fast!
@justahumanwithamask4089
@justahumanwithamask4089 8 ай бұрын
Was hoping he showed how to run firefox in docker
@dreamsofcode
@dreamsofcode 8 ай бұрын
If you still wanna know I can show you on discord!
@justahumanwithamask4089
@justahumanwithamask4089 8 ай бұрын
@@dreamsofcode I think I have a gist, create a container with x11, then install the program and alias the entire container? If I hop on discord I will take too much your time as I never used docker or podman before yesterday when I saw your other video. Actually I've never even heard of podman before yesterday, I just installed it because I thought docker couldn't be installed with pacman whereas podman could plus it's rootless and I most likely wouldn't need systemctl to autostart docker since it was a rare use case.
@justahumanwithamask4089
@justahumanwithamask4089 8 ай бұрын
I think I have a gist, create a container with x11, then install the program and alias the entire container? If I hop on discord I will take too much your time as I never used docker or podman before yesterday when I saw your other video. Actually I've never even heard of podman before yesterday, I just installed it because I thought docker couldn't be installed with pacman whereas podman could plus it's rootless and I most likely wouldn't need systemctl to autostart docker since it was a rare use case.
@justahumanwithamask4089
@justahumanwithamask4089 8 ай бұрын
KZbin deleted my other reply where I at your username. New youtube policy I guess?
@dreamsofcode
@dreamsofcode 8 ай бұрын
​@@justahumanwithamask4089 I saw it and tried to respond but it wouldn't let me, KZbin comments are still a mystery to me. You're close though with how you'd do it! The image I used in the video had a VNC server inside so I was able to connect into it using VNC
@KevinHaeusler
@KevinHaeusler 8 ай бұрын
Everyone has a test environment, some people have a seperate prod environment. :)
@sof1an
@sof1an 8 ай бұрын
These seem great as part of a cicd pipeline, not in the dev workflow. I experienced a lot of headaches waiting for containers to start just trying to run locally.
@Im_Ninooo
@Im_Ninooo 8 ай бұрын
this is awesome!
@dreamsofcode
@dreamsofcode 8 ай бұрын
Thank you!
@giant3909
@giant3909 8 ай бұрын
Seems great for specific uses cases where you have to replicate complex external services like AWS, but for something like postgresql there is a lot of boilerplate code for every single test. My current project runs 350 tests, imagine having to setup a postgresql test container for each of them
@dreamsofcode
@dreamsofcode 8 ай бұрын
In the teams I've rolled this out with we just made a reusable function or started the container up in TestMain. You only have to spin up a single container for one of them. I showed this in the video.
@NutzlastB0hne
@NutzlastB0hne 8 ай бұрын
Can't wait for this awesome project to get locked behind licensing fees 😬 Honestly, I'm getting kinda jaded with these "it's free... and now it's not!" rug-pulls. Of course there needs to be a way to support FOSS projects, but that approach can't be it...
@thenaman047
@thenaman047 8 ай бұрын
there I am, first of em all
@Laflamablanca969
@Laflamablanca969 8 ай бұрын
Honestly, a docker compose file is easier than this. If you aren’t cleaning up after your tests, that’s on you.
@dreamsofcode
@dreamsofcode 8 ай бұрын
Objectively false.
@Laflamablanca969
@Laflamablanca969 8 ай бұрын
@@dreamsofcode lol definitely not. Obviously you need to push for testcontainers because docker is the video sponsor and they’re paying you to say that. With that said, put your money where your mouth is and make a video benchmarking with and without testcontainers. Docker compose workflow: 1. Connect to running pgsql container 2. Migrate db schema 3. Run test code 4. Rollback db schema Testcontainers workflow: 1. Start pgsql testcontainer 2. Wait for initialization trigger 3. Connect to db 4. Migrate db schema 5. Run test code Not only is it more steps, it’ll also take much longer to run every test individually. It may be beneficial if you can run your tests in parallel, but I’m still skeptical as to whether upping and downing containers is quicker overall. If you show me your way is less code and faster, I’ll shut my mouth and use testcontainers from here on.
@herman6214
@herman6214 8 ай бұрын
First you tell us you dont have redis running, then you cant start your redis container because you do have it running. What gives?
@dreamsofcode
@dreamsofcode 8 ай бұрын
I think you may have misheard me
@berndeckenfels
@berndeckenfels 8 ай бұрын
Testcontainer modules should really offer more random passwords by default
@saggitt
@saggitt 5 ай бұрын
@@berndeckenfels why would you need a random password in tests?
@berndeckenfels
@berndeckenfels 5 ай бұрын
@@saggittto protect developer workstation and for compliance, it’s rather easy as the function thr query the password exists already
@itsfkf6106
@itsfkf6106 8 ай бұрын
second also hoi DoC
@brianmc1575
@brianmc1575 8 ай бұрын
first
@dayvie9517
@dayvie9517 8 ай бұрын
Seems like a waste of energy and time when u can just mock 😂
@dreamsofcode
@dreamsofcode 8 ай бұрын
I think mocks have their place. But they're not a replacement for testing against real services. You should always have some integration tests, otherwise you're just simulating behavior.
@dayvie9517
@dayvie9517 8 ай бұрын
​​​​​​​​​​​​@@dreamsofcodewell you are too just simulating behaviour with your test containers. Your test containers don't run in your production env or other staging envs? Still missing the point here. What are you really testing here? If your dbms acts as you expected? Also integration testing doesn't mean 'testing against dbms' but testing the integration of different software modules. You can integration test without simulating dbms instances or ever getting near a data layer. A module doesn't have to be the data layer module which goes against a dbms 😅. Of course you want to do unit, integration and system testing. Never said anything against it. Just curious about your motivation for this.
@eNtrozx
@eNtrozx 8 ай бұрын
1. Why not test with actual components when it's that easy? 2. Sometimes mocking is so complex compared to setting the env in the service. For example try mocking a Redis ReadWriteLock, It's a hell of complexity
@DoCLov
@DoCLov 7 ай бұрын
As you said it is a balance of getting confidence in your code, getting feedback faster, maintenance and reliability long term. Any testing is a optimisation against those 3 axis. For example you can mock and sql connection (and you should for handling exceptional situations) but how would you trust your sql works with this particular db version, I don’t. It is a healthy mix and testing against a container gives you a decent confidence and it is much faster and safer feedback than running against a production db without an intermediate stage.
When RESTful architecture isn't enough...
21:02
Dreams of Code
Рет қаралды 286 М.
Using docker in unusual ways
12:58
Dreams of Code
Рет қаралды 465 М.
My scorpion was taken away from me 😢
00:55
TyphoonFast 5
Рет қаралды 2,7 МЛН
It works #beatbox #tiktok
00:34
BeatboxJCOP
Рет қаралды 41 МЛН
It’s all not real
00:15
V.A. show / Магика
Рет қаралды 20 МЛН
How Unorganized Can a Roblox Game Get?
28:27
ByteBlox
Рет қаралды 3,2 М.
The Absolute Best Intro to Monads For Software Engineers
15:12
Studying With Alex
Рет қаралды 675 М.
The purest coding style, where bugs are near impossible
10:25
Coderized
Рет қаралды 1 МЛН
Debugging Code in Neovim (Rust, Go, Python etc)
20:32
Sliced Array
Рет қаралды 2,1 М.
The cleanest way to use Docker for testing in .NET
13:36
Nick Chapsas
Рет қаралды 91 М.
If you're not developing with this, you're wasting your time
14:30
Articulated Robotics
Рет қаралды 312 М.
Dependency Injection, The Best Pattern
13:16
CodeAesthetic
Рет қаралды 902 М.
The standard library now has all you need for advanced routing in Go.
13:52
SQLc is the perfect tool for those who don't like ORMs
28:11
Dreams of Code
Рет қаралды 112 М.
The Best Way To Use Docker For Integration Testing In .NET
19:34
Milan Jovanović
Рет қаралды 38 М.
My scorpion was taken away from me 😢
00:55
TyphoonFast 5
Рет қаралды 2,7 МЛН