When Test Driven Development Goes Wrong

[@thought-provoker]

Gotta steal that sentence, "If our tests are difficult to write, it's telling us something about our design." Yeah. Exactly. That's what I observe, too. Over, and over - and over again. When we end up with a test setup of 72 steps and a teardown of 90 steps for a single database update, that's not a test problem.

[@ContinuousDelivery]

💯

[@LarryRix]

Here is a place where Design-by-Contract can really shine. Consider a truth about your software-namely-your software is being written with setup and tear-down as a part of its normal operation. When your software runs, its job is to setup and then to (possibly) tear-down. What you really want in the flow of your operational software (at beta or alpha testing) are the assertions of Preconditions and Post-Conditions. If your software has these "contracts", then it is consistently "testing" itself as it is running. Sometimes it will be running because of an automated test and sometimes it will be running because of manual testing. Either way-the DBC assertions are there to bird-dog that operational running without any effort beyond the writing and inclusion of the assertions. In Eiffel, you get this as a gimme because of Design-by-Contract. For my part, I realized that I'd like the DBC assertions to follow my code into production at some level, but not to where my software fails, but to where assertion failures log the failure along with the data state that caused the failure. This way-I get a behind-the-scenes snapshot of real-time and real-life failures where my users are doing their own thing for their own reasons (and not me "guessing" at how they will use my code). This is something that the TDD to which Dave refers cannot really do. Why? Because the TDD to which Dave (and others) speak is always external to my production code. However, DBC assertions are internal and integrated into the code. These assertions can be removed at will or transformed when it comes time for production code. NOTE: Spreading your test code into your working code through DBC assertions is lighter in computational-cost than you think. In practice, I have rarely noted significant slow downs and where they do occur, I find that they can be selectively disabled in production through a number of means of isolating them and shutting them "off". However, in pre-production testing (automated or manual) having them in-line is imperative to gathering trust about my code.

[@tiagodagostini]

But that is why I am against the hardcore TDD. Development should not be DRIVEN by test, should be SIDE TO SID with it. When something other than understanding the domain drives the development, you end up with BAD design. TEst is part of development, not the reason (Driven means is is the thing that pulls all your moves into the develoment, i.e the reason) of the development.

[@rafellus1]

-- Anti-patterns -- 4:03 - The Lair 5:52 - Excessive Setup 9:54 - The Giant 12:57 - The Mockery 16:22 - The Inspector

[@onatkorucu842]

*liar

[@PutsOnSneakers]

@@onatkorucu842 *liar liar, pants on fire

[@drcl7429]

My university tried to teach unit testing to us by giving us already written software and asked us to apply unit tests to it. At the time I remember hating it because it was mind bending trying to get the tests to work due to the ridiculous amount of coupling meaning stupendous amount of setup. I didn't at the time really understand why I was finding it so hard but I knew something wasn't right. I couldn't imagine anyone would do this in a job and there had to be a better way. The tests that me and others designed got to about 80% coverage and that was seen as acceptable for assessment - it was probably impossible to go much higher. More than a year later they reintroduced the the concept of TDD (sort of) in a module called advanced software concepts which cited Martin, Fowler, Beck and the Gang of Four but it was all just piled in together - not making it clear how each part fit with the others and that TDD is basically the lynchpin to all the other ideas or emphasising the importance of the cycle and fail first. It's no wonder enterprise doesn't know how to implement it. The way this was all taught almost made me not want to pursue work in the industry if this was what even some of my days were going to be like.

[@PlayRiteProductions]

0:00 - Intro 2:39 - TDD 4:03 - The Liar 5:52 - Excessive Setup 9:51 - The Giant 12:54 - The Mockery 16:19 - The Inspector Amazing video with excellent points! I made some timestamps for review.

[@ContinuousDelivery]

But watch the other bit too, there is some good stuff😉 😎

[@rpopov71]

Clear, practical, not dogmatic. I like it. And I like also that Dave considers there is some design before writing the tests. TDD is not an alternative to design - the tests actually validate the design before implementing it.

[@kguentube]

This video is gold, as usual in this Channel.

[@ContinuousDelivery]

Thanks 😁

[@MagnusAnand]

It is indeed

[@urzytkownikYT]

Yeah, its common on this channel. So valuable i came across it

[@nickj69]

Deming said it (maybe) - "when a measure becomes a target, it stops being a measure."

[@ContinuousDelivery]

Yes, Demming was a pretty smart chap :)

[@JanMagnusson72]

This is what I am saying as well when I try to encourage taking up TDD. That TDD is not about ensuring that you have good coverage or some such. The main benefit is that it gives you feedback on your design before you have spent lots of time implementing it. If your design is hard to write tests for, its probably not a good design, but since you didn't yet implement it, going back and improving the design is cheap. TDD is a way to test design in addition to code so it pays off in better design, better tests AND better more error free code.

[@ContinuousDelivery]

💯

[@tonyjaradev]

Dave, as someone living somewhere without much access to information like this, or not surrounded at all by developers, I greatly value this videos. Thank you!

[@ContinuousDelivery]

Glad to help

[@SteinGauslaaStrindhaug]

One of my first experiences with debugging someone elses test, was a test with lots of mocks that happened to fail when I improved the code it was "testing". I moved an expensive repeated call to a procedure on an argument from inside the loop to outside the loop; as it was just recalculating the same value every time anyway. The test then suddenly failed because it expected it to call that procedure 8 times and got only 1. Once I managed to understand the "test" it turned out it mocked the database, the message objects and the webserver etc. and the only part of the actual code it were actually testing was that the loop were written inefficiently. This is the main issue I have with TDD, or at least "mock TDD" is that when I'm working with projects that have lots of "tests" written by TDD-enthusiasts (that rarely understands it) is that the tests often don't catch bugs, it just enforces existing bugs; so almost always when I cause a test to fail when refactoring it turns out the test was asserting bad code; and when I actually introduce bugs it normally passes just fine because the "tests" mostly just tests that the mocks mock and the fakes fake and very little else. So many articles about "TDD" just wastes so much time on fancy ways of mocking the entire environment in order to have "test coverage" rather than just assuming some parts of the environment should just work.. If you're not making a database, just using a 3rd party database, why would you waste your time testing that SELECT selects and INSERT inserts? If the database doesn't do that it's pretty worthless, and even worse testing that a mock-database mock inserts when you ask it to insert something is just stupid. If the code you're testing simply takes some data, validates it and forwards it to the database; just test the validation if it's complex enough to warrant it, and don't bother testing the database. I've also seen "unit tests" that test getters and setters on message objects... wtf? Are they trying to verify that language features work? That variable assignments assigns and return statements return? Are they even aware that the testing framework is written in the same language and presumes those features works in order to work. And even more stupid, is code that basically test getters and setters on a mock objec.

[@SteinGauslaaStrindhaug]

I think mock frameworks is part of the problem. So many articles about testing is just raving praise of some fancy framework and all the fancy mocks it can do, making it seem like thats the point of testing. And the ridiculous idea of test coverage as something magically good, causes people to write test code for pointless stuff that doesn't really require testing.

[@SteinGauslaaStrindhaug]

I'm a primarily frontend developer, where in my oppinion automated tests mostly don't work; as in frontend the main challenge is making something that "feels" intuitive for humans; and testing that would require perfecting human equivalent artificial intelligence. I do occasionally write more logically complex things and when I do I ofte make ad hoc tests up front or while coding; though I rarely use full testing frameworks because it's too much work to get it work for client side code for very little gain. But I do sometimes change some tests into sanity check "asserts" in the code itself to give more detailed warnings if someone in the future starts to feed it garbage inputs or changes the logic in a way that would break the logic.

[@SteinGauslaaStrindhaug]

The weird thing is when I or others ask TDD experts/enthusiast on online forums like StackOverflow etc. who insists that TDD is useful for everyone making any type of code, about how to test typical things a frontend developer writes most of the time; some of them responds "oh, you don't write tests for that" and others (the in my oppinion crazy enthusiasts) respond by talking about frameworks to mock the database, mock the webserver, mock the browser and record clicks and send screenshots of renders to some fancy webservice etc. Examples: - CSS and layout; which is a pretty large and probably the hardest part about frontend because the success is mostly determined by how humans work. (Fortunately almost everyone but the crazyest ones agrees this is untestable with unit tests) -interactive elements client side: events, DOM-manipulation, animations, layout calculations etc. (Some say you should only test the isolateable "units" like layout calculations and just leave the rest untested which is what I actually do. But quite a few insists that making mock buttons and put them in mock DOM models and fire mock events on them; or making a mock webserver that always takes a mock ajax call and does nothing with it, is worthwile) - Serverside endpoint code that is mostly plumbing inputs from request to and from other backend code or directly to database. (A suprising amount of people think that it's worthwile testing this by making mock HTTP data and mock database or mock interfaces to other backend code that presumably already is tested elsewhere; so that in the end you're testing that passing values from input parameters to a function call does work; even though how could it not unless the server loses power in the middle?)

[@darshandhabale143]

@@SteinGauslaaStrindhaug yup your comments make sense. What are your thoughts on using Selenium to emulate a user to test the UI.

[@SteinGauslaaStrindhaug]

@@darshandhabale143 booting up a headless browser in the tests tends to be really slow, adding such a test will ensure nobody runs them regularly while developing. Also DOM based tests are very flaky and pixel based ones even worse because minor layout adjustments might break them. If the test is DOM based (using xpath or CSS selectors etc) it's very possible to completely break the UI for users (visually obscuring a button somehow) while the test still works. Making them is also very complicated and time consuming so if they are likely to break all the time, it will tend to be neglected. If such tests is useful at all, I think only one or two very simple tests that mostly just is a "smoke test" to verify that the page loads at all; and that is only run automatically by the CI tests and not locally; is the only useful kind. Useful test suites test only things that are likely to break and runs in less than 5 seconds ideally less than 1 second; if they take much longer you simply won't run them very often or at all.

[@aly-bocarcisse613]

👏🏿👏🏿👏🏿 Not a heavy practitioner of TDD here however even I did see those mistakes over and over. This is gold, on top of gold on top of reflection & care. Thank you !

[@Stevexupen]

The most annoying thing of developer with tdd is there's some that has become so (falsely) over-confident of their code because all their test case passed that they use it to (wrongly) argue that the fault is not in their code while more often than not it is..

[@gubx42]

That's the thing I wanted to be addressed. It is easy to write code just to pass the test. An analogy: if you tell a lazy student what's on the test, that's the only think he'll study, and he will get good grades even though he doesn't know much about the subject. That's why teachers make students learn their lesson before they get the test.

[@brianwest7344]

TBH I've rarely met a SW developer who doesn't argue his code is not at fault initially, even when it plainly is.

[@DhanarAdiDewandaru]

i think this is an organizational problem. the solution i think is for the leadership to tell teams involved to check their code anyway and make sure that the check is done properly. :D

[@TheChodex]

Having bad tests in project that pass reminds me of that meme where dog is sitting inside a burning house and saying "this is fine"

[@somerandomchannel382]

Every test should be part of something that if failed result in an incomplete app. A test doesn't have to point at something visual or a component. If you setup an api client test that all GET request should work. This test should be green until you cannot receive information with "get". A test should be more of... AHHH my app BROKE... what happened! .. 'reading test statement' 'oh ... that part is falsy. Another bad analogy is that your code is a long line.. and you set colored stone along the line. making it easier to track what part of the string needs to be repaired.

[@EwaldDieser]

Learned a lot that I can apply in my work. While doing code reviews I often have difficulties to explain what I don’t like. A list like that is really helpful!

[@ContinuousDelivery]

Glad it was helpful!

[@BenDavis78]

Great video! Many of the pitfalls explained here are due to not using a TDD approach. I would love to see some advice for developers tasked with adding tests to existing projects that were not originally developed with TDD.

[@ContinuousDelivery]

Yes, that is a problem, and I think a reason why people sometimes dismiss it. They think that TDD is something that it is not.

[@zebcode]

@@ContinuousDelivery Isn't this still the case if you test afterwards though? Sure you need to refactor your code but if you can't test it then it's not decoupled regardless of whether TDD is used or not?

[@satyrkrieg]

Nice video. A suggestion, when you put code on the screen, please place it in a way that is clearly visible.

[@tordjarv3802]

On the Inspector anti-pattern. I am a nuclear physicist and are developing a large scale simulation code for atomic nuclei, and I do use TDD. However, in my code I have functions that produce transformation matrices which can be very large (with very large I mean that they can take several GB just to store the non-zero matrix elements as single-precision floating point values) in production code, so naturally I need test cases that cover these larger cases. The problem is that because they are so large and that I need to check very many versions of them it is unpractical to store examples of each and every test case, so instead I need to check that they fulfill specific mathematical properties. However, in the production code there should never be a need to check these properties, since it would be a waste of expensive computing time to check it in production runs when it should be guaranteed if the code is correct. This means that the code to check these properties are specifically written for the tests and only used in the tests. This to me sounds like what you describes as the Inspector anti-pattern, but I don't see any practical way to get around it. Other than that I think this is a great video and I really think that I learn good stuff from all your videos, so thank you.

[@justinbehnke8724]

I am shocked at how spot on the symptoms and causes of these anti patterns are. Well done you read me like an open book!

[@samm7334]

Minor correction: the getLastBuild function is indeed not called in the production code but it is still used productively. The user can call it themselves in the pipeline which can be quite useful. The point about the inspector anti-pattern still stands. It was just an unlucky example.

[@ContinuousDelivery]

Thanks for the correction.

[@alessandrob.g.4524]

Ports and Adapters / Clean / Hexagonal Architecture help a lot with the separation of concerns and therefore making your system testable.

[@ContinuousDelivery]

I think that you will like this video, where I talk about using Ports & Adaptors: kzbin.info/www/bejne/e4Srn2hpot51bpo

[@alessandrob.g.4524]

@@ContinuousDelivery Thanks, Dave. I'm gonna check it out.

[@leandronunes85]

I really enjoyed the video but I think that the Jenkins example was unfortunate. By transforming the functional test on Jenkins into a unit test on JenkinsUrl (or whatever the class name was) we’ve also moved from testing the requirement (“Jenkins should not have the word localhost in its url”) to testing an implementation detail. I’ve experienced times and times again that implementation details like these are not actually used in the use case I was working on (in the Jenkins example, what guarantees do I have that JenkinsUrl is actually being used?).

[@DryBones111]

By following this approach throughout the system you gain the guarantees. Proper TDD prevents you from doing "downstream" testing because of the trust you put into the tests taken "upstream". In this case, we create a JenkinsUrl value object. Our interfaces should then be using the value object in place of what would otherwise be a string or URL object and essentially have that behaviour propogated through the system. This also creates a single source of enforcement for the requirement with a single test for it. If the requirement changes, we need only to change this part of the code and this one test. Hence, good design becoming part of TDD.

[@dotnetapp7503]

we implemented TDD for Backend code around 2 years ago and are very happy with this. On the other hand for the frontend we ditched it we are allways writing the test afterward the reason is simple, Objects there change much more frequently, as a quick example is that we sometimes just dont know if a checkbox(boolean) or maybe a textbox(string/enum) or even a dropdown(array) is the best way to approach an problem (on an ui/ux friendly way), so it would be a waste of time to write the test beforehand.

[@darshandhabale143]

I feel for frontend acceptance testing with selenium should cover all the testing requirements, and unit test if the component is complex enough

[@ehx3419]

I often found myself in the situation where I wrote tests for classes someone else made long before I even saw them. So while this advice (which boils down to "use actual TDD, not fake TDD") should work, it's not always a possibility. Although, if you're sitting down to write tests _because_ you struggle to understand what the class is supposed to do, maybe you're not the right person to write them.

[@kiseitai2]

Hmm, the software company I work for has a 30 year old codebase and we recently switched to C# based unit tests. I see what you mentioned about writing the code before the test and indeed we have many tests that are giants. However, at this stage, you almost have no choice in some areas because you might need to test some functionality that is dependent on another functionality. At that point, I have to test both in one test function. Despite that issue, I do think writing the tests has helped me improve areas (including code I previously wrote), so I agree with your initial statements about TDD.

[@ContinuousDelivery]

It is a completely different thing to introduce 'testability' to existing code. I talk about techniques for that in this video kzbin.info/www/bejne/pl7SiHt7m714jNU

[@frank-michaeljaeschke4798]

@@ContinuousDelivery That would have been the point at which I would otherwise have recommended the book "Working Effectively with Legacy Code" by Michael Feathers. But you mentioned it already in your video. At that point, I realized you wrote with Jez Humble the famous "Continious Delivery". In my opinion the best book on Continuous Delivery / Continuous Integration.

[@ContinuousDelivery]

@@frank-michaeljaeschke4798 Thanks, glad you liked it.

[@ehx3419]

Any time you find yourself using reflection in tests is also an Inspector symptom.

[@a0flj0]

There is one case when complex setup is unavoidable, IME - when you test stuff that's strongly coupled to framework interfaces. Strongly coupling your code to the framework is often a conscious tradeof for lower development effort. There are also tests that might look odd, but are actually legit - tests you add late, as a reproduction of a bug report.

[@ContinuousDelivery]

Yes, but my preference is to avoid frameworks that force me to “strongly couple” to them. I usually prefer to implement my own abstraction between the bulk of my code and almost any third-party tech. I break this rule for some common library code, but very rarely for frameworks.

[@a0flj0]

@@ContinuousDelivery Me too. But sometimes, when a framework or a library provides excellent support for the task, it's simply unjustifiably expensive - you either reinvent the wheel or create an unnecessary wrapper for the wheel. IME.

[@ferrucciobongianni]

Hi Dave. Great video, as usual. I’m a big fan of yours. Just a note on the public method to get the last build in Jenkins. I use Jenkins quite a lot and it’s far from being perfect. However, in the specific method you couldn’t find in Jenkins code base, I just wanted to mention that Jenkins is heavily based on plugins, so that method it’s very likely exposed to allow plugins to get info about the last build. In fact, that’s even useful in scripted pipelines (accessed via Groovy). Once again, thanks for your work and knowledge sharing!

[@tyronfoston7123]

Who is this guy??? He's making my life easier. I need to buy this man a drink or two

[@kaptenhiu5623]

buy his book instead. It's "Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation" by David Farley

[@petermanger9047]

The most powerful important statement that really slapped me... "Testing is a tool to achieve an outcome".

[@ContinuousDelivery]

😊 😎

[@klaussfreire]

I think it took me near a decade to learn this on my own. Not understanding this, I think, is truly the root of all problems with testing. When you know what you want to achieve with your testing, everything else just falls into place.

[@gunderd]

Some real gold in there Dave. Thank you! This is why I prefer the term "test driven design" over "test driven development" - it emphasises the value as a design time activity. I worry about that those teams who, with best intentions, introduce metrics like minimum test coverage % without first training teams in the art of design.

[@ContinuousDelivery]

Yes, code-coverage is a poor metric.

[@jocphone]

@@ContinuousDelivery I recently heard a good quotation around this: "When a measure becomes a target, it ceases to be a good measure" (Marilyn Strathern)

[@ContinuousDelivery]

@@jocphone Thanks Joc, I like that quote - coverage targets are a pretty obvious, but wrong, approach to getting teams to adopt TDD. Nearly everyone makes this mistake, as far as I can see.

[@brianwest7344]

yes everybody knows lines of code is the only metric you need ;)

[@riwjin]

First thing first, that t-shirt is very cool and I really want one now. Yes, priorities.

[@llothar68]

I'm not a huge fan of TDD and writing tests first. I do write test first but they usually don't evaluate any correctness. They are just a way to drive lost of test data through the system so that many codepaths are run. Then i use DesignByContract (yes one decade in the past i was a heavy Eiffel user) and the code itself contains the test in forms of assert statements. Tons of asserts. And invariant statements. I find this much more efficient and helps to write more quality code.

[@LarryRix]

I have been an Eiffel developer for about 20 years. Like you, I am not a fan of TDD alone. However, DbC driven by TDD is a must. Most of my development will involve writing a test that exercises the features of the classes I am writing. What I do not do is the "red-green" test cycle (e.g. write a test first and then code to make it pass). The more DbC code I write, the less test code is required. In some cases, I have found that the only code I need is TDD code that creates an instance of my object and then calls the feature I want to exercise. The DbC code then handles all of the "testing" required to ensure my production code not only works but is robust in the sense of abnormal data inputs.

[@eloniusz]

This rule about not breaking encapsulation makes me wonder. I think I get the point. The methods or object should be testable without seeing its private members. You can assert that it returns correct values without digging in its internals. However, I can think of situations where I would like to test private members separately. E.g. I have a class that contains an array of private methods that calls each other. Each method perform some simple step to achieve much more complicated goal at the end. In this case I would want to also have tests for some of those partial results not only for the public members. Testing the whole object may tell me that something has broken inside but I'd still have a lot of work ascertaining which exact method has stopped working.

[@anagai]

You add spies that can assert values being passed to methods. One way to inspect internal values. Works if your code is broken up into single purpose methods and not monolithic spehgetti monster.

[@Zeioth]

Thank you for sharing. Your experience is very valuable. Specially for us entrepreneurs who not always worked for big organizations before.

[@ContinuousDelivery]

You are welcome, thanks.

[@LarryRix]

From your other videos, I know that you are well aware of Eiffel and perhaps even Design-by-Contract. I wonder if you would speak to TDD in the context of Design-by-Contract-OR-better yet-do a video on TDD + DbC. It would be interesting to get your take on the good, bad, and ugly of TDD + DbC.

[@TokyoXtreme]

Still laughing at that bell at the very end. I feel like it’s a fundamental aspect of your videos, and I would be saddened if it were ever removed.

[@ContinuousDelivery]

😎

[@ironmagma]

Could the volume just be decreased? Listening with headphones I feel like I go deaf each time.

[@pilotboba]

@@ironmagma I prefer loud and clear youtube videos. I can always turn the volume DOWN. It's very hard to turn it louder.

[@BryonLape]

Not organizing code by use case tends to make it necessary for everything to be public, which in turn allows the test swamp to be created.

[@alexandrerisi]

Hi Dave, I'm a fan of your videos, how you explain things and I absolutely agree with you on every point that you made but I don't believe TDD only brings good things. TDD is a very valid technique and I think every developer should at least know it but let me point a few things out that most people do not consider when they talk about TDD. 1 - Test is a code designed to test another code. We cannot expect that by writing a test we will automatically make the production code better. If tests are going to help, the first requirement is to be good at writing code. This may sound obvious but my point is that not every developer writes good code and in a situation like that we would be better without a broken or useless test. 2 - Your test will be as good as your requirements. Obvious again? Yea but when was the last time we saw solid requirements from the beginning? Requirements tend to be fluid (at least in the first stages) making most of our tests useless every time they change. 3 - Tests can make future modifications very expensive. How many times did we add a feature to a software in a couple of days and spent the rest of the week fixing tests? I know that is not supposed to happen but it will. It could be because the managers don't give enough time to deal with the tests properly or because a new developer was just hired and he didn't fully understand what the test was trying to do. 4 - Tests don't always affect the production code in a good way. It is very common to see developers separating the business logic from the provided apis. That makes the code very easy to test but it doesn't necessarily improve the design of the code positively. 5 - Around 80-90% of our systems will be automatically tested if we use frameworks. Hibernate? Spring? Vaadin? We don't write as much logic as we think we do. If we are using Spring Data/Hibernate/MongoDB we are not even writing our own queries most of the time. We don't produce endpoints in our RESTFUL webservice when we use Spring. When we do this, we are using someone else's code and this code is already tested. 6 - Extensive testing is expensive! It takes a lot of time from developers, time that could be used on the production code. Creating a toy application using TDD where we can easily spec all the requirements from the beginning and don't have to support code from other developers is very different than having to design something in an enterprise environment whilst the requirements are constantly changing, I'm sure you know that. I hope you don't think I'm trying to discourage the use of TDD, that is not my point, but it is clear to me after more than a decade as a software developer that TDD is not a silver bullet. It is a fine technique that may help developers in some projects but it is not what will ultimately define "good" or "bad" code. That usually lies on the skills of the developer despite the number of tests, we all have seen great code with no tests and bad code with lots of tests. Writing good tests is an art and it is not easy. Personally I don't care what technique was used to produce the code as long as it is well designed. My opinion is that every developer should try TDD, but they also should have the choice to use it or not. What REALLY matters at the end is the production code, not the tests.

[@lucysluckyday]

Agree, but you need to have a really good BA to ensure the user acceptance criteria are clearly identified and fleshed out, plus bring in a really good tester for the 3 amigos reviews to identify whatever else should be included or categorized as part of the integration and smoke testing requirements.

[@froobly]

Having played with a whole bunch of different strategies for test automation, I've come to the conclusion that there are certain tradeoffs that are necessary, and you cannot have a test suite that is a) performant, b) clear in purpose, and c) robust under refactoring. I used to be an acolyte of the "test at the class boundary" school of testing, and it was really convenient in that each test case ran in a millisecond, and because the unit under test was small, the test cases practically wrote themselves. Chasing code coverage was a breeze. And then when the time came to refactor the object contracts, we ended up having to rewrite all of those tests. And then I watched Ian Cooper's video, "TDD, Where Did It All Go Wrong," and told myself, "yes, this is what I've been doing wrong!" and I changed my tests to target the "unit of isolation," rather the unit of implementation, i.e. test the whole behavior end-to-end instead of just your one class. It was great, because I could do big refactors without the tests breaking (unless they were supposed to). But then something else happened -- the test started taking longer and longer to run. It turns out that when you tell the Angular compiler to build an entire page on a single test case, your tests go from 1ms to maybe half a second, and a test suite with 1000 test cases takes several minutes to run. And you start having to get economical with the number of test setups. So you start breaking that rule of "single purpose," bundling a whole bunch of checks into a single test case, because it'll save 10 seconds per build, per developer, and that adds up. And now we're at The Giant, as you call it, with no indication of ever going back. I would love to hear one day that there's an approach that actually gets me all three, but I'm skeptical that this no-compromises solution exists.

[@ContinuousDelivery]

I have worked on teams that have achieved all three, but it is a combination of techniques. You need the low-level TDD and higher-level fuctional tests. When we built our financial exchange at LMAX the whole enterpeise system as built tested and deployed together. Our Deployment Pipeline ran about 35k unit tests and about 20k Acceptance tests, plus perfrormance, security, data-migration and other tests. It did all this in under 57 minutes. We were in production for over 13 months before the first defect was noticed by a user. So I think that you absoultely can have it all, but not in one kind of test.

[@froobly]

@@ContinuousDelivery the tiered approach was what my team was going for when I described "class boundary" unit tests. The idea would be you would go for the nitty gritty details, 90%+ code coverage, etc. in the unit tests, and then have heavyweight functional integration tests that only verified happy-path navigation, i.e. that it was all wired up properly. The problem was that when it came time to refactor, those unit tests, rather than easing the process of refactoring, added friction to the refactoring process. They would break, not because they had found a bug, but because the class's object contract had changed. Sticklers like myself would update the tests, while more time-crunched developers would simply xit()/xdescribe() out the breaking tests, hoping to come back to it later. Requirements A and B are easily achieved using the orthodox test practices as you've described. C is the sticking point that I really want to hear more about. It's possible that my issue is really specific to the platform I'm on -- I imagine on a Java server, for example, you can new up the entire code path, from front-end request to back-end database query, in a millisecond, and the likelihood of a refactor affecting anything outside of those boundaries ends up being pretty slim. But on Angular or React, the controller/template boundary is the most commonly refactored interaction, and it's also the most expensive to test.

[@froobly]

Also, I'm both grateful and incredibly impressed that you gave a thoughtful response to a comment on a three-month-old video. I wouldn't have bothered posting if I didn't like what you had to say.

[@pilotboba]

@@ContinuousDelivery What is the Data Migration box I see in your diagram? Is that tests or is that actually running scripts? Seems to me I need to migrate the data in my test system in order to run acceptance and other tests on it after that. Do you have video about what happens in that box?

[@ContinuousDelivery]

@@pilotboba Yes, we did multi-layer "data migration testing" and this was just the last stage. We ran unit tests that tested migration scripts as part of our commit stage. Data migration ran automatically on deployment to any environment, so were always run for acceptance tests, manual tests and some performance tests. The problem is that most of these envs didn't have much data. So the last stage in the pipeline the "Date Migration Stage" was really a kind of "data migration performance test". We used an anonymised copy of production data and migrated it for each release candidate that transited the pipeline. Mostly we were interested in migrations that were too slow with production data. We also ran a few "sanity checks", but to be honest I don't remember them ever failing, so they didn't add a lot of value.

[@simontutek5945]

Hello Dave. Thank you for another useful video. I believe many coders do not understand test-first strategy. When people are being taught TDD most examples are small and academic and focus only on teaching the techniques but do not explain the value (the why). My personal opinion is that TDD/BDD is not as popular as it should be because many are missing knowledge that "product development steps have to be designed top-down backtracking value and asserting it on every step of the work breakdown". As long as coders/developers are micromanaged and given individual tasks all of BDD focus is lost and all of TDD benefits will not materialize. I am also heretic - my testing code stops at integration layers - I do not want to write unit tests for individual methods and classes as it is too fragile approach - I believe that only value of public interfaces/methods of packages or modules or services has to be protected - that means that all valid usage scenarios need to be asserted via tests and having coverage is probably lacking assurance that all of the value is delivered. I have never seen proof that more granular value assurance has any meaningful ROI...

[@ContinuousDelivery]

Yes, it is not practiced enough. I do think that there are ways of working that mitigate the fragility that you describe. That is taking a test-first, behaviour-focused approach to low-level testing. That is not fragile in the same way.

[@ianno3]

Love your videos. Really makes so much sense after getting a bunch of real world experience. So much rings true.

[@amadeuscrossing7061]

The Liar, very common, yet often necessary when a code base has been abandoned and the new team has to meet standards before rebuilding from scratch and adding TDD as a primary direction.

[@ContinuousDelivery]

I agree that it is common, but I confess that I see no way in which the 'Liar' is useful. A test with no assertions is doing nothing, but cost time and effort to develop and wastes time to run, all of this while giving a false sense of security. I think that this is a fuction of seeing "testing as a chore" rather than as a driver, a foundation for development. I can see an excuse for some of the other anti-patterms, Giants maybe, but I don't really see how the 'Liar' helps?

[@amadeuscrossing7061]

@@ContinuousDelivery sir, you make a great case. I believe the issue then does not lay with TDD, but instead with the culture around it. I've watched some of your other videos, and I understand your view of efficiency oriented programming. Yet no matter how practical TDD is without its anti-patterns. There is someone out there, who all they are looking for is metrics, and you nor I can remove that shade from their eyes to keep them from ruining TDD for their development team. Beauty it may be, yet the liar is usually how some people are exposed to TDD, no matter how good or bad the experience may be.

[@jasoncole7711]

Good piece, much along the lines of Vladimir Khorikov's excellent "Unit Testing" book which explores these topics in greater detail.

[@supervhs123]

"The Mockery" hits the spot for me. I have seen this pattern on a few occasions. I was working on some bugfix and did refactoring in the code base. I also identified some tests for this area. After analysing the test code, I came to the conclusion that this test provided almost no value. Everything was mocked, and the mocked elements, was all coded to “emulate” the production code. It felt like nothing of the “real” code was tested.

[@ContinuousDelivery]

Yes, first time I saw one of these, I nearly fell on the floor laughing. I'd spent about 30 minutes trying to unpick this test with someone, slowly refactoring things and at some point I realised that there was no code being tested.

[@merrickj.stemen3443]

This was my first viewing of this channel. I am excited to see the rest!

[@valentyn.kostiuk]

I like when people make private methods public to be able to test. And usually if you want to do such things it is telling about bad separation of concerns.

[@SnugglehPuppeh]

There's another factor I think I've found to be at play - engineers simply not being very comfortable with their testing tools. I've been a TDD evangelist everywhere I've worked and almost without exception every engineer's objections to TDD have evaporated as soon as they were given permission to really study how to write good tests and practiced doing so. I think the excuses and combativeness about TDD is usually a front for insecurity about a skill they lack.

[@a314]

Fantastic stuff Dave!!

[@GDScriptDude]

I purchased your book. Very interested in automation and testing. But I am involved in Game Dev where there is a lack of clarity as to what the spec is (evolving product).

[@ContinuousDelivery]

I think that is pretty common whatever the domain that you work in. They key is to create tests that are not too tigfhtly-coupled to the code that they are testing, and designs that compartmentalise the problem so that you can evolve the design over time. Check-out this weeks video for some more ideas on that, coming on Wednesday.

[@GDScriptDude]

@@ContinuousDelivery Thanks, looking forward to it

[@sanderdejong66]

Another of those videos that you would like to “like” more than once.

[@ContinuousDelivery]

😂

[@FURRYDUCKIE]

You had me at SKYNET! Yet another awesome vid thank you for your great insight.

[@ContinuousDelivery]

I have recently splashed out on some more T-shirts, so stay-tuned 🤣

[@metalmolisher666]

And now for those who work with microcontrollers and external hardware like stepper Motors without feeback? What should I do?

[@Kaizzer]

Bringing in hardware creates another beast of TDD! If you were to automate it, you'd need DAQ along with software. Despite being feasible, sometimes it would require so many resources (time and equipment) that it's more commonly done with manual inspection.

[@JuanVasquezq]

Answering your first question in this video, well sometimes the owner of the product does not want to pay for quality software... and yet they complain

[@a0flj0]

That's IME a fallacy. When doing TDD, you actually develop faster, and in the end cheaper. Bad quality software becomes more expensive than the good quality equivalent very early on, before you even have a minimum viable product, for non-critical projects.

[@andreid3979]

I like this, just one observation. Many of things are true and fixable whether you use TDD or not. Being able to write testable code is down to not knowing how to write useful tests to begin with. Too much setup and mocking is a problem that can be seen as solved with or without TDD. At the end of the day, this is useful for anyone who doesn't know how to write useful tests

[@ContinuousDelivery]

I don't think that you get to see the complex coupling and setup in a design as clearly in the absence of TDD. It surfaces design problems that I don't find as quickly any other way. Finding them sooner, of course, makes it MUCH easier to fix them. The result is that TDD applies a pressure to improve design that is otherwise absent. It is not the only way to get good design, but it helps us to do a better job of design. I can't think of much else that does so to the same extent.

[@giorgenesgelatti3987]

Pure gold. Thanks for this

[@d3stinYwOw]

About "The Giant", when test team and development team is two different teams, it happens all the time and it does not harm as much, when you test against requirements, please correct me if I'm wrong :)

[@MarioVapenik]

TDD is an illusion of quality. Quality is to make system perfect. What means use asserts in the system itself and do it the "right way". In unit tests asserts are just interesting. But never effective enough.

[@CodyEngelCodes]

Talking about TDD starts around 4:00. Also, great video!

[@ContinuousDelivery]

Thanks

[@khongthefork]

Wow, just found your channel and it's really been amazing!

[@ContinuousDelivery]

Glad you enjoy it!

[@Djdavidnyan]

Extremely insightful, thanks!

[@bachristus]

I watch Dave's videos partly to see another brilliant T-shirt

[@grimfistgaming7694]

Exactly my thoughts :D If I see a guy with a "nerd" shirt, I immediately know he is cool!

[@richardgeddes630]

TDD would be more ubiquitous if it was used to teach coding. As a separate track which can be optional, TDD can be perceived as "getting in the way".

[@ContinuousDelivery]

Yes, I agree, I think that anyone learning to code should start with TDD. I recently taught a friend their first steps in coding, and did exactly this. We started by writing a test to say what we wanted our code to do, add a couple of numbers, and then moved on from there. I taught him the basic principles of coding, functions, variables, loops, conditionals all from simple tests, that I think he, and I, made it easier for him to learn and from my perspective, he was also learning some valuable lessons subliminally. Don't know if it will work, but it worked very well in the context of this first step.

[@arandompersononearth]

Thank you for this video, and for your advocacy of test-first TDD! I would like to respond to the antipattern of the The Giant. At my work, there's a legitimate use case for writing such tests to cover the top-level expected behavior. We have a service that has to process multiple messages of type A, B, C in some particular orders, such as A, or AC, or ABB. Some of these common permutations deserve a "The Giant"-style test, to express the behavior expected for that use case. I am not saying that we write these "The Giant"-style tests for each permutation of _all_ of the underlying behavior - for that there are smaller tests of the underlying module(s).

[@ContinuousDelivery]

I think that there is a place for broader, scenario based testing, but I prefer to do that kind of thing as acceptance tests rather than the more fine-grained, lower-level TDD that this video is mostly describing. Acceptance tests are certainly part of my TDD approach too, but, intentionally, have a broader focus and work alongside the fine-grained stuff.

[@arandompersononearth]

@@ContinuousDelivery thank you for clarifying!

[@shanefeather-lopez5935]

A very useful tool for selling TDD to dev teams - thanks :)

[@ContinuousDelivery]

Glad it was helpful!

[@gmaglio]

That Skynet tshirt is epic dope!

[@iorch82]

This is lovely advice to write another fizzbuzz or calculator. Most of these tips simply dont fit well with complex systems with lots of procedural code - which is your average Java app -

[@ContinuousDelivery]

Well, my team built one of the highest performance financial exchanges in the world, trading financial derivatives, a notoriously complex business domain. Tesla make cars and robots this way, SpaceX make space rockets, and NASA built their first man-carrying space rocket this way. So no! You are simply wrong on this I am afraid. Just because you take your first steps in learning to drive in a safe place with no other cars, doesn't mean that cars don't work as a mode of transport!

[@TomHultonHarrop]

Excellent video! I'm definitely a fan of TDD but have found it very difficult in certain contexts. One particular example is writing a manipulator/gizmo system for an editor. There are a number of complex factors that I found made writing the tests first difficult. The first is that all interactions happen as mouse events and things like position in the 3D world and distance from the camera cause the same 2D cursor delta to produce a different result. We really wanted to test this (we didn't have any tests before 😬) and wound up having to write a test framework API for manipulators doing projections from screen space to world space and back. In the end we got some decent coverage but it was a big investment. I'm curious what your advice would be for TDD when it comes to these types of interactive applications (editors, games etc...). Also keep up the great videos! 😄 I loved the one on (not!) branching 🙂

[@ContinuousDelivery]

I think that testing any software is difficult at the points where the software 'touches' the world. When you think of it there is an obvious reason, that these points it is difficult to put something that can act as a measurememt point. We have to fake the inputs through a UI or collect the display off a screen or capture the data written to a disk or some other piece of hardware. So the first part of the plan is to minimise this stuff. Organise the design so that the points that 'touch' the outside world are as simple as possible. If you are drawing a complex display, then make the actual pixel-painting as straightforward and as generic as you can, and test the underlying behaviour. If you are capturing mouse events, what do these really mean, translate into what they mean as quickly and simply as you can, and then test the meaning rather than the events. That allows us to test the vast majority of the behaviour of the system in isolation from 'the real world'. Then we need to focus the testing of the parts of the code that do touch the world, the UI rendering and mouse event handlers to check that they translate ideas sensibly and accurately. I may try and do a video on this, though it is a complex topic to demonstrate in 15 minutes :)

[@TomHultonHarrop]

@@ContinuousDelivery Thanks for the reply! 🙂 Doing a video with something like this (or a game of some sort perhaps?) using TDD would be really interesting. Lots of people get immediately turned off the idea of TDD (and testing in general) as they assume it only works with certain types of programs (when I believe it could be used for anything!) You just have to be equipped with the knowledge to take the right approach. I test drove a Game of Life implementation as a learning exercise and got some really interesting results. Same for a handle based container, but something like a camera system or arcade game would be great to demonstrate how the techniques can translate to these settings. Thanks again!

[@SpeedfreakUK]

@@ContinuousDelivery you could take a leaf from video games and write a simple system that records inputs. You can then get a human to define the test once using actual human inputs, record them and use them as input for the test. This is basically how games do things like replays.

[@abogdzie]

My favorite style is Giant 🤪 However I find it really impractical to repeat some code only to keep one assertion per unit test. More over I like table driven tests with loops inside of single unit test just to make many tests for the same code.

[@a0flj0]

You have to keep in mind a thing: one assertion per test doesn't mean one assertion statement, but one logical assertion. One logical assertion might be expressed at a higher abstraction level than what your assertion library allows you to express in a single statement.

[@ContinuousDelivery]

I recently came across a good way of describing this from Roy Osherove, he says "on assertion per effect". Write test to determine "effects" and assert on the desirable "effect" - I quite like that description.

[@berkes]

@@a0flj0 When that arrives, I introduce new assertions in my DSL. So, rather than asserting user.email === String, user.invitation.state == ACCEPTED, user.roles includes 'reviewer'. I have a DSL that makes one assertion: reviewer_accepted_invitation(). Which then probably makes those three assertions. That does not mean you're never allowed multiple assertions: often they are fine, if they are logically grouped. But it does allow for much more expressive tests. And it moves implementation details into a sortof adapter: the DSL.

[@a0flj0]

@@berkes That kind of grouping and hiding away sometimes makes sense - I use it too, sometimes. Other times, however, evening you repeatedly perform the same assertions, it doesn't. I would never put the three assertions you gave as an example into a private method and call that method from a test. The three assertions are logically not directly related. Grouping them in a single function makes tests more difficult to read (or at least difficult to read correctly). Tests are not just tests, they are living documentation. By moving three unrelated assertions to a single method, whose name reflects just one of them, you hide important information from test readers.

[@thygrrr]

How do you deal with overcoverage, especially if the test is not redundant? It means that some earlier code may, if changed, require hundreds of tests to be changed and re-examined for a single api change. This is most serious in the case or updating to a v 2.0

[@xiretsa9166]

tests make no sense until the solution to the problem and the at least rough design is clear and understandable to the programmer. I program according to my alpha - beta - done - method: I have many tested modules close at hand in my private library alpha: I play around with the rough edges of the problem, the data and methods of which are not yet clear to me, until I no longer have any questions regarding the content. So if I start a program that doesn't behave as expected and I spontaneously have no idea what the reason might be, then I don't need a computer, but rather a piece of paper and a pen. beta: this is the first time I test correctly because this is the first time that I can ask meaningful questions about the code, which it then answers correctly or incorrectly. Trivial errors caused by copy and paste, ... I take it out here, these should already have been recognized in alpha. Of course, this can throw me back in alpha again and again in individual modules, but I don't know any author who claims to have written a book in one go without correcting the course of action, descriptions, etc. In short: whether and how I generate a test of my code depends solely on my understanding of the task. And, of course, what I expect from me to do a good job and not waste time creating new problems

[@TheJessejunior]

Thabks for sharing your experience... This clarified a lot of doubts for me

[@ContinuousDelivery]

Thanks, I am pleased if it helped.

[@emonymph6911]

What are some good sources: books, courses etc on how to actually write tests before writing your code for devops? Thanks

[@ContinuousDelivery]

I have a free tutorial on my training site bit.ly/DFTraining and I also recommend Cyber Dojo - which is a n on-line tool for learning TDD, set up as a charity, so you can make a donation to support kids getting into computers.

[@MadsterV]

If the code is difficult to test, it means it's design is .... difficult to test. When you switch testing frameworks, sometimes one framework will be a better fit for your design than another. Does this mean that you design suddenly improved by removing one testing framework and replacing it with another? You design for what you want to obtain. If you're doing test driven, you'll want it to be easy to test with the framework you've already picked, so the testing framework will dictate how your code is designed. When you have a hammer, everything looks like a nail. Testing should be in service of your project, not the other way around! Remember every library you pick is already a choice you're making and it will change what the project will look like.

[@jasonzuvela]

Stick a watt meter on the power cable and test for efficiency!

[@VitalijMik]

5:40 - i wonder how can you write a Test without assertion? In PHPUnit for example you will get a warning and this test will not cover anything, thus will not increase any CodeCoverage etc.

[@ContinuousDelivery]

Not all coverage tools look at assertions.

[@VitalijMik]

@@ContinuousDelivery didnt knew that, thanks

[@miracl]

I wanted to believe in TDD, but I was still hesistant. After all these examples, that I actually see in my own tests... well, now I believe 😁

[@jonathanhill7829]

This is a top video, really useful. I have one question please - what is meant by encapsulation in "The inspector" ? I've seen some definitions that basically say bundling of data with the methods that act on that data. Therefore "violating it" means creating methods that only exist for the purpose of testing? Am in the right ball park? Also does encapsulation only refer to OOP, as most searches I've found on the term seems to suggest this. Thanks!

[@ContinuousDelivery]

What I mean in this context is making something accessible only because a test needs it. You are right that the common OO definition of 'Encapsulation' is that behaviour and state are bundled together. So that is one take, if you access the internal state of an object or module, from a test then that is a bad idea. The other way that I mean it is probably a bit more subtle. If you modify the design of your code, to add a method that is only ever intended to be used in the context of a test or that is only there to give you a back-door into the otherwise internal workings of the code, then that is bad too. Where that second one gets tricky, is what is the difference between that, and designing your code to be 'testable' wich is something that I recommend. My approach to testing is to try to create tests as "mini-specifications" of the behaviour that I want from the code. If I do this right, then I can be precises, specific, in my specification of what I want. without knowing, or assuming, how the code achieves that. That means that I want access to the results in a way that makes sense at the level of the behaviour that I am looking for, but not at the level of the internal workings. This seems like a fairly clear distinction to me, but I confess that I can see that it may be confusing?

[@jonathanhill7829]

@@ContinuousDelivery Hi, thanks so much for the response and no, actually that is a very clear and detailed explanation which totally answers my question so thanks for that. I've screen shotted your answer as I think i'll be referring back to it a lot, really like the way of putting tests as "mini-specifications of the behaviour you want from the code" - thats really helpful. Onwards into the world of TDD for me :-)

[@DavidAtTokyo]

​@@ContinuousDelivery The Inspector one was the most interesting for me. I struggled with the distinction between "encapsulation" and "testability", between which there seems to be a tension (but more likely I've missed something, and might benefit by watching that part of the video again). For example (and to get niche), the way to interact with actors in the actor model is to send and receive messages to/from them. That's the only way to interact with them. Yet I encounter the desire to test that Actors will react in a certain way in response to a message received, and it seems that adding a message to the actor's protocol specifically to support that test is a common approach. Yet this is akin to the addition of a method on a class to support a test, as mentioned in the video. (I toyed with making those parts of the actor's message protocol private, which worked, but the whole thing still leaves a bad taste.) I guess an alternative would be to write tests on the logic that is to be internal to the actor, and then only expose the required behavior in Actor messages, but it feels like this would result in a tight coupling to the actor's implementation.

[@DavidAtTokyo]

​ @Continuous Delivery Having suspected that I had missed something, I found there was a "Read more" to your comment and it covers exactly what I was getting at.

[@stamatispsarras]

I have so many questions, even though I have practiced TDD on several projects. What do you do on mathematical calculations? I usually test a few known inputs and outputs but that seems to follow two of the anti-patterns you mentioned. The Liar and the Giant. I am just doing a test for the sake of the test and I am also having multiple assertions. FYI the calculation is returning a few values so I need to test all of them that they give the right result. I am also testing some extremes, i.e. x cannot be zero and throws an exception. Should I test the behaviour? i.e. when x is rising y is reducing? That seems too vague for a unit test and if I were to be super specific, I am just replicating the actual code...oof...

[@ContinuousDelivery]

It depends on the nature of the calculation I suppose. There is no value to testing that maths works, or that my programming language can do maths successfully. So if this is a complex calculation, then I want enough tests to confirm that I wrote down the calculation correctly, but no more than that. I'd always start with the behaviour, so what my tests are really doing is defining how I supply parameters, and how I get results, more than anything else. When I teach TDD I use Add Fractions as an exercise. It is nearly all about collecting and confirming how different results are reported - were the fractions rationalised correctly for increasingly tricky sums (e.g. 11/14 + 33/14 = 3 1/7 ) rather than how to calculate a LCD, though you need to do that to get to a sensible result.

[@foad-esad]

Great video. I will definitely reference your videos during my XP Dojo's. Thanks.

[@ContinuousDelivery]

Glad to help

[@deutscher86]

Great video. Thank you so much! I have a question please: at 9:20, say that I did not think of an edge case while writing my test for the first time, would it still be easy to add an edge case say 1 year later? (for the sake of argument the same exacmple you gave)? I am still new to TDD and I am asking myself how can I not complicate my test if I want to add another case to it after a while if I forgot it? What if a piece of code which already has a test changed later and thus my test need to change? for example, that method you showed at 18:58 .. suppose that method was really in the code base ( was added later ) .. shouldn't my test case adapt to these changes or should i "refactor" the whole test case from the gound up? I dunno it might be a stupid question but I think you also need to be very good in abstraction to think from the very first beginning how a function, for example, should behave right from the start, no?

[@ContinuousDelivery]

It should be ok to learn new things that require you to change either code or test. I think that a lot of the value of this approach is that it gives us more freedom to learn new things and adapt to them, or to make mistakes and recover from them. I think there are two scenarios in your question. You think of a behaviour that exists in your code, but wasn't tested and you think of a new behaviour that you'd like to add. You should be free to add new tests, and/or new code for each case. I'd write the test first in both cases, in the first case your test will pass, because the code is already there and working. Taking a TDD approach, I would, temporarily, break the code in a way that will make my test fail, to check that my test is really testing what I think it is. Once I know that, I fix my code again and check that everything still passes. In the second case, there is no need, write the test for the new behaviour, see the test fail, add the new behaviour to the code, and check that ALL the tests still pass. If your change breaks old behaviours (tests) then you need to think carefully what that means. If the change changes what the code is really doing, then the old tests may have been made invalid by the change, so you will need to change/replace them so that they match your new understanding. If the tests are still correct, but you broke them with your change, your change is wrong!

[@deutscher86]

@@ContinuousDelivery I understand now. It is not really that difficult as I thought it would be. As most of the guys here said, I've always wanted to start with a TDD approach but never had a chance (at an actual job) to do so. I am gonna start looking for a new job where one of the requirements is "You value TDD". Thank you so much.

[@pilotboba]

@@deutscher86 Lead by example. When you are doing your work, start with tests. Is there really someone looking over your shoulder that will slap your hands if you do that. When you code starts being more reliable and correct, others will start emulating you.

[@MaxAmadeusW]

Great video! I was pondering on some issues I've had with 'table driven tests' most notably used in the Go source code, and generally percieved as best practice for Go. What are yours or anyone else watching this opinions about them? For me, they seem more suited for smaller methods/functions which doesn't require mocking repositories or database interfaces, and for more complicated codebases they tend to produce really hard to read and understand tests. I've so far opposed this practice, but I have had no choice but to adopt it as well as that is the convention in my current project, and I've struggled to convince our team otherwise. Perhaps anyone shares my opinion, or perhaps this is an issue with the codebase itself (e.g. complex input/output for systems). I prefer writing one test per test-case rather than one test with an array of input/expected output parameters

[@ContinuousDelivery]

As a TDD person I think that this kind of approach misses an important point. I think that one of the key values of TDD is the way it thinks about how to make the interaction with your code a nicer experience. By requiring me to write a test, and do that in a variety of different circumstances to explore the behaviour that I want, it gives me live, visceral feedback on what the public interface to my code is like to use. If I don't like it, then I can take notice of that signal, and re-design it. Things like parameter-based and table-based testing are focus on, what seems to me, an 'old-school' view of testing, it is what you do after the code is written to confirm that you were a genius when you wrote it, so since you are a genius, you don't want to spend too much time proving the obvious! They can be useful for a narrow set of circumstances, but for what I see they are more prone to being used in dumb ways. I am not very interested in all of the values of variables that my code takes, this is testing by sampling. Instead I will grow my code via a test, if I do that I will be presented by challenges in code that I need to write, so I will have to be more inventive, and more precise about the test case that I need. I think that this works better!

[@seancooper5140]

How would you apply TDD to scientific computing, where the inputs are complex datasets, not strings, and the code is performing non-trivial computations that are dependent on the characteristics of the data?

[@ContinuousDelivery]

If it is is digital algorithm, it is deterministic, same inputs, same outputs. I have been involved with a few scientific systems. TDD works fine, it is very similar to high-performance finance in terms of complexity of systems and inputs, where I have a lot of experience. It is a mistake to assume that TDD only works for simple code, it shines most brightly in more complex cases.

[@seancooper5140]

@@ContinuousDelivery Thanks for the quick response. In principle, I agree with you. Perhaps it's just my lack of software dev experience ( I have ended up the primary developer as a side responsibility, because there are no dedicated software/CS resources). Any pointers toward applying this in practice (perhaps in another video?) would be awesome.

[@rastislavsvoboda4363]

hello at 20:02, can you elaborate this idea on some concrete example, pls...

[@gonzalowaszczuk638]

What is your opinion on handling the same "case" but with different data or cases? For example, imagine I have a system under test that must handle an input date and do different things based on a specific period and some other flags/inputs. Say I want to test the cases where the input date happens before the period; happens exactly at the start of the period; happens in the middle of the period; happens right at the end of the period; and happens after the period. Imagine I would also like to test different dates for each of those cases too, imagine 2 dates per case. In the above case, would you write a single test that had 10 inputs; 5 tests that have 2 inputs each; or 10 different tests? Based on the above response, what is the correct way to manage the huge increase in number of tests, both in terms of code organization (e.g lines of code, files, etc); naming (what do you do if naming becomes hard when you have so many tests?); and other possible issues that arise?

[@shardator]

These are all extremely important advices. For a startup. However pls note that _most_ work is done on existing production code in the industry. This code is 99% legacy code. Written waaaay before TDD was even a thought. It would be good to teach how to refactor legacy code to be testable EFFICIENTLY, without having management getting a stroke. Unless you can provide some replacement.

[@ContinuousDelivery]

Check out my videos on that topic: courses.cd.training/courses/refactoring-legacy-code-video-tutorial

[@shardator]

@@ContinuousDelivery Thank you! Will do!

[@Ownermode]

I'm trying to find the blog post written by James Carr, but his blog now points to medium.com (blog.james-carr.org/). His medium.com page doesn't have a post about anti-patterns. I found the archived version here for anyone that wants to read it: web.archive.org/web/20100105084725/blog.james-carr.org/2006/11/03/tdd-anti-patterns/

[@evans8245]

tdd-first // create a variable to hold the value: 0 // test test("it should hold 0", () => { // variable let value = 0 assert.equal( value , 0 ) }) // implementation let value = 0;

[@xinaesthetic]

Reminds me of when I was looking over NASA’s WebWorldWind a few years ago. They clearly had a policy of not using external libraries, but also that testing was a good idea… so it had half a dozen dependencies all to do with testing, and a single test that made a Vector3 and then tested that the values were the same as when they were created… var v = new Vector3(1, 2, 3); assert.equal(v.x, 1):

[@evans8245]

@@xinaesthetic how useful is tdd on a day to day basis if you are involved in web development

[@xinaesthetic]

@@evans8245 I must admit I'm not actively using it, it may have saved me some headaches with a project I'm working on at the moment. Just passed a deadline and contemplating reviewing things like that.

[@evans8245]

@@xinaesthetic It's one of those things where you sprinkle on top of the project to brag about it. I've seen it used in cases where it wasn't necessary. In the real world..you don't have to write tests for everything especially when you have an upcoming deadline

[@xinaesthetic]

@@evans8245 I have in the past used it for specific parts of a codebase (not web development, procedural geometry) and could definitely see benefits. With what I was doing recently, having tests in place might've encouraged me to be a bit clearer about my architecture. Trying to add tests during crunch was definitely not on the cards.

[@PowzLP]

I feel like a lot of programmers get that the design is bad and the tests are difficult to write because of that. But what if the bad design is just much too large to refactor along with doing your actual task and writing tests for it? How do you tackle those kinds of problems?

[@Immudzen]

The problem I have run into is how do I test code that depends on large math data structures without having a fair bit of setup? For instance I was doing work on a solver and it takes in a 4D array that is several GB in size. While I can make it much smaller that also ends up not testing the solver correctly due to the error it introduces. Saving the data structure to disk can also be a problem in terms of then trying to keep it with the code and not having the checkouts be too large. What I mostly end up with is a complex setup for some of these kinds of tests and also separate tests for every single piece I can test independently.

[@ContinuousDelivery]

It depends on what you are testing for. How does the variance in the data change the processing of the algorithm? It is the code that you want to test. You don't do that very effectively by throwing random data at it. So what is the minimal data set, that allows you to explore how the code works? Next question is how do you minimise the work to change the data that must be there to exercise all of the code? One of my approaches to this second problem, is to establish a sensible, minimal, but accurate data set, and then the tools that allow me to override default values in key parameters. that will allow me to exercise the system I am testing. All my TDD testing is based on synthetic data, I don't want "real data" because it is too big and unwieldy. Driving the design of your code from tests, helps a lot to focus on the minimum data that you need to explore the code.

[@lucasbittencourt8290]

Very valuable content!

[@ContinuousDelivery]

Thanks

[@mattpotter8725]

Isn't TDD built in doing Unit Testing and thus only one thing can be asserted? Isn't a great that tests many things more of an integration test? Developers write unit tests before they write any code and see if fall testers write integration tests for scenarios they come up with whilst trying to break the code. Maybe some devs can write integration tests, but isn't it better to have someone other than those who wrote the code trying to test how it all works from start to end?

[@malikrumi1206]

Hi. Just discovered your channel. This is the second one of your videos I've seen. Question: Most of what I do is text wrangling. The string(s) I am finding, deleting, replacing or changing tend to be unique to each document, and there may hundreds or thousands of documents per job. I have looked at a lot of testing tutorials but have not found one that shows how to test in this environment. Suggestions? Thanks!

[@ducodarling]

So, if I write a program to add two numbers, attempt to write tests around that addition program, but have trouble doing so... the reason for my trouble is that I wrote my addition program to add, and not to be tested. Is running the program not a test in itself? Can you imagine if Ford built a mold for every individual car, before it was assembled?

[@GalaxyCat001]

"Is running the program not a test in itself?" Yes, but thats what the test program does, it runs the target program, looking for (ie testing for) a specific thing or set of things. So with a system comprising many programs, the test 'system' can run all those 'target' programs in a fraction of the time it would take a person to run the system through all its possible uses. Also, in many cases a person would not know all the things a system can do but they don't need to because the test 'system' runs through everything for them. In real life a 'system' can consist of dozens, and often hundreds of programs. Even though a programmer might make a simple change to one program that they can easily test by running the program, being able to have a test system that can run through the entire 'target' system in a few minutes gives that extra level of assurance that no unexpected behavior occurs somewhere else in the system because of the simple change made (which happens more often than you'd think). In the case of Ford or car manufacturers, thats why they use a lot of digital design tools these days, so they can 'virtually' make a mold of an entire car for many 'simple' mechanical or electrical changes.

[@henrywebster9318]

Great video! Quick question for 11:38 , if I have an object returned from a method, and it has 3 or 4 publicly accessible members, you're saying even though I'm testing the same function with the same input they would be different unit tests per member? I've never thought about it that way but I will try it out. Also, would you say using something like assertj's assertAll takes care of the issue?

[@ContinuousDelivery]

I am not entirely sure that I understand your question, so let me unpack it a bit. If you are testing a function that returns an object, you are interested in the behaviour of the function, not the object. Figure out what you care about that, and test for that. If you are testing some object then, practically, you will be testing separate methods separately, if the code is designed well, but that is not the aim or focus of your tests. Instead aim to ALWAYS test the behviour of your code. That should guide your testing, and your design. Let's try and combine these ideas. Let's imagine some code that returns two different types of address, a regular address, and a business address (maybe a stupid idea, but I am on my first coffee this morning!). If I am testing the code that returns the address, then what I am interested in is how it decides which to return, and then did it return the right one? So we could imagine writing a test like this: shouldReturnBizAddressForOrg() { Party org = new Org(new BizAddress()); addr = myService.getAddress(org); assertTrue(org instanceof BizAddress.class); } and another like this: shouldReturnAddressForPerson() { Party person = new Person(new Address()); addr = myService.getAddress(person); assertTrue(org instanceof Address.class); } So I am focused on what I want the code to do, saying nothing about how it does it, and having one reason of failure per test.

[@KogiSyl]

I don't like tdd because it decreases the flexibility in the initial phases of development where you don't yet know how it all will turn out. You add the overhead of tests to your work but mostly you will have to scrap them along with the code. On the other hand is you start from the code (after design of course, which should be very flexible if you are unsure of the outcome), you just modify it along with what you discover and then start making the tests, you easily refactor and restructure as you go without having to redo the tests.

[@ContinuousDelivery]

That is one of the times when I like it best 😁 As I am playing with ideas I can see how good they are through my tests, the main point of TDD is that it is a design tool, and so using it to design your code is where it has the greatest value. There is a concept of "spike" which is focussed on learning, and doesn't require you to write "production quality code" but it does that by demanding that you throw away any code that you write while working on the pure exploration.

[@markbromell7909]

"Try to write tests that test the desirable behavior of the code, rather than the implementation" No matter what though, when you introduce the use of mocks and stubs, a test will become coupled to the implementation. I don't understand how I can mock the dependencies of a class, and then write an actual 'unit test', without the test coupling implementation. If A depends on B, and I mock B in the tests of A, when I test a part of A that uses B.method(), and that test has a desirable return value from that method, I must configure the return value of B.method() for that test, therefore coupling the test to implementation. Although apparently that is a sign of bad design? How else do make a class that uses a dependency without needing to configure the mocks? A unit under test almost never operates without having calls to outside dependencies. I want to write a unit test with the dependencies mocked, but without needing to configure what the responses are from the calls on the dependencies. Is this possible? I have been told many times to mock all dependencies of a class that is being tested -- although in my mind, as long as there is a concrete implementation of a dependency, I want to use that dependency without mocking it (as long as it doesn't interact with IO, network, or database). The reason for this is because the code I'm testing literally *depends* on the correct functioning of that dependency, so if I don't mock the dependency is that bad? Is it an integration test now, and not a unit test, even if its functionality is self-contained and not interacting with an outside system?

[@tuxino]

In my view (but be aware, that I am not really experienced with TDD), when you test a unit, you should consider it as much as you possibly can as a black-box, where you only think of its inputs and outputs. If it calls other things, they should be assumed to be working properly, because they are separate units and will already be covered by their own tests. In other words: only mock those things that really must be mocked. If it is possible, I would even prefer to have a local database-setup that could be quickly reset to a known state to use for testing things that call databases, such that the only thing that is mocked is the contents of the database.

[@markbromell7909]

@@tuxino I agree with what you're saying, and that is generally how I go about writing my tests. I also don't have much experience in TDD, it was nearly 2 years ago since I was first introduced to unit testing and I've only undertaken 2 serious projects that I wrote automated tests for. So maybe in the future I will understand fully what is being said in this video.

[@LarryRix]

Hello, @@markbromell7909! What you both have described very well stems from a deficiency in TDD, where "tests" are always from the outside in. Ole makes the point of a "black box" with assumptions about the inner workings of that box-that is-one is forced into trusting the correctness of the inside-the-box calls. There is a question that many programmers do not know they can ask. Namely-what if I could bring testing inside the box (e.g. function or feature)? This is the elegant beauty of Design-by-Contract. Precondition "tests" (assertions) define not just the proper state of feature inputs, but of the proper (stable) system state in general. A precondition or requirement simply states: Here are the assertions that must hold true for me to operate without defect. A failure of a precondition points to the calling Client as the one at fault for not ensuring a stable state before making the call to the Supplier feature (method). Post-condition "tests" (assertions) define the correct (defectless) state of a function once it is complete. This might also include stateful assertions about data it has computed for a calling Client routine. A failure of a post-condition indicates a failure in the Supplier method-that is-it failed to ensure its proper exit state at exit-time. Beyond preconditions and post-conditions, there are three other significant types of Contracts: 1. Check assertions-think of your routine (method) as an assembly line responsible for producing a product. Along that assembly line, you can employ Quality Assurance checks. Therefore, a failed check assertion points to code preceding the check itself as failing to meet the correctness rules. So, either the rule is wrong or the preceding code is wrong, or both (in rare instances). 2. Loop Invariants-similar to check assertions, loop invariant provides assertions that define stable state requirements after each iteration of a loop. A failure in a loop invariant means the fault or defect is to be found in the preceding iteration. Following iterations may (in fact) also have defects, but early detection of iteration-defects means if you solve one, you might solve them all! 3. Class Invariants-these are the most powerful Design-by-Contract assertions of all. They define the stable stateof an entire class (object at runtime). The class invariants are checked before and after each access to any feature of a class (property or method). Therefore, if a class invariant fails, then the culprit is either a bad assertion (assumption of state) or the last entity to access the object. Design-by-Contract assertions are built with the context and purpose of each class and method in mind. They are not separate from it, but part-and-parcel with it. So, in using Design-by-Contract, your testing code is brought in from the cold to the warmth of the requirements, architecture, and design of each class and method in its use-case context. Defects detected by contract failures point to either a defect in the contract or immediately identify the offending code (Client or Supplier). TDD is then properly utilized to exercise code with its contextual contract assertions no matter what entry point or "muck" or "stub" one uses.

[@МихаилЧон]

How do you write the mininum amount of application code to make a test pass but at the same time not couple to implementation?

[@EmilNicolaiePerhinschi]

TTD tests should not look like unit testing, simply write code pretending you're using your code and add asserts from place to place to see it does what it is supposed to do start transforming your tdd tests into unit/automated tests when you're fine with the API and you believe you understand the business managers complain tests are expensive and they are correct, if you add all the scaffolding for unit tests and the API changes three times a day it will be indeed expensive

[@FlaviusAspra]

I've seen the mockery so often that I could throw up. The most dangerous kind is done by mid level programmers who think they know everything. Personally, I push for - following principles like layered architecture, throwing exceptions in constructors, always keep the objects valid, respecting tell, don't ask, and the bunch of 20+ other principles or guidelines which are not SOLID (which are good, but just the beginning) - unit testing only the domain layer - the rest are just some lightweight integration tests, sanity checks, smoke tests - the rest of the testing goes to the infrastructure level: staging environment, blue/green, dark launches, feature gates, etc TDD has only one benefit: it keeps an inexperienced team from doing big mistakes. It does not foster great design, just good enough design. Sorry to hurt anyone's feelings, but there is a horrendous number of good-enough teams. Yes, they provide business value. But yes, no great design. 98% of the teams are like that. Signed, CTO

[@ContinuousDelivery]

I sometimes teach TDD to groups, usually as a 3 day course, and I quickly learned that the mechanics of TDD take about 1/2 a day, the rest of the time is really teaching design. I think that this is a big problem, and one that we don't talk about enough in our industry. SW Dev is not just the ability to write code, any more that writing is the ability to run a word-processor. What you write is what matters and that is all about design, and we almost never talk about that stuff. We have religious wars about language and frameworks and tools, none of which matter at all compared to the difference between good and bad design.

[@FlaviusAspra]

@@ContinuousDelivery while writing tests, the biggest first design is naming things, which very often does not even occur to tdd people as neither important nor design. Either way, great content. Prior to watching anything I thought "a grandpa talking about outdated stuff", but you have very up to date content in tact with the current industry.

[@ContinuousDelivery]

@@FlaviusAspra Disagree, what better way to focus on naming things, than trying to write some code (a test) that is using the code that you are naming? It forces you to think carefully about the semantics of your code. Certainly when I teach people TDD, I have a whole section on Naming. Nearly ALL of my TDD training is really training in design because that is where most people struggle.

[@FlaviusAspra]

@@ContinuousDelivery that's exactly what I also meant to say. Naming is a huge design step done while writing tests. But those pesky mid-level programmers who are proud of mocking everything away most often do not even think about naming.

[@danm6189]

@@FlaviusAspra presuming you don't class yourself as a mid level programmer?