Gain CONFIDENCE With Approval Testing

[@shashank.c]

I haven't watched the full version of this Engineering Room episode, but I can see what a gem this conversation was. I first understood what an approval test is from Dave's earlier videos and immediately knew how important it was for a team like mine which have been working with a lot of legacy code.

[@Dehibernation]

Sounds like snapshot testing, commonly used in React development. The downside I've seen working with it is that unexperienced developers can often just approve things blindly just to make the error go away. This can put more load on approvers to also make sure their diff is correct, which can be tough without explicit context about the components and how they work.

[@sarabwt]

It is snapshot testing, just the implementation Dave showed is a worse version of the type Jest has. Devs blindly accepting the change is basically the same as updating the test to become green without asking yourself if that is the result that you want. However, I think snapshot testing for UI is not that great, because as you said, the approver has to know the context in order to be useful. I find it is much better used with validating API responses

[@EmilyBache-tech-coach]

That's why I prefer to call it "Approval testing" to emphasize it's not just a snapshot, it's something you've approved. A "snapshot" implies it's here today, gone tomorrow, update when you feel like. An "approved" record or text file is something you explicitly decide whether it's good enough. I find junior developers just need a little bit of training and they stop blindly approving things in the way you describe. It's always in version control history who it was who approved a wrong result 🙂 The other aspect you're missing in React development is a custom 'printer' as I described. The text should be designed to be readable and possible to make an approval judgement about, if you understand the domain.

[@lkyuvsad]

It’s more correct to say that React snapshot testing is an implementation of approval testing. Approval testing is a broader, and earlier, idea. Both snapshot testing and approval testing are things I tend to use sparingly. But they’re useful in their place. As Bache says, they are helpful when you want to test high level behaviour when there are a lot of outputs to test and they change frequently. Maintaining large, complex assertions is a pain. Blindly approving changes is equivalent to deleting tests when they fail. If devs started deleting tests, I’m not sure we’d say that was a problem with the tests. Why did they feel pressure to act without understanding something? Why is somebody approving or working on something they don’t understand without someone on hand to ask for help? Why didn’t they understand their responsibilities? Pairing seems like it would be helpful for both of the problems you mention- making sure inexperienced engineers understand what is expected of them, and that code reviewers actually understanding what they are reviewing.

[@DmytroPoplavskiy]

It works with the code review when the code change is reviewed together with the approval test result change.

[@JohnMitchellCalif]

I'm a DevOps Engineer with tons of Dev experience. In one of our projects the Devs had a Snapshot test which was failing. I thought: "hey, I can read code, I can read the test, I can fix it!" That did NOT work: I didn't understand the overall *application* well enough, so I didn't understand if I should fix the test, or fix the code. I did the correct thing: let the failing test continue to fail, vs burying a bug to explode later :)

[@manishm9478]

I watched Dave's earlier video on approval testing, but something about this discussion clicked for me :)

[@AttilaButurla]

Interesting. I've worked for a company that would write unit tests for UI frameworks, such as React, where it would store the HTML/content that was generated as a string. The test would then compare any subsequent runs against that exact string. I wasn't a huge fan of that approach because anytime you adjust the UI, the test would fail. Yes, you could just update the tests, but it didn't seem like in this case the test was providing much value, because anytime the test would fail people would just accept the new output. Perhaps the scenario I'm thinking of isn't a good candidate for acceptance tests.

[@dontanton7775]

We also use the term in games QA but in a different sense. It's the approval of the feature and quaility criteria on user level. Interesting to learn that this term comes from coding itself.

[@barneylaurance1865]

AKA "golden master" testing or in the Micheal Feathers terminology "characterisation testing".

[@jangohemmes352]

Golden Master testing is new for me, I like it most

[@woosix7735]

I mean this sounds all well and good, but it sems to me that one limitation are programs where the correct output is not unique

[@researchandbuild1751]

Used to call this gold testing. The result we would compare against works be the gold standard

[@Tony-dp1rl]

It's almost ironic, that Approval Tests seem to be just a higher level test, which we were already doing with test automation and integration testing without Unit Tests. It could almost be used as validation that Unit Tests at the code level are less valuable than advertised. i.e. Testing the results of a process involving several steps IS exactly what higher level tests do, and they do it in a place that is easier and less costly to maintain than at the code level.

But the higher level tests won't capture all important details easily (too many inputs too indirectly related to the outputs).

[@EmilyBache-tech-coach]

The advantage of the lower level unit tests is you potentially get faster, more fine grained feedback on your design.

[@barneylaurance1865]

@@EmilyBache-tech-coach Yep, also the unit tests check that the individual units do what you think they do. If you're going to re-use a unit later when you're building a new or improved feature, you want to know that you understand the behaviour of that particular unit first.

[@buddymoody]

While watching this I thought of something I often think about (and have never seen written about)... are these "compare output as text" tests good to maintain and keep in your build or are they possibly more useful just DURING REFACTORING. I am a fan of TDD (perhaps not the biggest but a fan nonetheless) but in my mind I do see a subtle difference between how I write tests during development and the tests I want to leave to help the next developer 1) change the code and 2) simply understand it.

[@sarabwt]

It is absolutely good to maintain. It saves you the writing all of the assertions. It is equivalent running the code by hand and comparing the results, except that this is automated. If you like what you see, then you accept the change, otherwise, you change the code to produce different output. It allows you to move much much faster, especially if you have to validate large payloads. Also, when adding new features/removing old ones, the assertions are automatically "kept in sync", meaning that you don't have some tests that validate the new field for example and some tests that you forgot to edit. All of the tests will fail, and you will have to go through 1 by 1 to verify that the output was correct. This type of testing basically makes it so that you actually write the tests once and don't touch them again and you also don't need to be paranoid that you forgot to add some assertion somewhere, as any changes in the output will result in tests failing. I ditched most of the unit testing in favor of doing these kinds of API "e2e/integration" tests in test first style.

[@EmilyBache-tech-coach]

Whether you keep the tests once you've done enough refactoring to get other unit tests in place depends on the cost-benefit analysis. If you've done a good job with the printer and the approved text makes sense, these tests can be cheap to maintain and good as documentation, as well as continuing to find future refactoring errors.