Rails Conf 2013 The Magic Tricks of Testing by Sandi Metz

Tests are supposed to save us money. How is it, then, that many times they become millstones around our necks, gradually morphing into fragile, breakable things that raise the cost of change?
We write too many tests and we test the wrong kinds of things. This talk strips away the veil and offers simple, practical guidelines for choosing what to test and how to test it. Finding the right testing balance isn't magic, it's a magic trick; come and learn the secret of writing stable tests that protect your application at the lowest possible cost.
Help us caption & translate this video!
amara.org/v/FGa6/

[@harunguven8581]

Hello from Odin Project Students :)

[@mdmosabbirhossainkhan4523]

Count me in

[@Brent-The-Carpenter]

me as well

[@deveren]

me too!

[@emannuelmartinez]

Funnily enough, I feel like this talk basically invalidates a bunch of Fun Fun Functions explanation on testing. Their tendency to break every test on every update of their order_total function as a result of testing the code itself and not the side-effects felt repetitive and expensive coming from the business side of things. But Sandi really helped shape how I see testing in an organic way, as we really only should care about the side effects as a result of the class/function/whatever. Sure, the underlying code might be a spaghetti mess, but if the communication pipelines between them is functional and stable, then we really shouldn't have to worry about.

[@harunguven8581]

@@emannuelmartinez chill bro, I just said hello 🙂 I test my programs in my own way and get inspiration from them.

[@jhoylangoncalves3127]

To use ruby language: this video is pure gem.

[@AlexandriaRohn]

02:00 Why do you hate your tests? They're slow. They're fragile. They're expensive. 03:11 It doesn't have to be this way. There's a few tricks to fix the problem and I'm going to go over them during this talk. And for the most of them it's going to be deleting tests. 03:55 Today's talk is about unit tests -- not integration tests. 04:46 We want unit tests to be. Thorough. Stable. Fast. Few. 07:33 Three origins of messages: Incoming, Self, Outgoing. 07:45 Two message types: Query or Command. 09:38 The point of this talk is to fill out this grid. 10:35 Testing Incoming Query messages. Assert result. 13:33 Testing Incoming Command messages. Assert direct public side effects. 15:52 Testing Messages sent to Self messages. Don't. 19:58 Testing Outgoing Query messages. Don't. 23:20 Testing Outgoing Command messages. Expect to send. 27:45 You gotta use mocks. But aren't they fragile? Make sure your mocks stay in sync with the API.

[@ravikumarkumar1983]

Ii ii I ii III nnok

[@stanleysathler2329]

Thank you!

[@fishermand46]

Man Sandi is good at explaining things simply. She'd be a great professor!

[@jrludwig1]

This is a great talk. I have also found these principles through hard experience with poorly written tests. I really like Sandi's table. It give a clear set of rules for unit tests. There is, however, one important piece that is missing. That is that the complexity of the classes in the code leads to either complex unit tests or lack of code coverage by unit tests. In my experience, this leads to bad unit tests more than other factors. It is important to follow the single-responsibility principle when designing objects. If an object has multiple responsibilities, the number of messages-to-self gets bigger than incoming or outgoing messages. Just following Sandi's table and not keeping objects simple, would lead to a lot of untested code that is no longer trivial. A lot of messages-to-self are a sign that the single responsibility principle has been broken and the object needs to be decomposed into sub-objects. I still really like Sandi's table, just make sure that if there is a lot of non-trivial code to handle messages-to-self or a lot of messages-to-self, that you break it into multiple objects each with their own tests.

[@Andriak2]

That signal type/direction table really is a god-send.

[@mlntdrv]

This is all great for the most part. The only problem I have is with the definition of unit - a single class being the unit is too fine-grained. This prohibits some inter-class refactorings like move method or inline class. Worse, it creates a lot of APIs which you have to replace with test doubles CORRECTLY. "Correctly" is the hard part. Better test a cluster of closely related classes than each of them individually.

[@sergeynovik6688]

That's really helpful and the second part was especially insightful, thanks Sandi!

[@CatchTheBus]

Amazing talk. Has really cleared a lot of things for me.

[@RuslanPrytula]

incredible talk! Thanks!

[@saxxi]

Great speech, well done

[@FairyRat]

Great lecture!

[@jonathanyiv6624]

Outstanding

[@InPlainEnglishHQ]

Great talk!

[@rudedoc]

There is no body as concise as Sandi Metz.

[@lovepreetkaul]

nobody not no body dude. that is some serious change in tone due to a space.

[@deveren]

@@lovepreetkaul XDDDD

[@cathu8604]

Amazing talk !!!

[@elissonmichael2807]

Thanks for Sharing This!

[@ghostsquadme]

I'd argue that conflations are just telling you about the side effect that just occurred. In a distributed system, this might be look like "pop returns a pop_id", which you can then query the pop_id for the object that was removed from the queue. Even things like "save!" technically tell you about the operation. If no exception is raised, the save was successful. This is made much more explicit in languages like GoLang, where you would return an error from the command.

[@dgmstuart]

One thing I’ve never quite got a grasp on is where/how best to test outgoing query messages to external services, like APIs (or any code we don’t own. But let’s use API calls as the example). Making sure that we send an expected request to an expected URL definitely feels like a property of the system that we care about, and so should be covered by tests. But GET requests feel pretty unambiguously like queries: in a Restful API they’re supposed to be idempotent, ie: they don’t change any state ie: query method. Maybe the idea is that this is purely an integration test concern, so the rules described in this talk don’t apply? In that case I guess unit tests for methods on an API client object which make GET requests (ie. Query methods) would be pretty minimal: only asserting that given a certain API *response*, the client returns a certain value (based on the response)? I guess that feels OK, as long as there’s an integration test which makes assertions about the actual outgoing http request? Though it does feel like there’s a bit of an unnatural asymmetry: API client methods which make POST requests feel like outgoing command messages, so according to these rules we should make assertions about what was sent? (eg the request body)

[@kobac8207]

This is actually majorly a talk about Outside In TDD school (a.k.a. London school) and it's a pity she didn't mention that or the book that introduced and popularized this approach (Growing object oriented software guided by tests by Steve Freeman and Nat Pryce)

[@Broax]

Her voice is uncanningly like Jodi Foster. I feel like I'm listening to Contact...

[@kenheery5737]

10/10 easy

[@re.liable]

so don't conflate unit tests with integration tests. Right. Thanks

[@ES11777]

Nice

[@nikosc]

Does anyone have a link to Katrina Owens video Zero Confidence, mentioned in the video, or any similar?

[@csbc92]

In case you did not find it already: vimeo.com/68730418

[@nikosc]

@@csbc92 thanks

[@Yoolan]

What does she mean with direct public side effects? I dont get it.

[@jordan4220]

Side effects that can be determined from the outside of an object. Who cares what it does internally so long as it does exactly what it's public API exposes.

[@JFreex]

But sometimes there can be side effects in methods we expect to be queries... How to test them ? Example: a message A calls a function B that queries further C. At some point, C becomes a function with side effects (like incrementing some counter in Database for each call). This side effect of C may lead to unexpected results in the overall program, so how should we approach this test ? I think It is not unit testing anymore, so we should not care about this in our unit tests, right ?

[@ScoFF1k]

If all these methods exists in one the class. You will just test the side effect as it is happing in your class and your class is responsible for this action. I think about unit test as isolated tests that should not test side effects. If you test side effect it is more integration test at this point. I would say it does not really matter how you call tests. It is important to understand what and how you want to test.

[@kacperkepinski4990]

Test incoming query Mes 11:59, not resting implementation 13:22, dont test private methods 18:17, invisible to the test od my app - why dont test messages with no się effect 23:04, 8:13 difference between query and command. 27:47 chart

[@RCMoret]

16:26 don't test your private methods

[@junli4741]

Neil Cicierega

[@ClayShentrup]

you shoulddn't test "within delta". your tests should be deterministic so you know _exactly_ what value to expect.

[@sixro]

I'm probably missing something, but I don't like the example at 13.00 at all. She tests gear.gear_inches with assert_in_delta(137.1, gear.gear_inches, 0.01). Using that solutions (and not using mocking) *she depends* upon implementation, because she has calculated the result knowing exactly the message behaviour of a collaborator (wheel.diameter). If the implemenation of wheel.diameter changes, she has 2 failing tests. I know... it's a wheel.diameter, but passing these ideas creates lots of problem when you don't have an object simple like a wheel...

[@marcink5800]

As long as implementation produce the same value with same entry parameters she will be fine. If a object you depends upon is changing it is natural that your behavior will change as well and your tests must fail. This way she will detect if new implementation of wheel.diameter is correct or not .

[@sixro]

OK, but depending upon implementations of your collaborators will produce a lot of tests to change. Changing all of them will raise costs. Your colleagues will ask you "and now what about 'embrace the change'?" But now I understand that she is talking about Test Automation, not TDD. I think this type of test development can be used in Test Automation... I don't like it anyway. In TDD is not admitted, because you will use a tecnique to drive your design and you'll have lots of tests that won't change in few weeks (and so designed badly) (of course IMHO Marcin).

[@marcink5800]

Roberto Simoni Idea behind implementation is that you can build same functionality in many different ways. If you change implementation functionality should not change. Like sorting you can use different algorithms that will by faster or slower for given dataset but all of them should return sorted result. On the other hand if you change functionality and thus change what given method/class is producing on same input this should be picked up by tests. If you don't check how collaborators are effected by your change in functionality, you get buggy system with subtle bugs. You can still embrace a change if your code is proper OOP you can add features without changing existing codebase by adding new classes.

[@sixro]

Marcin K Marcin, if you like to do TDD creating objects and passing *always* the real implementations of your collaborators and writing your assertions on the behaviours of your real collaborators: go for it. I have done it. When you'll see your tests becoming *the reason* of your slowness in *embracing a change*, let me know. In the meantime, I'll use this video to teach "how badly design you obtained doing that" :)

[@mateusaugusto1812]

Give this woman some water for God sake

[@polegrammer]

The volume quality is terrible

[@HoracioBertorello]

It is. >_

[@ClayShentrup]

i prefer "virtual mocks", which simply assert the expected behavior happened. e.g. if a file is deleted, don't say expect(File).toReceive('rm').with('myFilePath.txt'), but instead just say expectFileToBeDeleted('myFilePath.txt') -- it's no more verbose to write but it completely decouples you from implementation details.