Back to Basics: Unit Testing in C++ - Dave Steffen - CppCon 2024

cppcon.org​
---
Back to Basics: Unit Testing in C++ - Dave Steffen - CppCon 2024
---
Unit Testing is a big, complicated subject. With good advice coming in from books, conference talks, and blog posts beyond count, it's a daunting topic even for experienced developers. Can we make the subject more approachable?
Yes we can. If we look a little deeper, there are some fundamental principles behind the advice. Further, these principles seem to belong to a small number of "domains", each addressing a very different aspect of unit testing and software development.
For example, "Test error conditions separately", "Tests should be easy to read", and "Write the unit tests first" are all great advice, but each is aiming for a very different goal.
In this talk, we'll survey these domains of unit testing practice, identify some of the basic practices involved in each, and put the larger discussion of unit testing into a more useful context. We will also see how some unit testing practices enhance or conflict with others, and how these controversies reveal deep philosophical questions that have real consequences for how we go about the day-to-day activity of testing our code.
---
Slides: github.com/Cpp...
Sponsored by JetBrains: www.jetbrains....
---
Dave Steffen
Dave Steffen completed his Ph.D. in theoretical physics at Colorado State University in 2003, and promptly changed course for a career in software engineering. He has worked primarily in defence and aerospace, and is currently a technical lead at SciTec Inc.'s Boulder office. For reasons unknown, he has turned out to be the expert, champion, and occasional street-corner evangelist for unit testing at most of the companies he has ever worked at.
---
CppCon is the annual, week-long face-to-face gathering for the entire C++ community. The conference is organized by the C++ community for the community. You will enjoy inspirational talks and a friendly atmosphere designed to help attendees learn from each other, meet interesting people, and generally have a stimulating experience. Taking place this year in Aurora, Colorado, near the Denver airport, and including multiple diverse tracks, the conference will appeal to anyone from C++ novices to experts.
Annual CppCon Conference - www.cppcon.org
/ cppcon
x.com/cppcon
/ cppconference
/ cppcon
mastodon.socia...
---
Videos Filmed & Edited by Bash Films: www.BashFilms.com
KZbin Channel Managed by Digital Medium Ltd: events.digital...
---
#cpp #cplusplus #cppcon #cppprogramming #unittest #backtobasics #softwaredevelopment #softwareengineering #coding #code #technology #unittesting #programming #programmer

[@vishalramadoss668]

Excellent talk, lot of very useful information regarding software testing philosphies

[@tHaH4x0r]

What an absolutely great talk. Perfect introduction on unit tests without getting stuck in the nitty gritty. Thank you!

[@KX36]

One of these years they'll do it somewhere else and forget to take out Bjarne talking about the rockies.

[@sebastianschneider326]

good to know that "#define private public" is undefined behavior, I have used this a lot in the past without any issues 😀

[@lf9177]

@davesteffen2410 Really good talk! It helped me a lot to structure the knowlege. I have one question regarding unit testing, that's based on my real life cases. I vividly remember two opposite cases: tests passed, production software failed and tests failed, production software worked. In both cases tests were correct and code was correct! It was very difficult to find the source of the issue and we couldn't find a way to run the tests correctly (at least for the failing tests case). The cause for both issues was timing: some messages were arriving in the wrong order because they were too slow/too fast. The tests have to impact the timing of the execution and we couldn't find any way around that. So, my question is: what to do, when the very presence of tests impacts the results?

[@traister101]

I mean in short fix your tests. You've got some sort of data race if running tests changes the outcome. That's the classic debug print line that can't be removed just in reverse

[@lf9177]

@@traister101 It's hard real time system. If you change timing even a little bit, system returns error. Usually the margin is big enough to test easily. However, sometimes it happens that margin is so small that even one check in between messes up the timing and system is in wrong state when next messages arrives.

[@ABaumstumpf]

44:34 - the test for "empty can be filled" is written poorly: The CHECK-Macro is looking at the state of a boolean variable, so a failing test could only report "Expected true, got false" or something equally meaningless. this runs counter to the earlier talk about using Test-frameworks and their capabilities of concise errorstate descriptions. (And yes we all do code-reviews - some just do it AFTER deployment as i had to learn recently)

[@davesteffen2410]

Ha! Yes -- good point. I was stuck between "keep the examples simple enough for beginners to read quickly", and "Make the point I'm trying to make". At the time CHECK( cup.fill() == true ) maybe struck me as slightly too opaque for the topic? But, that was also probably at about midnight the night before (putting this talk together was very challenging) so we can also put this down to just late-night last-minute tweaks. I'll definitely fix this the next time I present this talk. Thanks !

[@IsaacC20]

@21:00 wait so is the falsifiable hypothesis that "our code has no bugs" or "our code has bugs"? the ppt says one thing and he says another Is the falsifiable hypothesis a statement we're trying to prove false? or prove true?

[@konpet4248]

@@IsaacC20 the falsifiable hypothesis is "our code has no bugs". That's something actually disprovable (by finding a bug). " our code has bugs" is extremely difficult/impossible to disprove, as you'd have to prove that your code is correct

[@cantrell9088]

Intro ends at 7:00 ish

[@paradox8425]

What framework should I use for > C++23 and using modules?

[@stefanalecu9532]

Using modules, huh? You're really brave. Any of the testing units out there work for C++23 as well, I don't know what benefit you'd gain from specifically putting a lower bound as strict as that. To my knowledge, Boost.UT is on C++20 (and doesn't have macros, which is really nice to see). I suppose that's the closest thing you'd want.

[@ABaumstumpf]

Unit-testing is "new" ? it is old than nearly everybody developing C++ - it is certainly older than C++ is. But a big problem is that encapsulation and unit-testing don't mix well if your software doesn't have a good design that makes it testable. It sadly often is the case that say a frontend-API has implicit assumptions and tight coupling to the backend. The function that receives a user-request also is making calls to the Database. This can serve as a good indicator: If your code is hard to unit-test (or worse - testing in general) then your design is hard to reason about, hard to understand, hard to maintain or change.

[@davesteffen2410]

Well.... "new" -- no. That book by Glenford Myers is old, and it's still my favorite! But, unit testing in the current modern meaning, is "new" in that it became widely known in the community in say the early or mid 2000's and wasn't widely recognized as a best practice until sometime after that (depending on where you were in the software community). I took a job in 2013 at a place that had never heard of the idea. 🙂 So some places are still catching up. Also 'new' in that we're still learning about it, and figuring out what the basics are. Part of my motivation for this talk was the fact that every talk, book, and blog post has a different list of "the most important things" (except all the blog posts that just repeat stuff from Hyrum's and Titus' talk!) .... which must mean that we don't all agree on what the foundations are. I think that's the sign of a still-immature field.

[@possumcode]

is OOP a joke?

[@rutabega306]

Yes

[@davesteffen2410]

Not a joke. A very-oversold idea that has become dogma in too many places? Probably. Still a very important technique that solves problems _in certain uses_. Now -- "A joke the way some people do it"? I might agree :-)

[@possumcode]

@@davesteffen2410 Let me clarify. We encapsulate stuff in a class, so testing the stuff is instantly more complex.

[@toby9999]

​@@possumcodeNot really.

[@stefanalecu9532]

​@@possumcode how is it more complex exactly? Give concrete examples (ideally not using any LLMs in the process)

[@adamrushford]

it's an overkill added step with no real value so long as I thoroughly test a function and all it's parts extensively before hand... I have never needed docs at all, no tests, no demos.. a simple example is as good as all that, you won't eliminate bugs, you won't prevent bad launch or patches, you won't eliminate hotfixes and maybe 3 people might ever read that shit while you can explain it lightspeeds faster and better. code documentations and testing are for noobs, simple as that, I built an entire game engine without it.

[@rutabega306]

This will get the code working the first time, but do you have to retest everything every time you refactor?

[@charactername263]

And then you slightly rework the function 3 months later, or optimize one of the data structures it accesses, etc. Unit testing means you can actually touch your code without worrying about regressions flying under the radar. It tells you almost immediately after making changes whether they are breaking changes. That is invaluable.

[@davesteffen2410]

I'm not following -- what is "it" that you think is overkill? Unit testing in general? I'm afraid I have to disagree. You personally might be able to succeed without them, on your personal project that only you work on. When you've got teams, or multiple teams working for years on a project, I don't think so. The overwhelming opinion of the community is that unit testing (or something very much like it) is absolutely necessary. My personal experience matches this, but you don't have to believe me. Talk to people from Google or Bloomberg or any large-scale software project. Heck, even small-scale projects that have real engineering (in the sense of 'long-term maintainability is important') requirements.

[@adamrushford]

@@davesteffen2410 I guess it's the team aspect and the graphical aspect, graphics programs have visual feedback where as other software development has no other form of feedback... so it's a graphics programmer thing to not use them, as well as autodidact.. but at least I finally understand what they are, and what they're for lol

[@toby9999]

That's a recipe for disaster. I've been developing software professionally for 27 years, and most of our regressions can be traced back to insufficient unit testing. It's just not possible to maintain, e.g., 100k lines of code reliably without automated testing.