How do Time Travel Debuggers Work? - Design and Implementation of a Time Travel Debugger - Greg Law

www.cppnow.org
---
How do Time Travel Debuggers Work? - Design and Implementation of a Time Travel Debugger - Greg Law - C++Now 2024
---
Time-Travel Debuggers are powerful tools for debugging complex C++ code. They present the programmer with an infinitely detailed view of program execution: the programmer can inspect any piece of program state (memory or registers) for any line of code that executed. And all with surprisingly low runtime and memory overheads. This is especially relevant for C++ which, by design, allows for many kinds of bugs that are impossible in other, safer languages.
But how do these seemingly-magically tools really work? This talk dives deep into the implementation details of the three highest profile time travel debuggers: rr, Microsoft's TTD, and Undo.io. All three have some common traits and important differences. We examine the low-level interactions with the CPU, the Operating System and the language.
---
Slides: github.com/boo...
---
Greg Law
Greg is the co-founder and CEO of Undo.io. He is a coder at heart, but likes to bridge the gap between the business and software worlds.
Greg's experience includes creating and working on Operating Systems, Networking Stacks and Developer Tools. He has held development and management roles at companies including the pioneering British computer firm Acorn, as well as fast-growing start ups, NexWave and Solarflare. It was at Acorn that Greg met Julian and on evenings and weekends, they invented the core technology that would eventually become Undo.
Greg holds a PhD from City University, London and was nominated for the 2001 British Computer Society Distinguished Dissertation Award. He lives in Cambridge, UK with his wife and two children and in his spare time, catches up on email.
---
C++Now 2025 - 28th April - 2nd May
C++Now is an annual onsite international C++ programming and coding conference held in Aspen, Colarado. For all C++ developers, C++ software engineers and those involved with the C++ language, CppNow provides an indepth and technical content provided by the best and brightest C++ experts of the C++ world.
Annual CppNow Conference - www.cppnow.org
/ cppnow
/ cppnow
/ cppnow
/ cppnow
mastodon.socia...
Video Sponsors: millennium and think-cell
---
Videos Filmed & Edited By Bash Films: bashfilms.com/
KZbin Channel Managed & Optimized By Digital Medium Ltd: events.digital...
---
#boost #cpp #debugging #debugger #programming

[@AK-vx4dy]

Today easiest option would be instrument VM

[@lightlegion_]

You’re producing some amazing work!

[@greglaw9145]

thank you! 😍

[@phovos]

Very interesting presentation, thanks!

[@greglaw9145]

thank you! I had a lot of fun giving it

[@johetajava]

Thank you for the talk, and for building this amazing product! The live patching and time travel debugging mentioned might have been referring to this: kzbin.info/www/bejne/bWPcY3h5asuemac

[@valshaped]

One gripe: "gen" in the "Gen AI" buzzword doesn't stand for general, it stands for *generative.* It's intentionally misleading, to make people think it's more capable than it is. Grift, all the way down.

[@greglaw9145]

TBH I don't remember where in the talk this was discussed, but I was aware, promise! If I implied anything else then I, as they say, "misspoke"!

[@JohnDlugosz]

Years ago, in the early or mid '00's, I eagerly upgraded VMWare because of a new feature: recording/replay. You know the step-over should have been step-into deal, especially when trying to find the location initially -- I realized that the VM recording feature would abolish that. Furthermore, if the debugger attached to the process is _also in that VM being recorded_ , then it works seemlessly. At its most basic, the issue of starting over repeatedly to figure out the step-over vs step-into pattern to get to your bug is no longer a problem; one can just revert after the step-over that passed the bug. Why did it take 15 years for that to become mainstream, integrated into the debugger itself?

[@greglaw9145]

You ask a good question! TBF we at Undo have been doing this for many years, it's just that until recently not many knew about it. (Customers and budget for marketing chicken and egg problem which thankfully is now solving itself, albeit slowly!). My take is that VMware discovered what we also discovered the hard way: to some of us this is just so obviously useful everyone will want it, right? In reality it turns out to be _really_ hard to persuade 99% of programmers that they should try it!

[@pollathajeeva23]

In the Distributed systems world we have a very niche space for time travel debugging considering Raft consensus can't make it for DST because of the non-deterministic leader election of nodes in the cluster. How can we resolve it with TTD?

[@greglaw9145]

Beautifully! :) You can record all (or a subset) of processes in the network, and then stitch together all the recordings using something we call Multi-Process Correlation. e.g. you're at a recvmsg() and you want to know where it came from? Simply reverse step through to the corresponding sendmsg() call in another process. If you'll excuse the hubris, it's super cool! :)

[@GeorgeTsiros]

I disagree with the "debugging is harder than programming". I can accept the similar but _different_ : the effort and time spent debugging a program is inversely proportional to the effort and time spent designing and implementing the program

[@greglaw9145]

You make an interesting point! But perhaps a little theoretical -- the reality is that most programmers deal with a codebase that they themselves wrote a tiny proportion of. I also claim that for most of us at least, no matter how much care is taken when designing and writing the program, there is still an awful lot of debugging to be done.

[@douggale5962]

Reverse debugging doesn't work, at all. What magical distro has a build where it works? Some bleeding edge daily build distro? No thanks. If you are wondering why nobody uses it, it is because it doesn't work. The authors always think their stuff works, unfortunately. Maybe it does, in the daily. Distros are making it look bad.

[@destiny_02]

spotted the debian user

[@greglaw9145]

You may be right that what is provided in most distros doesn't work "out of the box". You're totally wrong to claim that it doesn't work though. Thousands of engineers are using Undo routinely to debug some of the biggest and most complex code bases in the world. And that's just Undo, I know people are similarly using rr, and Microsoft's TTD is used a lot by Microsoft themselves as well as their customers. I mean, it must work to some degree otherwise Undo wouldn't have a business, right? :)