Record and replay debugging with "rr"

Robert O'Callahan
linux.conf.au/schedule/30013/...
Debugging is expensive and not fun, especially debugging intermittent failures. rr changes that by recording Linux user-space process execution and letting you play back a recording and debug it using gdb, getting exactly the same execution as was recorded, as many times as you need. rr is used by Mozilla developers to debug Firefox, so it works on real applications. It runs on stock hardware, requires no kernel changes, and has low recording overhead (usually less than 1.5x). rr supports gdb's reverse execution features, which (combined with breakpoints and hardware data watchpoints) make debugging fast and fun. This presentation will describe how rr works, discuss some of the kernel issues that created difficulties for rr, and speculate about what we can do to make rr and related tools even better.

[@simivb]

Its just a tiny nitpick, but it drives me crazy that so many people use the term nondeterministic wrongly, like how "exponential" has just become a synonym for "very fast". You have nondeterminism when you know everything there is to know about your current state, but you still can't predict the following state. User input or timing does not introduce nondeterminism. The fact that you don't know the timing or user input simply means that you don't know what state you are in. But if you knew ALL about yout state, you could perfectly predict the next state of a program. In fact, the very approach used here, of replaying these input demonstrates that all software is deterministic!

[@YoloMonstaaa]

by that definition, nothing would ever be non-deterministic because if you know everything about everything, you can always predict the next state.