CppCon 2018: Matt Godbolt “The Bits Between the Bits: How We Get to main()”

CppCon.org
-
Presentation Slides, PDFs, Source Code and other presenter materials are available at: github.com/CppCon/CppCon2018
-
When you run your C++ code, have you ever considered how the linker, loader, operating system, C and C++ runtime all work so hard to get everything set up for you to start running your code in main()?
In this Linux-focused talk, Matt will talk about how the linker stitches together your code and how that fits in with dynamic linking. He'll touch on debugging issues with the loader, and how ODR violations can manifest themselves. Then he'll take a look at what's going on behind the scenes to get the C runtime up, and then the C++ runtime, along with all the global object constructors - showing more reasons why you shouldn't be using them!
By the end of the talk you should have an understanding of how a bundle of object files are brought together by the linker, along with the relevant runtimes, and then loaded and executed by the operating system.
-
Matt Godbolt, Coinbase
Senior Software Engineer
Matt Godbolt is the creator of the Compiler Explorer website. He is passionate about writing efficient code. He has previously worked at a trading firm, on mobile apps at Google, run his own C++ tools company and spent more than a decade making console games. When he's not hacking on Compiler Explorer, Matt enjoys writing emulators for old 8-bit computer hardware.
-
Videos Filmed & Edited by Bash Films: www.BashFilms.com *-----*
Register Now For CppCon 2022: cppcon.org/registration/
*-----*

[@ulysses_grant]

You know the lecture is damn good when it's been 10 minutes of talk and you didn't even notice.

[@Sidelobes]

Absolutely stellar, top-notch talk! While watching this, I felt scattered pieces o my brain all falling into place and going 'click'. I love how he manages to deliver this in a format that I perceived as quite interactive - a real walkthrough of sorts. I wish I had been able to watch a talk like this at the beginning of my career as a C++ developer.

[@andreasfett6415]

Great talk. Apart from Matt being a really good speaker this is really a topic every C(++) programmer should know about.

[@xplorethings]

This topic is great, during my undergrad years, I often wondered about the process myself. Looking back, I think I was almost angry at linker implementers for not making it faster, well, then I went down the rabbit hole and had to change my opinion.

[@jaime7295]

this is by far my favoirite C++ talk!!

[@SmugDarkLoser10]

This is one of the few language talks at a conference that I find actually useful. Usually this stuff feels like language trivia, but this actually has a level of depth that really impressed me. Good job here.

[@lorenzo42p]

44:44 that is interesting, so clever. the whole presentation is interesting, but this part is something special. the way calls are made to the dynamic linker, it's like built in cache.

[@DatMilu2K]

I thank you very much for your great website and this talk! I have learned a lot about how higher level languages compile into assembly code! Im just a kid who tries to learn everything by myself so it was a great help for me to understand what compilers do!

[@phip1611]

I really like his talks! Really interesting stuff! CS student from Germany here

[@kim15742]

Woow, Ozric Tentacles is amazing! Thanks for the tip!

[@ansakyt]

Oh wow... not just great technical content but reference to some really interesting music. It's a sign of how sheltered my life has been that I'd never heard of Ozric Tentacles before. Thanks Matt! For the tool (compiler explorer), the talk and the tunes...

[@brentjackson6966]

Brilliant talk! I'd say it's a great watch for any programmer who writes compiled or assembled code. You don't *need* to know what happens to make your C or C++ code into an executable, or what is in that executable, but you'll be a stronger programmer if you do. I think it's really important to have some grasp of this compile+link process so, if nothing else, you know there's no magic going on beneath the hood.

[@indianbandhu]

extremely informative talk...thank you so much!!!

[@stephanm.g.2111]

Great talk.

[@realcundo]

Good talk, looking forward to the part 2. :) I had to delve into these once when tracking down a problem with instantiating Foo and returning a pointer to it in one translation unit and then in another TU doing dynamic_cast which always returned NULL. Turned out both translation units saw a Foo class but each TU had its own instance of type_info (with the same contents.) IIRC dynamic cast compared the pointers to type_infos, not the contents so my Foos were never the same type.

[@PamirTea]

Great talk

[@acho8387]

for 95% of this talk I have no idea what's going on, but it's amazing nonetheless

[@SteveHill3D]

Another brit hand up for the Ozric Tentacles!

[@afigegoznaet]

I love this guy

[@benoitrousseau4137]

This is interesting. I think those linker scripts are a GNU thing. The way global constructor calls are implemented in Visual C++ is similar but involves no script at all, it just uses the fact that link.exe sorts section names alphabetically to sandwich global constructor calls between two boundary sections, and then the bootstrap code invokes all function pointers between those boundary sections.

[@williamchamberlain2263]

Heresy! Possibly..

[@severemaisjuste]

Amazing speaker, love the energy and the Archeology metaphor ! That eerie feeling when it feels like reverse-engineering an alien technology gifted to earth :)

[@Swampdragon102]

Wait his name is Godbolt? I always thought the tool just had a really cool brand name, idk

[@nmihaylove]

I am compiling the empty example in early May 2020 with gcc 9.3.0 on Ubuntu 20.04 and both C and CPP compile to 16456 bytes.

[@rohithill]

What software is used for presentation?

[@MattGodbolt]

I use reveal.js. The entire presentation source is at github.com/mattgodbolt/cppcon-bits-between-bits

[@gajananpatil2838]

How cool is that ! Is there any followup video on this topic ?

[@jaredteaches894]

This whole talk is a very brief introduction into computer systems! By far the best book for this is computer systems: a programmers approach! It gives over all of this and more!

[@meow75714]

beautiful

[@CppCon]

Thank you! 😊

[@jimwinchester339]

Best sources I have say BSS = "Block Storage Segment", an old IBM assembler directive used for the exact same purpose (uninitialized data).

[@SimonBuchanNz]

Interesting to see how differently this works from Windows. The Linux dynamic linking in particular seems... bizarre. (Not worse, necessarily, but really weird)

[@GeorgeGeorgalis]

53:09 there must be application of this technique in LLM, eg (for CPU w/o GPU...) load from model into cpu cache (or RAM) all the tokens 2 or 3 nodes deeper than context, before evaluation...

[@zinyang8213]

@ 26:26 5 is the new 6 ;)

[@indi13lli]

Linker command file?

[@Omnifarious0]

In the Linux world, DSOs are all position independent code, so they can be placed at any address in memory.

[@sanchopansa1950]

19:30 linker

[@mywtfmp3]

gcc 8.2 give 18k for the first example, using same compile options.

[@undeaddutch]

Bruce Dickenson has lot of knowlegde!

[@mage3690]

Make linker faster? This may be the ignorance of youth speaking, but I have never waited on either the linker or compiler to do anything. As a matter of fact, I have sometimes told the compiler to rebuild the entire project instead of updating that one line I edited, on the somewhat naive assumption that what I changed could lead to faster execution because changing that one line could lead to the wonderful, nearly sentient optimizing compiler finally figuring out what I was getting at and subsequently optimizing nearly the entire function away into the ether. I'm sure that compiler writers have thought of that sort of thing so I don't have to do that, but still. We all think we can beat the compiler, until we fail miserably.

[@sanchopansa1950]

27:48 symbols

[@henriquealrs]

48:05 why is it you shouldn't use dynamic libraries in financial industry?

[@MattGodbolt]

I was probably too strict there. In many cases one chooses not to use dynamic libraries as they can be a barrier to optimization - calling across a SO boundary is more expensive than a regular call, and you lose opportunities for inlining that using static libraries (with LTO information) gives you.

[@VictorOrdu]

Does this mean that to REALLY understand C++, knowledge of Assembly Language is required?

[@snailnailnailnailnailnails]

no

[@bjarnestronstrup9122]

@@ControversialOpinion You see to use C++ you don't, to understand it you sort of have to.

[@mattr2238]

If you use GDB to debug, you learn both.

[@jonaskoelker]

I would say it's always a good idea to understand the layer below you.

[@x87-64]

To be honest, learning C++ is harder than Assembly

[@edwardmacnab354]

The way everybody throws around the words Make , linker , compiler you'd think everybody just knew exactly what they were and how they worked . Now we see that hardly anybody knows , and if we actually do look at it all in a debugger we will soon understand why hardly anybody knows how they work.

[@anuragkashyap8026]

It's not that hardly anybody knows , I guess you need to get in a good circle .

[@williamchamberlain2263]

1:05 - oh yes

[@__hannibaalbarca__]

This is why CS should be mathematics first before starting in Computing

[@GeorgeTsiros]

main(){} also compiles. It warns about implicit int, but then again, int main() {} compiles without any warning (which imho is worse, because it implicitly returns 0) to the exact same assembly.

[@GeorgeTsiros]

whose problem is that, anyway

[@LunarLambda]

main; also compiles

[@stefanheinzmann7319]

Isn't it curious how computer nerds consistently use the wrong words? They use a compiler to translate the source text to a symbolic machine code representation. Then they use an assembler to transcribe that into a binary machine code representation. Finally, the linker is used to compile a list of sections, seperate them between code and data and such, and to assemble all that into an executable file that has the code in the text section. When you finally want to execute this file, the loader reads it into memory and links it with the shared libraries. You couldn't make this up! That's nerds at their best!

[@MrSapps]

It's more due to hysterical raisins

[@patto2k358]

Culture is a runtime

[@emiliadaria]

I need that shirt.

[@niking987]

Emilia M i got mine from shop.spreadshirt.fr/includecpp

[@persemake6090]

Terrible shirt, no reason to bring this political shit everywhere. Plus it implies that everyone hasn't been included, which simply is not correct. Only good reason to wear this, is attention from naive college girls. (or trolling :D)

[@emiliadaria]

@@persemake6090 How is this shirt 'political'? It's simply wholesome. 💖 And it doesn't imply anything, other than Matt wanting everyone to feel nice and welcome. I'm very sorry you do not feel that way.

[@persemake6090]

@@emiliadaria You must be living under a rock, to not recognize 'politicalness'' of this shirt. Also what would you think if he had a shirt saying "I don't rape"? For me it's obvious that you "include" everyone, stating that in a shirt, is just suspicious, and implies either that wearer doesn't actually think what the message says on the shirt, or that he thinks that others don't think that, so he needs to "educate " them, or that he just wants to virtue signal to stupid people.

[@chrisparker9672]

"White male programmer proclaims social problems have all been fixed. News at 11."

[@xrafter]

So linker is magic? I hate this

[@trejohnson7677]

Linkers are hardly magic. They’re actually straightforward proggies. Read Linkers and Loaders is all I can say. Read “How to Write Shared Libraries” by Ulrich Drepper. One bang them both. Supplement with actual gdb & glibc manual. When you’re ready for your pilgrimage to mecca crack open the System V ABI supplement, and respective documentation for your processor. No magic, just lots of plaintext.

[@richie7425]

Embedded programming is the worst. It's like every embedded related company decided we will make our own way of doing things just because we can...

[@yaroslavpanych2067]

It is 2018, I don't understand why such topic exists!! Developer MUST know such thing almost from the very beginning of career. And no, you will not get duplicate symbol error. Linker will take symbol from the first object/library file it has in input. This way linkers work at least 20 years. I've gained huge benefit of this.

[@noxabellus]

@@higaski Yeah and them not knowing this type of stuff is why those fields host some of the least performant, poorly designed code in the world. Not saying they need to know the specific stuff covered in this video but using these fields as an example of something that works without low level knowledge is giving them way too much credit.

[@ShakkIsLife]

I'm sure many developers don't know this topic well and comments like this are why the online tech community is terrible, especially for beginners.

[@jakearkinstall5313]

5000 views in one day on a specialised topic like this, as well as the overwhelmingly positive feedback, is evidence to how wrong you are.

[@HalfdanIngvarsson]

"And no, you will not get duplicate symbol error. Linker will take symbol from the first object/library file it has in input. This way linkers work at least 20 years. I've gained huge benefit of this." I don't know which linker you've been using, but all the linkers I've been working with for the last three decades, or so, soundly reject linking object files with duplicate, globally visible symbols into a single binary. Sure, if you mark them as local to the translation unit, via 'static', or as weak references via 'inline', then it'll happily link them, but otherwise, no. > echo "void foo() {}" > test1.c > echo "void foo() {}; int main() {}" > test2.c > cc -o t test1.c test2.c FAIL And, no. I would absolutely not expect anyone who works for me to understand this, in the same way I don't expect them to understand the implementation of tgamma(3) or how exceptions are implemented under the hood on any platform. There are bigger things to worry about. Also, knowing this stuff pays /my/ bills ;-)

[@tarickw]

why are you watching a video from a C++ conference and are saying that everybody needs to know every little thing. The whole point of C++ is good abstractions, which in turn can be used to build better abstractions so that people DON'T need to know all the nitty gritty details. That of course doesn't mean that some developers need to know such details. The branching limitations of arm is such a detail that embedded developers should be aware off.

[@anon_y_mousse]

Even though I already knew most of this, it was fun. I just wish Brit's would stop intentionally mispronouncing so many words as well as pronouncing abbreviations that don't really form pronounceable words. For instance, xor is an abbreviation of eXclusive OR, so you should say X-OR, and the char type in C and by extension in C++ is an abbreviation for character, so it should be pronounced like care and not the fire adjacent form.