An Interesting Redirection Bug. How Redirection Can Change Program Behavior.

Patreon ➤ / jacobsorber
Courses ➤ jacobsorber.thinkific.com
Website ➤ www.jacobsorber.com
---
An Interesting Redirection Bug. How Redirection Can Change Program Behavior. // In this video, we check out an interesting bug. Most programmers assume that redirecting input and output doesn't change anything, and usually for common practical purposes, it doesn't. But, this video takes you through one scenario, a bug that my student ran into, where redirection changes a lot. Enjoy.
Related Videos:
***
Welcome! I post videos that help you learn to program and become a more confident software developer. I cover beginner-to-advanced systems topics ranging from network programming, threads, processes, operating systems, embedded systems and others. My goal is to help you get under-the-hood and better understand how computers work and how you can use them to become stronger students and more capable professional developers.
About me: I'm a computer scientist, electrical engineer, researcher, and teacher. I specialize in embedded systems, mobile computing, sensor networks, and the Internet of Things. I teach systems and networking courses at Clemson University, where I also lead the PERSIST research lab.
More about me and what I do:
www.jacobsorber.com
people.cs.clemson.edu/~jsorber/
persist.cs.clemson.edu/
To Support the Channel:
+ like, subscribe, spread the word
+ contribute via Patreon --- [ / jacobsorber ]
Source code is also available to Patreon supporters. --- [jsorber-youtube-source.heroku...]

[@unperrier5998]

Nice bug to demonstrate. And it's nice to be able to redirect directly from gdb as well. Thanks!

[@neerajkale]

At the start of the video I assumed the seg fault has something to do with "> out.txt" been treated as 2 arguments to the program in argv[] 😅😅😅😅 As usual learnt something new again from this channel. 😇

[@lean.drocalil]

There's always highly valuable stuff from this channel. Again, this is spot on👌

[@JacobSorber]

Thanks, Leandro.

[@peterg76yt]

The bug is the bug you deliberately wrote in the fake allocation routine. Redirection is working exactly the way it's supposed to. I'm pretty sure I remember that in the documentation on the standard file descriptors.

[@KevinInPhoenix]

If you write a custom memory allocator and don't test it with small and large values then you are not done yet. Inadequate testing is the root of all evil.

[@fburton8]

And zero and negative memory sizes and different sequences of allocations and... Something as low-level as a custom memory allocator needs to be tested up the wazoo (unless you don't mind random seg faults and futzing around with gdb).

[@unperrier5998]

I thought it was premature optimization that was the root of all evils. Maybe things have changed :)

[@JacobSorber]

So many roots of all the evils. :)

[@Takyodor2]

@@JacobSorber All -roads- roots lead to -Rome- evil!

[@hashinggz]

stderr is not buffered, and back in the day when I learned about it, big deal, performance. Now I know. thank you again!

[@reptilicusrex4748]

Thanks, very interesting and learned a couple gdb features as well.

[@andrewnorris5415]

Like an airline pilot practices checklists in simulators for when things go wrong. So we should practice debugging skills. You never know when they might be needed and would save a ton of time when we need it most and make meeting critical deadlines possible.

[@andrewnorris5415]

Imagine if a critical random bug appears in released code. Would fix it much quicker and with fewer headaches than if all your skills were was print debugging.

[@ranchu3203]

I didn't expect this. So basically, if I am understanding this right, the OS trying to "optimize" with more data being buffered leads to the segfault. Interesting. Oh well you learn new things every day. Great video

[@MarekKnapek]

Not the OS, but the C language run-time.

[@SourabhBhat]

How does the program know that its std output will be redirected by the OS? That seems a bit strange! I thought redirection is done by the OS or the terminal.

[@mikes9016]

Good question, I wonder as well.

[@JacobSorber]

Honestly, I don't know. Never dug that deep. But, now, of course, I need to find out. If the answer is interesting, then maybe it'll show up in a future video.

[@mikes9016]

@@JacobSorber thanks 👍

[@csbnikhil]

A nitpick. A terminal has nothing to do with this. It just renders text on the screen. What you probably meant was a shell.

[@tacokoneko]

i think the program has undefined behavior so until it executes it is in a superposition state where it could either work perfectly or do something completely random and the decisions the operating system makes about where to allocate memory for the program at the moment it is called are having an effect on this balance. so in brief i think the operating system is influencing whether the segfault occurs not the program itself, the program is only providing the undefined behavior

[@razu1976]

This is a lesson dressed up as related to redirection, but is in fact SUSPECT ANY DIFFERENCE when things go a bit weird 😅 Cool drill-down to the details 👍👍👍

[@rkroshan1997]

Awesome learning

[@edgeeffect]

That's fascinating.... a bug that "can't possibly exist".

[@HansBezemer]

Never knew that! But I can fully imagine you were scratching your head when seeing this behavior with your own eyes!

[@JacobSorber]

Yeah, it definitely took a few moments.

[@tacokoneko]

i have arch linux with kernel 5.16 and the program behaves exactly the same on my system, i was also unable to find any arguments for gcc or clang that could change the result. however i wonder if it is possible to create an environment where the program always segfaults or never segfaults, or segfaults without redirection some of the time but not all of the time

[@genz01234]

Nice Practice. Although the bug is created by self(customized allocator).😂 It's good to know "redirect" will also use malloc() to create a buffer for output.

[@benjaminshinar9509]

how did you make the program use the custom allocator? can you share the compile/link command from the makefile?

[@JacobSorber]

Just link the function in with the program. Either include it in your .c file, or compile it in a separate .o file and then link them together. It will use your malloc first before looking for one in libC. And, of course, you can do this at runtime using a shim, as well.

[@will_i_craft5555]

I’ve seen that curl changes it’s behavior when redirecting (printing progress to stderr). Is it detected with a similar behavior there?

[@kebien6020]

It probably uses isatty(3)

[@adrianojordao4634]

Probably there is a vdu that explains the need to do custom alocators. I fully respect your programing skils, but this i miss. And wy 1k alocation is enauff?? Very cool but i feel that i m missing the point. Cheers

[@i007c]

make your intro 5s so when we seek once its complitly seeks the intro

[@wojcieszekk8581]

I wonder what functions did your students use? If have written an allocator for my operating system classes and i used sbrk. Is it better to use mmap or sbrk?

[@BoundedByte]

Just commenting in here so I can learn too-I usually wouldn’t use mmap for this sort of thing. I prefer it for buffering very large file I/O, whereas sbrk is appropriate for custom allocators since that’s usually what malloc is going to actually use in the first place. But if those assumptions are flawed or wrong I’d love to be corrected

[@wojcieszekk8581]

@@BoundedByte I saw mmap implementations and I was just curious what is Jacob's recommendation for this. My requirement was to use standard sbrk. I was wondering what are the differences and what are his requirements for students and how different are my classes compared to his 😀

[@JacobSorber]

I don't know that it really matters. I often have my students use mmap, because then it doesn't interfere with the built-in allocator (which uses sbrk). But, you can definitely use either. Both are just mapping memory for you.

[@wojcieszekk8581]

@@JacobSorber Thank you 🤎

[@alexisrodriguez7127]

Wow super cool and weird

[@ghostfjdgcsusvsgsj]

Never knew Mathew McConaughey could teach programming...

[@YannAriell]

Nice video , but , how to include a function c++ in c

[@yellowkll2853]

write a wrapper

[@YannAriell]

@@yellowkll2853 Yeah but he dsn't work

[@yellowkll2853]

@@YannAriell do you know how to compile source files into object files, and then link the object files together?

[@YannAriell]

@@yellowkll2853 not too much but I know how to include C files in cpp

[@yellowkll2853]

@@YannAriell kzbin.info/www/bejne/aIrJfmB-rbaXlZo