Stupid C Tricks: Unsafe Functions You MUST Avoid!

[@DavesGarage]

My apologies for the erroneous prototypes that indicate a return value: I stripped the return values at the last minute to simplify things, but didn't change the prototypes, proving there's "no such thing as a trivial last minute change!"

[@decky1990]

Read my mind!! In my head I was screaming “why is it not void!!??” 😂

[@richardyao9012]

You still have the problem of describing snprintf() incorrectly. It will always write a valid string, unless you tell it the buffer is 0 length. Then it writes nothing.

[@mcg6762]

Please also clear up your erroneous statement that snprintf is unsafe. snprintf is safe and will not cause buffer overflow or unterminated strings. See my other top-level comment.

[@shahqu5dohcoh9ri88]

I have to questions: 1. I read that the _s function are supported only by msvc and they're not standard. They were like standard in c11 if I remember correctly, but it was optional to implement them. In fact gcc and clang don't support those. Doesn't the use of those drastically reduce the portability of the code? 2. How can it be that the standard library, on which all the compilers I've written before rely on, failes to be secure? Is there a way to use it 100% safely which doesn't consist in rewriting the string library from scratch with safe _s functions?

[@LordErnie]

I'm curious about the book. Something I'll add to my collection for sure in the near future. Any advice for when I'm reading it? Great vid btw. It really shows the evolution of C, and how a language known for its dangerous capabilities can become so friendly to use. Isn't there a performance drop in using the safe functions though?

[@mathewbenson5447]

Your first two functions have a size_t return type, but there is no return call.

[@DavesGarage]

My apologies for the erroneous prototypes that indicate a return value: I stripped the return values at the last minute to simplify things, but didn't change the prototypes, proving there's "no such thing as a trivial last minute change!"

[@tweakoz]

@@DavesGarage I do not recall what would happen in windows/msvc, but on Linux/gcc/clang, in my experience - not returning something when the signature specifies to do so can definitely result in a crash. I believe the crash is stack/postamble related, as opposed to some downstream bug due to returning garbage.

[@stephenhookings1985]

It's definitely not wise to return garbage especially if someone tries to use it

[@johnshaw6702]

@@tweakoz You reminded me of a time when I tried to fix all the functions in the code I inherited to actually return something known, instead of whatever just happen to be in the ax register. It broke the code. 😂 I didn't have the time to figure out why, so I just left it alone after that. 🤔

[@erikkonstas]

@@johnshaw6702 Yuck, sounds like somebody knew some particular undefined behavior and used it...

[@uzimonkey]

_countof is not new to C because it's not in C at all. It's a Microsoft macro buried in one of their headers. Probably still best to use sizeof(x) / sizeof(x[0]) or since we know these are char arrays, just sizeof(x). Edit: Further, I don't think any of the _s functions are available on glibc. They're in C11, but in the optional annex k which they never implemented. I don't know why, but at least on my recent GCC/glibc installation they're not present.

[@doBobro]

You can try to define __STDC_WANT_LIB_EXT1__ before including string.h

[@jongeduard]

Thanks! Dave is clearly a programmer mostly on just Windows. This is all kind of matching with my other reply here, where I explain that I never used those _s versions as well, because again, it was MSVC only suggesting these and deprecating the older ones, while general standards didn't even have those functions. This may have changed today, but I still don't believe the non-_s versions are actually deprecated according official standards. Or can anyone confirm otherwise?

[@packmandudefake]

@@jongeduard My K&R book copy tells it’s fine.

[@cianmoriarty7345]

The general convention is anything indentiers with a leading underscore is an implementation detail which might change, so if you don't own it, ignore it or everything might break.

[@jongeduard]

@@cianmoriarty7345 I am not so sure about that explanation as the only possibility. Just think about things like __stdcall and __declspec, which really never changed and probably never will, but for which the explanation is that they are absolutely Windows-only, rather then that they are expected to change.

[@mcg6762]

snprintf will not overflow the buffer and it will add null termination, so it is safe. The thing to be aware of is that it returns the number of characters it would have written (excluding null terminator) if the buffer would have been large enough. That way you can check if the resulting string was truncated due to the buffer being too small.

[@mcg6762]

@@JS-oh2dp I think Dave got this one wrong. I don't know what he was talking about. See "man snprintf" in any Linux distro and it is clearly explained there consistent with what I wrote above.

[@johnshaw6702]

@@mcg6762 I haven't used it in years, but that sounds right. Although I would need to verify that it guarantees null termination. I'm definitely a trust but verify guy.

[@richardyao9012]

@@johnshaw6702It does. We rely on it in OpenZFS because it does this.

[@paulk314]

This video seems to have an incorrect description of strtok and strtok_s. The strtok_exampleC1 is actually totally fine (aside from not being multithread-safe). In the video, he says "The problem here is that it's partying[?] on your memory and you have no idea how big your memory is." This seems to be contradicted a couple of sentences later where he implies (accurately) that strtok just modifies the input string: "So by using strtok_s, we can do it safely. Rather than modifying the string in place, it provides you with another value, rest, which is where the rest of the string can be found". This contradicts what he says earlier about strtok "partying on your memory". No, it replaces any occurrences of characters from the delimiter string in the input string with null terminators and returns a pointer to the current token (see the man page). As long as the input string is null terminated (i.e. a valid C string), it's totally safe for single-threaded use. The issue with strtok is that it stores a pointer to the input string in a static variable (which is why on subsequent calls you pass a NULL pointer as the input string parameter so that it knows to continue tokenizing the same string) which isn't threadsafe. You need to use the reentrant version, strtok_r, to be threadsafe (again, see the man page). According to my quick scan of the MSDN page on strtok_s, it appears that it is the Microsoft equivalent of the POSIX strtok_r (or roughly so). Both functions use a third parameter as an input/output to store a context for tokenizing an input string instead of a static variable. Passing NULL in the first parameter indicates that it should continue tokenizing from the context pointer. This is totally threadsafe. The incorrect understanding of strtok in the video is evidenced by the name of the variable passed in as the second argument to strtok: "tok". That should be "delim" or something since it tells strtok which characters to use as delimiters, it doesn't output the next token there or anything.

[@CapnSlipp]

An alternative fix for strcat_exampleC2() is to just (always) set the last character to null- szStrOut[MAX_PATH-1] = '\0’; It doesn’t matter if the string ends up being shorter; all string functions stop reading when they hit null, and setting the final character unconditionally means no branching and thus produces simple fast assembly.

[@stolenlaptop]

I use this truncating effect often.

[@TomStorey96]

OG strtok also has another flaw that makes it unsafe for threaded applications: it uses a static variable to keep track of where it finished off the previous call so that it knows where to start on the next.

[@hofertyp]

I remember the early 00 years sitting in programming class and my teacher just throwing that red/white kerninghan and Ritchie C book -> still to this day pointers and functions with pointers extremely scares me. Somehow I'm glad that I've started with C. It takes you much much longer but because it's more leaned towards hardware you learn so much more important background basics that let's you better understand how the software integrates with the hardware!

[@ralfbaechle]

Red and white? Sounds like you're a German speaker and were given the German translation. The original English issue has a white and blue cover. I've red both and think the German version is a very careful translation by somebody who understood the subject and was very careful about certain things including pointers. It's how I learned C but from mid 80's copy about K&R C. Pascal's pointer concepts are similar but gave me the feeling somebody was doing his best to make the syntax more obscure to discourage their use. Pointers can be incredibly elegant.for building efficient data structures. Things are getting ugly once reaching the point of multithreaded code with lockless data structures scaling to many possibly thousands of processors. Few people need to care about such mindbending exercices.

[@hofertyp]

@@ralfbaechle exactly the book was well written but still very difficult for a beginner. Barely made the exams and then we switched to JAVA and C# with .NET framework. Currently working in the IT-Sec field but as an hobby programming on Mikrocontroller like ESP32 (differnet Sensors via WiFi or LoRa) so trying to improve rusty C skills ;)

[@ralfbaechle]

@@hofertyp I had some low-level programming experience by the time I read the K&R C book. Plus it was loaned so I *had* to absorb it within 24 hours which worked ok for most part. Good prep for an eventual carreers as a Linux kernel developer. Java and C# all sound useful in sense of they might land you an actual job. My own introductory course was taught in Pascal. Turbo Pascal to be accurat which probably the most popular programming language on PCs at the time.. My university observed there was a complete split between people with piror and no prior programming experience. The one groupswas able to get top grades with little effort, the other had to work hard. No middle ground. So the following years they thought they could level the field between the groups by using a different pgramming language. And of all the things they could have picked, they picked Postscript. I kid ya not. Untill the end of the semester printers were running the Sieve of Eratosthenes or other stuff printing out prime numbers on paper. That is if they were not outright spitting out the entire paper supply. The trays of the university's computer centre where people could pick up their printouts were particularly well filled that year. Of course students were supposed to test on ghostscript after that they they sent their code straight to the printer not realizing that would cause the printer's RIP to execute the code. The following year the language was change again. That timei it was LISP. ITsec can be fun as well :)

[@Ryan-xq3kl]

If functions with pointers scare you use a c++ smart pointer, you can automate your resource management process and have peace of mind.

[@johnshaw6702]

I actually took my one and only C programming class because I was having an issue understanding standing pointers to pointers to etcetera. One thing I liked about C is I could look at the code and practically see what the assembly would look like in my head. There is a reason why writing ++i is good practice and writing i++ is not, but that's another story.

[@WolvericCatkin]

Correction: `endl` isn't just the newline character, the standard specifically specifies it should explicitly _flush_ the relevant buffer, which in the best case, is a no-op, given most buffers are designed to flush on newlines, but at worst, can be introducing unnecessary overhead if the flush is poorly implemented.

[@superscatboy]

Absolutely correct. std::endl is honestly a curse IMO

[@ufufuawa401]

std::endl manipulator is used inorder to behaving stream line buffered when encounter newline, because std::ostream is fully buffered. C stdout is Line Buffered

[@tomysshadow]

Visual Studio won't even let you use most of these anymore without setting a specific flag. Which is a good thing, it's certainly saved people from making these mistakes.

[@richardyao9012]

If you use LLVM/Clang like Google does for Chrome, you need no such flag.

[@Scotty-vs4lf]

always get in my way cuz im intentionally writing unsafe code lol

[@tomysshadow]

@@richardyao9012 What do you mean exactly? IMO you shouldn't be using the flag anyway, at least not for new code.

[@MattSuguisAsFondAsEverrr]

it's sad that the bots have arrived

[@richardyao9012]

There are uses of strcat() and strcpy() that are safe, provided that you know the lengths of the strings in advance to compare against a buffer length. There are a few of these safe uses in OpenZFS. If I recall, strcat is used to make a mount options string for a network mount in ZFS where a bunch of fixed length strings are concatenated to make the final string. The total number of bytes in the longest possible output string is smaller than the buffer length in that case.

[@DocaTafner]

I was a C and C++ dev back in the late 90's, self taught. After that, I told myself I would not have use of my skills because C/C++ in general is not used for developing business solutions, instead it's used for some background scrubbing of bytes and data, but definitely not a full business solution (not without a significant amount of time and effort, that is). So I turned to IT infrastructure. I haven't touched programming through all those years. This video makes me wonder why did I stop, it was all too natural to me. Thanks for the video, Dave. Perhaps you reignited something which I love.

[@turdwarbler]

first example also had another bug, it says it returns a size_t but there is no return statement !!!

[@alakani]

And a bug in the PowerPoint.. it should have started as "Can you spot the bug?" and then added the "s" when he said "actually" :P

[@beowes]

Won’t C just return whatever is in EAX? I guess the return of puts() remains? C 6.9.1, 12 If the } that terminates a function is reached, and the value of the function call is used by the caller, the behavior is undefined. ?

[@JCWren]

Not to mention it didn't check the arguments for being null.

[@turdwarbler]

@@JCWren true, but that in itself isnt a bug. It may be bad programming, BUT if you know the parameters are never null then you dont need to check. However, it would be wise to check unless there is an overriding reason not to.

[@JCWren]

@@turdwarbler That same argument can be made for strcpy() and friends. If I >know< the incoming lengths won't exceed the buffer, it's "safe". If it's a function that's entirely local to your program, then yes, you might not need to check the arguments. If it's exposed at all, it should return an error code or at least do an assert() check.

[@nickwallette6201]

This reminds me why I code almost exclusively on Linux. I really would like to learn how to write Windows programs (and I’m into retro computing, so that would be C/C++ 7.0 on Win 3.0 to VS 2000-whatever on Win 7), but I am completely lost on the quagmire of string handling and file IO calls with different char sizes, and having everything abstracted behind cryptic type definitions. I’ve tried learning this a few times, and it seems like every example says “don’t use all the other ones, just do this ...” and then directly contradicts the last one I read. It just feels like there’s standard ANSI C, and there’s ... whatever the 🌮 is going on in any given Windows source code example.

[@johnshaw6702]

I sometimes thought I should have become an MSVP, because half the things they wrote wouldn't have passed my basic test for good coding. But was useful as a starting point.

[@derlukas8182]

another problem with the first example is, that a non-void return type is declared but no value is returned. If any caller function attempted to use said return value it could lead to undefined behavior

[@HaydenLikeHey]

As a relatively new C programmer, I find stuff like this really helpful and fascinating! Thanks for this, Dave!

[@Ryan-xq3kl]

Consider yourself lucky, I started 3 years ago and now have to rewrite a whole library of unsafe code xD

[@lwilton]

@@Ryan-xq3kl You can use all of the old functions safely, but you have to do ALL of the checking that is buried inside the new Windows "safe" functions. It is easy to miss checks, especially if you don't have someone to look over your shoulder and review your code and ask questions.. And the _s functions aren't a panacea, because some of them do strange things like padding the output buffer with nulls out to the full count, where the old length-specifying functions would have stopped after the input length.

[@johnshaw6702]

I learned and wrote C code on my own for years (my first real language). My one rule for coding in any language is to constantly check that it does what you think it does. You can write safe code, but the string functions are an area that you have to pay close attention to. Languages like C/C++ does not even specify what the character size is. It basically says a short is >= a character size, int >= short, long >= int, but doesn't say what size char is. That is implementation defined.

[@grahamheath3799]

Useful; Reminds me of a problem. Many years ago when when the UNIX time (number of seconds since the epoch) went from 999..... to 1000..... i.e. ASCII length increased by 1. An application that used this to generate a file name suddenly started misbehaving. As a UK support group we worked out what was happening. The only difficulty was convincing the code writer that somebody without an american accent knew what was wrong with his code.

[@godfreypoon5148]

I didn't know they had computers in the UK!

[@grahamheath3799]

@@godfreypoon5148 yes we made that leap just after we took our socks off and could count up to 14.

[@4X6GP]

@@grahamheath3799 So they figured out how to make them leak oil?

[@grahamheath3799]

@@4X6GP ??

[@russlehman2070]

@@godfreypoon5148 Joke: Q: Why don't the British manufacture computers? A: They couldn't figure out how to make them leak oil.

[@landonkryger]

I've recently been doing a ton of of string printf upgrades at work. Instead of sprintf or it's variantes, I've been using std::format and love it. It does require you to be using c++20 though.

[@jope2]

Could you clarify the issue with snprintf? You say "the size specifier is generally based on the format specifiers", which is also true with sprintf_s. If you don't have a big enough buffer, snprintf will truncate, put a null terminator, and return how many bytes it would have written if the buffer was big enough (excluding the null terminator). sprintf_s and snprintf_s do the same but sprintf_s returns how many bytes it actually wrote, and snprintf_s returns how many it would have written, same as snprintf. In all three you need to check the return value to know if the string was truncated or not, but in terms of not going off bounds and returning a null terminated string they all seem equally safe.

[@richardgg2889]

On the STRTOK part: - tried both C and C++ ways and STRTOK was maybe 90% faster than using stringstream in c++ when parsing very large text files in loop.

[@Gragledump]

yea c++ stream-related functions tend to have this problem in my experience

[@perwestermark8920]

That's because strtok() is allowed to modify the input string. So there is zero copying of data. The issue with strtok() isn't amount of RAM, but that it isn't thread-safe. It uses a global variable to keep track of where to continue. And Posix has a strtok_r() that solves that by just as strtok_s() take one parameter extra to hold the state.

[@DavesGarage]

It's not first one to crash wins, though. It matters more if the code is safe, THEN make it fast.

[@frankhenigman5117]

@@DavesGarage I think your strtok explanation could be better. You say something like "partying on memory of unknown size" which, apart from being unclear, implies (to me at least) there is some kind of overflow danger. But as I understand it the only issue is thread safety (which you don't mention) and the only thing strtok_s (and strtok_r which is the same) does differently is to address that.

[@turdwarbler]

@@perwestermark8920 Yes strtok()isnt thread safe, but so what, that doesnt make it a bad function, it just means it isnt thread safe. If you are writing a single threaded program its perfectly safe as it has always been. Its horses for courses. Use the most appropriate code for your use case. Its like when people say, oh that code isnt portable, again, so bloody what, if I am writing code to target x86-64 who cares whether its portable or not. Too many people worry about things that dont matter. If non portable code is faster, then thats what I will use because it matters to me.

[@modolief]

Great instructive video, perfect for my needs: I'm teaching computer science to a budding software engineer and C is one of the languages I'm teaching him. I always remembered string handling in C as difficult, and am glad to have an update on the proper methodologies for doing this important task.

[@frityet2840]

The _s functions are not the proper methodology, and should not be used as such

[@modolief]

@@frityet2840 So then ... how should we do strings nowadays?

[@nicolefischer1504]

@@modolief As you always have. Safe code existed in C since before these functions and we don't need them now.

[@technowey]

This was a good video. I'd forgotten, if I ever knew, that strcpy_s always added a null terminator. The bugs are very obvious to an experienced C programmer. The C++ function had a bug too. The return type was size_t, however, the function didn't return a value.

[@Dizintegrator]

Loved the exaples for both plain C and C++, showing how much easier those basic things are now.

[@skiera0000]

I would love to watch more videos about C/C++ code by you!

[@mathewbenson5447]

Make more 'spot the bugs'. I love these. I use my own as interview questions.

[@richardblain4783]

Since you’re only printing the concatenated strings anyway, you don’t need the output buffer, or any string copy or append functions. Just puts the first string, then puts(“ “), then puts the second string.

[@GamersUniverseOE]

This will create newlines between them tho. printf("%s %s", str1, str2) is probably better.

[@richardblain4783]

I forgot (if I ever knew) that puts() adds a newline. Thanks for catching my mistake.

[@lonelymtn]

These should be basics when starting out with C.

[@TheRealStructurer]

Much appreciated. On a hobby level I have dabbled with Objective C, C and C++ and Swift, and of course have had some issues. Right now deep into an Arduino project and good to be reminded of how to think and to write safe code. And for me as a hobbyist, some tools and languages are easier and safer than others

[@ikouzaki]

Amazing vid. Very educational! I'd love more of this. Thanks for dishing out such high quality content.

[@askdiatom]

I learned something new!. Please make more of these videos!!. Thanks, Dave!

[@Unfinished80]

Thank you! This answered a lot of questions I've had about the number of string function versions.

[@richardrisner921]

This is a very strong argument against writing programs in C now that alternatives exist.

[@richardyao9012]

How is this a strong argument? Anyone can write any code incorrectly in any language. Use functions correctly and you will not have a problem. Use a static analyzer to warn you about such functions being used incorrectly (or used at all if you are really unsure of yourself and performance overhead from using the “safe” versions where you do not need them is not a problem) and you should be fine.

[@richardrisner921]

@@richardyao9012 if 80% of the standard library is deprecated and unsafe, not only is it unclear which standard functions you should be using (besides the fact that the names of those functions are esoteric), but also if you just guess you are more likely to do something unsafe than safe. At this point it is more of a puzzle game like minesweeper, where it's fun because if you are not excruciatingly careful you will blow up. For some applications C is still the only serious tool for the job at this point, but I would not want to use it if there was an alternative available that would work with me rather than against me for writing safe programs.

[@ailivac]

There's also asprintf, which I think isn't part of any standard yet but is still available most places. It allocates a buffer just long enough and returns a pointer to it which you do have to free later, but you don't have to choose between wasting a bunch of memory every single time or risking some of your strings getting truncated.

[@BaseNAND]

You can easily do that with snprintf. If you pass it 0 as size, nothing will be written to the output (which may then also be a NULL pointer) but it still returns the size of the resulting string. With that size you can allocate a new block of memory that is large enough and then do the same snprintf call for that block of memory.

[@ailivac]

@@BaseNAND Forgot about that. Still, asprintf does it all for you in a single line that's basically impossible to get wrong.

[@flatfingertuning727]

@@BaseNAND Note that doing so may not be safe if one is receiving a pointer to a string from code running in an untrusted context, since the length of a string could change between the two calls to snprintf.

[@BaseNAND]

​@@flatfingertuning727 You mean strings passed as arguments for the format specifiers? That is true, but that is also true for asprintf since it uses the method I described (only with vsnprintf because it is variadic). If you do it manually you can actually catch this case by checking the return value of the second call to (v)snprintf against the size you allocated with malloc/new/etc., since the second call will return something else than the first. asprintf does not handle this case and will always return the size of the string from the second call, not the size of the memory block which is based on the first call.

[@richardblain4783]

Sounds like an invitation for a memory leak.

[@kensmith5694]

size_t type function has no return. You didn't check buffer lengths. You trusted the input to actually have the zero on the end.

[@ralfbaechle]

So much truth in that. I inherited an old codebase which probably started off as a quick hack and from that point. And now time and time again I keep finding similar bugs. Sometimes they do bite, sometimes using various code checkers, sometimes by inspection.

[@peterg76yt]

When I first learned printf I was astounded at the idea of a function with absolutely no type safety, and then more astounded that I seemed to be the only person bothered by it.

[@seancollins9745]

Dave, thank you, asa newb to C and programming in general this was very very very useful information !!!! thank you for these videos !!!

[@oscarmilstein8026]

These safer versions of the standard functions are unfortunately only implemented in MSVC and not in for example glibc. This makes code using these functions not very portable.

[@perwestermark8920]

If you are on Linux, then snprintf() is safe - it will give zero-terminated output. But will give different result for invalid formatting string, doing best effort instead of giving empty string and an error. So snprintf(buf,NELEM(buf),"%s%s",str1,str2) is a safe concatenation without access to strcat_s(). And strtok_r() is the thread-safe replacement for strtok() on Posix-compliant systems.

[@DMLou]

I was about to say... but you beat me to it. Yes, they're in the C++11 standard, but only as optional extensions. The glibc folks decided not to implement them. I don't have a link handy, but they even published a paper to the C standards committee saying why they aren't that great to begin with. I only skimmed the paper, though, so I have no opinion on whether I agree with them or not.

[@brycemw]

I often ended up writing some of these kinds of functions myself so I can do it the way I want from the start. I also hate how using string functions ends up with a lot of redundant calls to strlen. So I’ll often just do the strlen once and write everything else using more generic functions like memcpy

[@perwestermark8920]

Note that sane implementations of snprintf() is guaranteeing a zero-terminated string as long as buf is at least one character large. So same as sprintf_s(). The difference is sprintf_s() will issue an error and result in an empty string if the formatting string is invalid.

[@pavelglosl8221]

To Dave Plummer (mainly): Can you please describe the differences between functions snprintf() vs. sprintf_s() in more detail (again)? - Because I still can't see problem with snprintf() regarding to overwriting 'foreign' RAM after end of dest. buffer. Can you show/spot some example case (situation) where it happens? - I mean particular format string altogether with combination of given parameters/variables (and of course theirs values) to be ('formatedly' - sorry for my English..) printed (out) to the destination string (buffer)? My native language is not English and I tried to play given part of your video at slower speed, but still don't fully understand it... :-( Thank you very much. Oh, and I would like to personally publish my ('own') bellowed trick how to do the same function like strncpy() but replacing it by strncat() this way: first do assign binary zero char in the very 1st byte (at index zero) of destination string/buffer and then use strncat() with right/adequate destination buffer size (most typically using sizeof() operator on dest. buff.). That's it - quite simple, isn't it? ;-)

[@perwestermark8920]

@@pavelglosl8221 BSD +C99 specifies that snprintf() reports number of characters written (or that should have been written if the buffer was big enough) excluding the terminating zero. So a return value >= the buffer size means the buffer was too small and a new buffer that is return value+1 or bigger is needed. But the produced output will be a truncated and zero-terminated string. Before c99, some snprintf() [such as before glibc 2.2] did return number of actually written characters. So no way of knowing if a larger buffer was needed. And some old snprintf() did return -1 and set errno. But no snprintf() I have ever seen have resulted in any memory overwrites or unterminated strings. Just complications figuring out how to allocate a larger buffer. With old snprintf() that returned -1, one sad way was to have a loop and double the buffer size each time -1 was returned. The variant that instead returned number of written characters was even safer since the return value doesn't indicate if any error handling is needed.

[@pavelglosl8221]

@@perwestermark8920 Thank you a lot for your detailed explanation! I think I understand it now. More to that - I was focused mostly (and only) on possible buffer overrun/overwrite rather than considering all kinds of errors in program and it's behavior + output like David's aim to point out on unfinished/ non complete (truncated) output in text form; which other program can read and hence be dependent on it.

[@perwestermark8920]

@@pavelglosl8221 If the output is important, then function return values should always be processed and there should be some strategy for how to take care of non-recoverable errors so a user and later developer can be properly informed. If it's debug output, then truncated texts can be quite common - you get a best effort message where a smart developer focuses on most important info early. And in a multithreaded application it doesn't work well to try and split a long log output into multiple output because that can mix up the output with other thread logging. If the purpose of the output is to create text that is actually the focus of the program, then the code should be written so it can't fail unless you run out of RAM or disk. So the developer should keep track of what outputs that can be unbound aka huge and then make sure that is splitted into some loops with partial delivery. If the code produces an output file, then write to a temp file with arbitrary number of smaller writes and rename the file when last data has been written ok. The problem is that the code to check return values can make code much, much bigger. So it's common to just hope for the best. Just that this means making assumptions about what may fail. And assumptions are quite often wrong. Spelled ass-u-me for a reason... Sometimes it can help by having own primitives that are built on top of the standard functions but also includes either recovery logic or integration with the error logging functionality. So a save_file() function has either done what it can to recover from problems or have made sure there is a good error message + debug logging generated about why it could not deliver.

[@mittelwelle_531_khz]

char s2[N+1]; strncpy(s2, s, N)[N] = '\0'; does the job very well. No new function required. BTW: there is a reason why the good ol' strncpy does not add the '\0'-byte if all available space is filled.

[@vfjpl1]

THX

[@xav500011]

Great video. Oh well I will stay in the safe waters of C# for a while longer.

[@Aybex97]

I started programming in C/C++ at young age, then discovered C# later on, it's a bless man ! I can't help but feel spoiled with all the goodies of that keep getting added to dotnet and C# year after year.

[@taishi-sama-tfc]

Try Rust first, it can be pretty difficult to learn, but all things that make Rust hard to learn is just good practices from C and C++ how to write safe(firstly memory- and thread-safe) code, enforced by compiler, such as borrow checking, lifetimes and etc. Also Rustc'a compiler errors are really helpful, and Rust language haven't heavy legacy (hello C++), so you would learn not about ways how to not shoot in your foot, but actual useful concepts. And after learning Rust this will be easy to learn about modern C++ and C

[@WndSks]

Correction: lstrcpynA&W will always zero terminate the string correctly.

[@pdrg]

Thanks for reminding me why I don't do C any more if I can help it!

[@HassassinCat]

Fantastic video. Please, please, please, start a course of C/C++. The way you explain is gold, showing each step, explaining without making us feel like babies. I cant have enought of this. At least, please make more of this kind of video explaining things. Love ya

[@dymastro788]

Dave making a course would be amazing

[@Hiram8866]

As someone who has only dabbled in C programming at a very basic level. I enjoyed this video.

[@garycharpenter543]

Thanks much for this. One day all these depreciated error makers will be pulled from the libraries (not). Of course you assumed your input strings are null terminated even if larger than expected, however they can also overflow outside your memory space even while being read if they are not. Love to see an episode on string input verification.

[@josephcsible]

The _s family of functions are nonportable because they're from Annex K, which the standard doesn't require, and in practice basically no platforms actually implement as defined.

[@DavesGarage]

Try a C compiler from about 1994 or later!

[@josephcsible]

@@DavesGarage I tried with GCC 11.3 and glibc 2.35 on Ubuntu 22.04 (all released in 2022), and the Annex K functions didn't exist there. Other than MSVC (and even it doesn't quite adhere properly to it), what mainstream compilers/toolchains do support it?

[@peterjansen4826]

C is a fine language, sadly too many of us haven't had proper training with it which is what gives C the reputation to be a bit dangerous (software-security wise). In my case the university also is to blame, in my opinion. Why does programming get teated like any subject while it is an art. You can't become a skilled programmer in just 8/12/20 weeks, however long a period lasts at a university, teaching programming should be done in a different way than for example teachint physics or chemistry, it should be taught by being a red thread throughout the curriculum, not so much by giving assignments and not giving any feedback but by giving pointers, checking what the student has done, then explaining what a better approach would have been...You shouldn't get graded on it with an exam, you should get continued feedback. The major point is that programming iis an art and a skiill, not so much about reproducing knowledge.

[@HannibalLecter-w3r]

the only 2 bugs in original function are: -function is size_t but there is no return -while explaining the function you didn't specify the 260 chars limitation, it should be in the docs or comment near the function MAX_PATH is used specifically for working with path in windows which is also limited, so if you somehow exceed it, your program is not going to work even with larger buffer. p.s. there is some fancy way to remove this 260 limitation, but no one does it except microsoft because 260 is always enough

[@lostwizard]

Even worse with that first example with the strncat()s is that the "n" in strncat refers to the *source* string, not the destination. So strncat(szStrOut, szStr2, MAX_PATH-1) will still risk overflowing szStrOut.

[@kc9scott]

Wow, I never realized that, and it’s truly awful. What you say is backed up by a quick internet search. I’ll have to check if I ever use that function.

[@protheu5]

Great video as always. Would be cool to see more advanced C++ techniques.

[@russlehman2070]

Some of my earliest efforts at programming involved C programs in DOS. DOS had no memory protection at all. Any program could access any address in memory (the whole 1mb). If you somehow messed up the null termination and then output a string, you would get some interesting results, as in, a bunch of binary garbage on the screen, usually with "MICROSOFT CORPORATION" fairly early in the garbage. Apparently that string was stored somewhere pretty close to the heap.

[@Chalisque]

_countof is a macro that is defined in VC but not e.g. GCC. It will also give erroneous results when given a pointer. That is, char buf[64]; char *p = buf; _countof(buf); // will return something sensible _countof(p); // will not, as then the sizeof() will return the size of a pointer, instead of the number of elements in the array. Another gotcha worth pointing out.

[@perwestermark8920]

I always have a NELEM() macro in my code since 25+ years. Sizeof array divided by size of initial element. But same problem if it's a pointer and not an array. Which is why the size should be computed by the buffer owner and then propagated. One guy did change a backup function that did disk transfers of sizeof(buf) bytes. His last optimization before shipping was to change from a stack buffer to a dynamic buffer. So one night at 3 AM I did a backup on a live system where each read/write was one pointer size big. That consumed most of the system update time I had available - 2 minutes became over an hour for the backup.

[@DavesGarage]

I always use #define ARRAYSIZE (sizeof(x)/sizeof(x[0]))

[@perwestermark8920]

@@DavesGarage Same macro as mine. Just different name. But very handy.

[@SauvikRoy]

I think _countof() would probably not compile for non-array types. Oh sorry, it's C++! ☺️

[@Chalisque]

​@@SauvikRoy sizeof(x) is always turned into a literal integer during compilation, so e.g. sizeof(x)/sizeof(x[0]) will turn into something like 4/1 an experiment is int x; char y = sizeof(x); // compiles fine vs. int x[1000]; char y = sizeof(x); // warns about overflow I had, prior to doing this kind of experiment, thought that sizeof(x) worked like a function with return type size_t (so would have expected char y = sizeof(x) to give a type error).

[@jonweinraub]

I like these educational videos a lot more than the bells and whistles of others. The speaking speed too is easier to understand. The code is concise and gave me aha moments. Now the challenge is to use it because the _s suffix is awkward.

[@ericerpelding2348]

I learned C years ago using Borland's Turbo C 2.0. It seems that I have to forget all that and learn the proper, modern way to program in C/C++.

[@CT-cx8yi]

I hope this is the start of a new playlist called "Stupid C Tricks", because I am looking forward to more videos like this.

[@lerssilarsson6414]

"Stupid C Tricks" But they are the essence of C. 😁

[@zyxzevn]

Did most programming with Pascal. Later Object Pascal. But also some other languages. These unsafe string problems were only in C and some C++

[@stomah9832]

using a buffer with a maximum size is not correct. you can make it “safe” but not correct. you should first calculate the size of the output buffer and then you don’t even need the safe functions because you know the output buffer is big enough

[@rasimbot]

2:28 No need to create the complete string. Substrings and space can be sent to cout in sequence

[@hrznn]

After the first few tries I quickly learned to never ever trust standard str functions. I always write my own that goes character by character. Reverse, concat, split, per word reverse, lowercase, uppercase, shortening, extending, inserting, number conversion, whatever. Although I try to optimize them, I know it's slower than the std ones, but at least I trust my own functions.

[@eazegpi]

Thanks for all these C tutorials and tricks!

[@Hauketal]

For all functions: check the return value. Most report errors there, even printf. Only if you have no way of handling them (like an error to fprinf(stderr, "...") during program exit), ignoring is all you can do.

[@Guru4hire]

I don't do coding except for personal interests, but if it were me (and what do I know, I am just some dude on the internet), I would pass a const string & into the function, construct a std::string_view front it, then use the find first of member function of the string_view class to find the token, use the substr function to pass the token to cout, and then use the remove prefix member function to move the front pointer in the string_view up to the postion after the delimiter. repeat until size or length = 0; Then we have to decide what to do with 0 length tokens, as there could be 2 delimiters in a row. The other assumption is that the string isn't destructed while the function is running.

[@laserspaceninja]

Bookmarking this for later. Thanks for the tips!

[@jacoblf]

I wish I had you for my CS prof.

[@Psychx_]

It's baffling that these functions have been rewritten dozens of times and yet were still unsafe.

[@ambuj.k]

One thing to note is that strcpy_s and strcat_s are windows only and will likely not work on Linux and glibc. Cross platform and backwards compatibility is the reason why you don't see those functions used anywhere.

[@SassyToll]

Thank You @Dave they was excellent and I learn a lot from this. Can you make more videos like this please?, especially for C and C++. Thank you John Ireland

[@hicknopunk]

I only ever used C in DOS. Also, I only had a free 16bit complier. The big challenge was trying to make a "game" which uses over 1MB of disk storage, while only using 64kb of ram and therefore a dynamic executable which would read procedure files and overwrite parts of the 64kb of ram. Most songs, which had to be interleaved with game code would take up 20kb of ram just to drive the PC speaker. If you didn't properly guess how long code took to execute the music would have halts in it. I don't know if I was doing things wrong or not as I was self taught and didn't use libraries. At least I could specify which memory addresses were overwritten to not overwrite DOS, TSRs and the core game code to overwrite itself in ram.

[@djrmarketing598]

Its unbelievable how many programs over the past 40+ years were written the old way and were mostly fine except when hackers are injecting unexpected code. I picked up my first book K&R ANSI C at the tender young age of 11 and read it cover to cover, did every sample program in the book. Really back in the day if you were working on combining a last name and first name with a comma into a string from sanitized user inputs, everything is fine. Even on the database side, if your field lengths are set properly (and back then when we were using DBase and other ISAM databases you had storage constraints so you wouldn't just arbitrarily have 512 byte name fields), reading and using this data was fairly safe. It's really when these inputs are coming from unknown places like API calls, web calls, etc. and honestly programmers SHOULD have been checking length and not relying on assumptions. But I guess when people make a "protocol" and specify the maximum length of the incoming URL in a GET request, and developers assume that the third party is following those requirements, things get really dicey. One thing I've learned is - if there's some way to externally interact with something, you make sure everything is bulletproof. I used to get paid to figure out how to break existing embedded systems for a small company that made vending systems and guess what people in the field were doing to cheat systems. And found MANY ways people would HACK systems from cloning NFC cards and simultaneously loading/using points, to using static charge devices to short out electronics.

[@flatfingertuning727]

Fundamentally, zero-terminated strings are a good format for a few common situations involving string literals, and are lousy for pretty much everything else. Zero-padded strings, which are what databases used, required that programmers keep track of the sizes of buffers they were using, and allow for the possibility of data precisely filling a buffer, but avoided leaking data (in languages that use fixed string buffers and don't use zero-padded strings, setting myThing to "SuperDuperSecret" and then setting it to "X" before writing it out to a disk record would write out to disk a string which, when read normally, would appear as "X", but which if inspected would also contain all but the first chracter or two of "SuperDuperSecret". Zero padding may be less efficient than letting unused bytes hold their values, but is better in situations where confidentiality may be important.

[@syth-1]

Just saw a video couple weeks ago on how to use PowerPoint for those amazing code transitions, wasn't expecting to see it so soon .-.

[@jimadams7765]

Yes, the misappropriate use of "inheritance" in "language design". In most cases, bad (or assumed) data definitions and abstractions in the evolution from Fortran to C and other derived or variant languages. In that sense, COBOL, with its explicit "Data Division", was always going to be a superior "language" mechanism.

[@billbez7465]

Thank you! Please do more videos on "can you spot the bug" theme.

[@annieworroll4373]

I wonder how many of these unsafe behaviors were compromises in the face of the hardware they had available, or they just didn't see where it would go wrong? Which isn't necessarily a slight, when you're advancing the state of the art of something, you're going to miss at least a couple small details.

[@jazzerbyte]

In the old days, function inputs were considered friendly. In the example, the original programmer was thinking in terms of "hello" and "world" and buffer sizes of 128; "no chance of an overflow". Today all function inputs have to be considered unfriendly, even in the core of the program where the inputs should have already been validated.

[@zzco]

I usually just let calloc fail and crash if it gets too big, because at that point, you've lost the plot anyway and your input is likely corrupt. Crashes are bad, but are better than corrupting memory.

[@BoloH.]

Last Crusade would've had a completely different ending if they had Indy choosing the correct C string function instead of the holy grail.

[@MichaelButlerC]

That book sounds interesting. Best of luck in your future!

[@KabukiStarship]

I just made a better C++ Unicode library called the Uniprinter, part of Script2 (Serial Chinese Room, Interprocess, and Telemetry Script). Also too it contains the world' fastest integer-to-string algorithm that I wrote called the Puff Alogirhm which uses some discrete math to eliminate over half the division instructions.

[@mcg6762]

There is no need for any division instructions when doing int to string even with the trivial algorithm. Division or modulus by a constant (10) is converted to multiplication by the compiler.

[@KabukiStarship]

@@mcg6762 I've written 5 different integer-to-string algorithms, and I've done it without division using advanced CPU optimizations, printing from both sides using out-of-order instructions, and that is 100% not true. The fastest way to print integers is to print them in powers of 2 using my bit-shifting trick to find the length of the string. Read my article "The Fastest Method to Print Integers and Floating-point Numbers" and you'll see there is a math trick.

[@ufufuawa401]

You can limit buffer on scanf function, just use max field width with the format

[@erenardakaplan7984]

Amazing video very informatic, thank you

[@theondono]

Then your colleague calls the function with the same pointer twice and demons come out of your nose!

[@mkukri]

strncpy is for writing to fixed length, nul padded, but non-NUL terminated string fields in structs. It really shouldn't have str in the name. The _s functions are Microsoft specific extensions that somehow got standardised under pressure for ms, but no Unix devs use them nor Unix oses support them. And strlcpy and strlcat are fine and serve the same purpose as the _s versions. Anyhow fixed size string buffers should be a thing of the past and user provided strings should be dynamically allocated.

[@firstnamelastname3389]

To be fair, I haven't touched anything C since college, but in my over decade of programming I've never heard str pronounced "stir", it's always just fully expand it to "string". Which I guess if you watch to the end he does do this.

[@jongeduard]

I don't know the exact current status, the main problem that I found at moments when coding in C with many of those _s versions was that they belonged to C++ standards (or even just MSVC? ) but not general C standards. Or at least that was according to documentation that I found. So I still ended up still using the older non _s versions, but combined with extremely well testing of my code. In order to keep things compatible with actual C standards. It was mostly MSVC recommending those en deprecating the older ones, but compatibility issues arose with other C compilers, so I tended to ignore it and make my own choices here. My solution for the situation shown at 1:45 with strncpy and strncat was often to simply manually place the correct zeros in the buffer or fill the entire buffer first with zeros using memset, although that costs a tiny bit of performance of course.

[@erikkonstas]

At least in the C17 standard (note C23 isn't finalized yet), the _s functions are there. What is also there, however, is the permission of a compiler to decide whether it wants to support them or not (it has to let you know via a macro), and many (such as gcc) don't.

[@gigachadkartik]

This is why I love C++

[@maciejlabanowicz8640]

First example: there final solution should be: char szStrOut[MAX_PATH + 1], also return statement from function ismissing.

[@johntitor.]

Counting sheep don't work for me, i'll use this tonight. Cheers.

[@DavesGarage]

Dream of me.

[@Alex-F.]

And that is a great example of why I don't use standard functions/libs in my (automotive) embedded software. They simply cannot be trusted...

[@MrRubikraft]

Very interesting, thanks.

[@RAndrewNeal]

The safer a function, the slower it is. More safety checks and condition handling means more computing time spent. In my opinion, one should write their code in a way that makes it impossible (barring the inevitable bugs to be fixed) to overflow memory (like has been done since the beginning, right?), rather than relying on the standard library functions to do so. Also, GCC doesn't have these MS functions.

[@chri-k]

i agree completely

[@danielrhouck]

Another bug in your first example: the function claims to return `size_t` but has no `return` statement. Most compilers say something here but for reasons I cannot fathom this is usually a warning not an error.

[@rasowa2958]

It all depends on context. If you need to process millions of strings in time-critical part of the code and their size is well defined, it still makes sense to use these unsafe functions which are just faster.

[@HenryLoenwind]

When you're on a time-critical path, it's better to not use any of those functions at all. Instead, use pstrings so you can just memcopy them, that's the fastest you'll get if you need to move them around at all.

[@rasowa2958]

@@HenryLoenwind That's a fair point. Although with short strings you may even do much better by copying them directly, (preferably using uint32_t or uint64_t pointers as to use full length of CPU registers) and not waste time on calling any functions.

[@HenryLoenwind]

@@rasowa2958 Yes, copying a pointer if you just want to copy the whole string is much faster. But for concatenating strings, this doesn't help unless you add support for a string pointer array to your code. But now we're getting deep into special-case optimisations. ;) I just wanted to point out that Pascal-type strings, especially the subtype that also includes buffer length (struct { int buffer_len, int str_len, char[] data }, excuse my pseudo-C) very nicely solve the buffer overrun, the "cannot contain 0x00", and the "most operations are somewhat slow as they need to check each and every byte for 0x00" problem in one fell swoop. And all that for the small cost of sizeof(int)x2 - sizeof(char) bytes of memory. (Which could be 3 bytes if 64k-long strings are enough for an application.)

[@joshuakb2]

This video strengthened my motivation to learn Rust

[@ptravers]

Aren't the *_s functions basically an msvc extension so practically non-portable? I've checked the code on compiler explorer and it only seems to compile without implicit declaration warnings with msvc.

[@chachaboy2]

"*_s" are MSVC extension functions. I don't get why he says it's part of the standard. No, it isn't part of the standard, unless you're talking about Microsoft's standard.

[@thatcreole9913]

More of this please!

[@lohphat]

I stop coding in the early 90s when I went into ops. I started with TRS-80 BASIC, the Z80 ASM in the 70s, then UCSD Pascal, then C in the 80s. Since then I'm starting to get back into coding. But SO MUCH has changed. Instead of just one standard lib, you've pointed out all the deprecated lib calls which have risen up and have fallen in the 30 years since I've stopped. Now, it seems you're not really coding unless you have an encyclopedic familiarity with all the available lib calls to do the heavy lifting since the libs are also optimized for CPU arch steps so that the lib call can take care of using esoteric opcodes if necessary and available. It seems to me that instead of "coding" the ideal software developer is "linking" pre-existing lib calls if code is to be optimized.

[@EricKolotyluk]

I love your videos... and this one validates my decision to abandon C/C++ in favour of Java, Scala, Kotlin, etc. If you still love C/C++, I would like to see you do a good video on Rust, particularly how to use Rust to get around the problems you fix in this video.

[@CatbertDeVil]

7:09 I was a little confused when I saw you using the word 'token' in the place of the arguments which is meant for 'delimiters'. The way I understood the term 'token' is that it refers to the extracted characters (the chunk), not the separators. Did I miss something?

[@Ryan-xq3kl]

2:33 disproving Linus Torvald's ridiculous claim that C++ hasn't fixed any of C's issues in a few seconds.