Split C strings into tokens with strtok.

[@BABABOOWIE369]

I was aware of strtok and used it once in passing, but I never knew it was destructive or not safe for threaded programs. I guess I got lucky that one time, haha. Good video!

[@adammontgomery7980]

Good vid. I used strtok to parse a csv file and output it to a table in markdown syntax. Very handy!

[@matheusmaaia]

Hey man, can you help me with this table?? i need to do the same thing. Already parsed the csv file into an ADT List now im having troubble to print it in formated table.

[@garyadamos]

lol im doing that rn for my homework xD

[@JustinZaf]

Thanks Alot sir I'll keep watching your videos to learn every thing that new to me I didn't know about this function as well thanks again

[@Mr_ToR]

thanks a lot for this small video. this was exactly what I needed. I find strtok to be very counterintuitive though.

[@mixedmashup778]

Thanks sir Helpful video.

[@JacobSorber]

You're welcome.

[@Hevletica]

Thanks

[@oliilo1]

Why is the strtok inside the while loop pointed at NULL? I can't wrap my head around why it would't be at input. Does the strtok remember what variable it worked with last?

[@JacobSorber]

Exactly. Passing NULL just means, keep working on the last string you were working on. It stores a pointer to that last string in a static local variable. That's actually, why it's not safe to use in a multithreaded program. See this video for two alternatives. kzbin.info/www/bejne/mXTHn2p5rpuHnKc

[@ze2user318]

Clear to understand. Thanks

[@danieldixon5944]

You are a good teacher. Hve you written any books that show alternatives to parsing strings?

[@JacobSorber]

Thanks. I haven't written any books. Maybe someday.

[@HenrryWith2Rs]

excellent voice

[@ssaguiar]

Good morning sir. Excelent video. I wish to know what is the name and configuration of the ide you used to make this video. I supose it's Atom or Visual Studio Code, but I also wish to know the steps to have this configured as I found it to be very usefull for learning and debug applications. Thank you very much and, again, congatulations for the video.

[@JacobSorber]

Thanks, Sergio. I generally use Atom with minor changes to the defaults (I use a terminal plugin, for example). I like VS Code, as well. I think I talk a bit about in this video, as well (kzbin.info/www/bejne/hqLJZ3tnjLiVm7s). Let me know if there are specific things/behaviors you aren't sure how to replicate.

[@ssaguiar]

Thank you very much. I will ty and get back to you if I there's something I don't get. Congratulaions for the very good videos and for your help.

[@dstange91]

Hello Jacob! This might be a longshot but im going to try either way. I seem to have some trouble understanding when to use the '*' sign in a while loop. Ive just completed a section of my studies which included searching an string for characters. An example is removing ' ' at the end of the string. To solve this i created a pointer (char *arrPTR = string) and placed it in a loop. while(*arrPTR != NULL) -> if(*arrPTR == ' ') then *arrPTR = '\0' if not ->arrPTR++. This was the gist of it. But when i started using strtok i saw that the we dont use the value, i.e *arrPTR, but the adress. while(arrPTR !=NULL), why is that? I thought i understood how this worked, and for me the last example does not make sense, cause arrPTR is just an adress right? So why do we compare an adress to NULL when using strtok and not at other times? Hope this makes some sense, its kinda hard to explain via the comments.

[@JacobSorber]

It is tough sometimes to discuss code in comments, but I think I'm getting the gist of your question. I'm guessing your main issue is that you're not understanding pointers and their relationship arrays (or strings in this case, which are just arrays of characters). The following videos might help. kzbin.info/www/bejne/q4WQin97fdyJiZY kzbin.info/www/bejne/oWeohJ5vr8aZfq8 (not quite as relevant, but maybe) Also, planning to do a beginner video on strings soon (just need to find time). That one might help, too.

[@K0D0R0]

What if my string looks like "str,str,str,,str,,,str,str" and I want to parse it so that it would display maybe a '0' or '\0' character where consecutive delimiters occur?

[@JacobSorber]

Then you should use strsep. strtok will ignore the empty tokens, but strsep will give you empty strings, and it's easier to use.

[@mnarjb7259]

if I'm scanning a word from the user, how it will be?

[@Balawi28]

Thanks!

[@eceblog7454]

Thanks man I have the following string could you assist me to split it . The input as following:- +CMGR: "REC READ","12345",,"26/05/21,21:26:48+18"hello-world OK The out out should be as following:- 12345 26/05/21 21:26:48 hello-world

[@jamesstark4136]

I realize this was 5 months ago, but I would suggest you look at the man pages for this, there is an example of nested for loops. Not very fast but effective.

[@eceblog7454]

@@jamesstark4136 thanks

[@embeddedbastler6406]

In the Linux source code of string.c it says, that strtok() has been replaced by strsep() and there is no definition of strtok() anymore. But I can still use strtok() in my code, what does this mean?

[@JacobSorber]

It's defined somewhere in one of the headers that you're including. The world has definitely been moving toward strsep and other options.

[@LeeWalton_1999]

is it possible to use this to split multiple key value pairs, so a nested delimier. e.g. speed:100 distance:10 status:active so split key vaue pairs by then split each pair to extract the key and value

[@JacobSorber]

It is possible, but it requires care. strtok saves its state between tokens in a static local variable, so you would need to finish the initial pass (the lines) and then parse out key and value in the next pass. Alternatively, you could use strtok_r or strsep, which both give you more options.

[@rustycherkas8229]

If you are certain about your data, you can do this in one pass... char *pairs = "speed=100 distance=10 status=active "; bool bKV = true; // true for Key, false for value char *thisKey = NULL; for( char *cp = pairs; ( cp = strtok( cp, "= " ) ) != NULL; cp = NULL ) { if( bKV ) thisKey = cp; // store this else printf( "Key: %s, Val %s ", thisKey, cp ); bKV = !bKV; }

[@alexantollin564]

why no one is showing split string with all occurences of sub-string and return the tokens to double pointer char**, this function is more important than strtok.

[@barbiani]

Why is "i" changing?

[@275phuongvy]

Hi Sir, I do the same with your code, but my input file is 10, 20, 30, 40 When I print out the output, it just show 20, 30 ,40 which lost of the first element. Why I got that trouble?

[@JacobSorber]

It's hard to say without seeing your code. If you post the code (at least the relevant part), I might be able to give more helpful feedback. Also, I recommend stepping through with GDB or your favorite debugger. Also, I provide some more parsing examples in this video (kzbin.info/www/bejne/mXTHn2p5rpuHnKc), which might help. Best of luck.

[@m0n3yman23]

If I wanted to print the total amount of token how would I do that?

[@JacobSorber]

Declare an integer. Set it to zero. Then just iterate through the tokens, but increment the counter rather than printing out the tokens. When you're done, you'll have the number of tokens.

[@unperrier5998]

Note: strtok wouldn't work on static strings (not writable ones)

[@Ian-vn9fh]

Incredible video, gotta be one of the easiest people to understand. Thanks

[@Juanbaez_]

What is this editor?

[@JacobSorber]

atom

[@nikyabodigital]

delimiting with space doesn't work. Why?

[@JacobSorber]

Delimiting with spaces should work (works for me). I'm guessing the issue is something else in your program.

[@aaronbraun841]

Damn Jacob... them patron bottles. C got you stressed?

[@JacobSorber]

😂😂😂 Definitely... Actually, I don't drink, but they make good vases.

[@mainaksanyal423]

how to return just the second word?? it prirnts all the wwords.. i want to return just luke!! what i have to do then??

[@JacobSorber]

strtok works sequentially. You can write code to ignore some of the words if you want, but you can't get strtok to just give you the second word. It's just not how it works.

[@mainaksanyal423]

thanks a lot sir!! sorry for the dumbest question..

[@mainaksanyal423]

sir i was trying a problem based on strtok . here is code.. can u just point out my errors.. i would be greatful.. void printWordsInLines(char* str) { const char s[2]=" "; char *token; token=strtok(str,s); while(token!=NULL) { token=strtok(NULL,s); } return token; } int main() { char arr[] = "Hello WoRld"; printWordsInLines(arr); // This should print the words "Hello" and "WoRld" in separate lines char arr[] = "India is My Country"; printWordsInLines(arr); // This should print the words "India", "is", "My" and "Country" in separate lines }

[@JacobSorber]

A few things. You're not printing anything. "arr" is doubly defined. And, you're returning a char * from a void function. Might be other issues, but those are the ones that I'm noticing first.

[@M6MDR]

You would probably need to store each token (word) as it is found in something like an array of tokens. Then you with each token stored in its own location, you could print out whatever token (word) you like by knowing it's index in the array. So for example, Token 1 would be stored in array index 0 and token 2 would be stored in array index 1 and so on. At least that's how I'd start....

[@iamthejustin7420]

Can anyone explain to me what the below for loop does and how it works please? void tokenize(char *line, char **words, int *nwords) { *nwords=1; for(words[0]=strtok(line," \t "); (*nwords

[@JacobSorber]

Looks like someone's getting fancy with for loops. for(initializer; condition; increment) {body} The initializer calls strtok to get the first token. The "condition" says loop as long as there's room in the array (presumably MAX_WORDS is the size of the array) AND (&&) the calls to strtok keep returning non-NULL results. The incrementer just increments the number of words/tokens seen so far. And, there's no need for a body, because they did all the work in the initializer, condition, and incrementer. Not the most readable way to do this and using a global variable/macro to define the array size seems fragile, but otherwise, this code seems like it should work.

[@danieldixon5944]

How do I delete a token?

[@JacobSorber]

When you say "delete" a token, I'm assuming you mean deallocate the memory that holds the token. If that's not the case, the rest of this answer may not be helpful. When you're using strtok, you probably don't want to delete an individual token. The reason is that strtok breaks up the tokens, in place, by inserting null characters where the delimiters were. The token pointers returned are just pointers into a part of the original input string. So, trying to free one of those pointers will end badly (probably cause an allocator assertion to fail), because the pointer points into the middle of a block, which may or may not even be on the heap. If the input string was allocated on the heap, you can free that block of memory (which will free all of the tokens) once you're done with them.

[@rustycherkas8229]

char *str = "123,456,789,abc,def"; for( char *cp = str; ( cp = strtok( cp, "," ) ) != NULL; cp = NULL ) printf( "%s ", cp ); Piece of cake... DO NOT expect to use strtok to chop-up CSV files.. 1) strtok will NOT return an empty string for two adjacent delimiters. 2) CSV fields may contain 'embedded' commas protected by double quotes.

[@manojuniversity8044]

Too fast

[@JacobSorber]

Sorry about that. These videos are often made in a hurry. Fortunately, with video you can pause it and rewatch the parts that are moving too fast.

[@axlslak]

noooooo. don't parse strings with strok. if you do... ohhh boy.... poor you.

[@ubiquitous9173]

why

[@michaelboss7968]

wow what's next, you between two purple bath towels in the bathroom ? just teach and stop the stupid theatrics with all the different rooms and backgrounds , it's wrong ok?