What is a semaphore? How do they work? (Example in C)

Patreon ➤ / jacobsorber
Courses ➤ jacobsorber.th...
Website ➤ www.jacobsorbe...
---
What is a semaphore? How do they work? (Example in C) // Semaphores cause a lot of confusion for students, largely because they are super flexible and a little weird. In this video, I try to provide some insight into how they work, and an example of where you could use semaphores to coordinate different processes.
Shared Memory Videos:
• How to Set up Shared M...
• Simple Shared Memory i...
Threads and Processes:
• Programming with Threads
• Programming with Proce...
***
Welcome! I post videos that help you learn to program and become a more confident software developer. I cover beginner-to-advanced systems topics ranging from network programming, threads, processes, operating systems, embedded systems and others. My goal is to help you get under-the-hood and better understand how computers work and how you can use them to become stronger students and more capable professional developers.
About me: I'm a computer scientist, electrical engineer, researcher, and teacher. I specialize in embedded systems, mobile computing, sensor networks, and the Internet of Things. I teach systems and networking courses at Clemson University, where I also lead the PERSIST research lab.
More about me and what I do:
www.jacobsorbe...
people.cs.clem...
persist.cs.clem...
To Support the Channel:
+ like, subscribe, spread the word
+ contribute via Patreon --- [ / jacobsorber ]
+ rep the channel with nerdy merch --- [teespring.com/...]
Source code is also available to Patreon supporters. --- [jsorber-youtub...]

[@vaggelismanousakis6147]

Bookmarked! If I had watched this video 6 months ago I would have passed the Operating Systems class. Thanks

[@playerguy2]

Oh, well. Better luck next time.

[@vaggelismanousakis6147]

@@playerguy2 thank you sir

[@JacobSorber]

Sorry I was late. Next time around will hopefully be better. Best of luck.

[@justinyork6006]

Literally just found this as I am trying to understand my OS assignment!

[@vaggelismanousakis6147]

@@justinyork6006 Keep working hard, you can do it! Same here

[@chadj1797]

This is exactly what I needed for my OS class assignment. Thanks Jacob, cheers from India. You're doing God's work!

[@DetectivePoofPoof]

Huh, that's cool, I always thought it sounded like some kind of civil engineering structure. "Oh we used to have _dreadful_ floods back in the day! But then the gu'vment came and they installed that 'ol semaphore up the creek and we've been as dry as a nun's gusset ever since!"

[@peepypoopy8917]

This is what I needed. Would be cool, if you were to make a video on Mutexes also and some sort of concrete comparison between the binary semaphore and a mutex. Thank you.

[@kylek.3689]

It's a bit late to respond but I can share anyways, In a Mutex, only the thread that originally locked it can unlock it, but with a semaphore, any thread can wait() or post(), regardless of who has called what.

[@dusan7777]

"Getting fancy with semaphores is a great way to end up with software that doesn't work" I laughed so much at that :D Great explanation tho, thanks a bunch

[@rickarmbruster8788]

but fixing it is kind of easy because you know directly that you caused trouble with 01010 sequences that just doesnt seem to run in the right order ^^

[@godlyradmehr2004]

This video was really helpful, to understand semaphore in my os course ❤❤

[@yonasmulissa5083]

Very helpful video. Thank you for making this. One little comment: To get all points across easily can you please slow down the pace at which you speak and take a brief pause in between important points?

[@m4l490n]

Be careful with semaphores, can turn things very confusing and bugs would be hard to find.

[@JacobSorber]

Amen.

[@u263a3]

Just because it’s there doesn’t mean you need to use it

[@shamanthakrishnakg1978]

Man !!!!, Damn!!!! ,you are doing a great help to the humanity, your videos are very helpful,thank you very much. 🙏🙏🙏

[@cambrown5777]

Using this to prep for Northwestern's OS midterm, didn't realize that Prof. Hester took a job at GT! Sad, but glad he's killing it

[@JacobSorber]

Yeah, it will be nice to have him closer to home. I'm sure he'll be missed up north. Good luck on your midterms.

[@raidtheferry]

Incredible video, thank you very much!

[@agustinlawtaro]

Muchas gracias, Jacob!!! Clarísimo.

[@LDS-Hellgineer]

This video saved my sanity in a current project. Thank you very much, keep it up!! greetings from Austria

[@anon_y_mousse]

I like the messages you send to the consumer. Used to hear that all the time as a kid.

[@thepickicool97]

thank you for saving my semester

[@cubedude76]

Thanks for the good video, but please get a filter to smooth out the s's they are very loud

[@JacobSorber]

I've added one since this video was recorded. Has your listening experience improved?

[@avineshwar]

Great resource. It would be great if you could shed some light on memory barriers (possibly with an code-level example) and why typically we should/not worry about.

[@unknown2.03]

Hi can you please tell me where this named semaphore has been stored after creation as mine. # cd /dev/shm directory isn't having my named semaphore.

[@miladgolfam49]

Matthew Mcconaughey is teaching C, I loved the this tutorial thanks.

[@bartekburmistrz8679]

Amnazing Tutorial, well explained, a very good example, helped me a lot. Thanks!

[@JacobSorber]

You're welcome. Glad it helped.

[@mrcrackerist]

semaphore sound interesting, but thanks to this video I am going to lookup atomic types more. Thanks :)

[@vidhyagopalan4581]

the internal implementation of wait seems like polling and would use CPU cycles rather than sleeping that is what wait() is for? for sleeping?

[@bhaveshvarma4313]

I wasted 2 hours to exactly understand but don't get difference b/w mutex and semaphore , But only one line makes clear that any thread can call post(Signal) where in mutex owned by thread only that thread can release it.

[@tete1445]

Thanks Jacob! I've been following your videos recently and they're all informative and amazing. I gotta say I'm quite intrigued! I have a few questions though; I'm learning C++ and I've gotten to an "advanced beginner level" and I wonder if I get to a decent level in C++, will the transition to C be easier? And what about Java, C#, python etc? Second question, can these programming concepts (such as Semaphores and whatnot) also apply to C++? Thank you!

[@JacobSorber]

Yes, you can (in most cases) think of C++ as just C with a set of language extensions. So, anything you can do in C, you can generally also do in C++. So, learning C++ will strengthen your C, and vice versa. And, any of my C examples, can be used in C++. You just have to keep in mind, what is supported in each language. How this relates to java, python, c#, etc, really requires a longer answer than I can give in a comment. Maybe in a future video.

[@tete1445]

@@JacobSorber Thanks, Jacob. I intend to transition to other languages once I feel unstoppable in C++ haha.

[@KiyotakaUchiha-f3p]

This video was so much important for learning about semaphore, thanks for Knowledge!!

[@thinktubers]

Hello sir, Thanks you for the video , I tried the same but when I try to run the program am getting error like sem_open/producer: no such file or directory Am I doing anything wrong , if anyone know how to solve this issue , please help me to solve this problem Thank you !

[@thinktubers]

Hello??

[@jesusalanlopezaldaran1663]

Hello! I had the following problem: "shm_open no such file or directory" I solved it by changing "IPC_CREAT" to "O_CREAT". To make this change you will need to import the library "#include " "O_CREAT" creates the semaphore if it does not exist. Here is the code snippet for the "readshmem.c" file ... //setup some semaphores sem_unlink(SEM_CONSUMER_FNAME); sem_unlink(SEM_PRODUCER_FNAME); sem_t *sem_prod = sem_open(SEM_PRODUCER_FNAME, O_CREAT, 0660, 0); if(sem_prod == SEM_FAILED){ dogr("sem_open/producer"); exit(EXIT_FAILURE); } sem_t *sem_cons = sem_open(SEM_CONSUMER_FNAME, O_CREAT, 0660, 1); if(sem_cons == SEM_FAILED){ dogr("sem_open/consumer"); exit(EXIT_FAILURE); } ...

[@Kumasan1986]

Finally I understand it xD Thank you!

[@yaroslavpanych2067]

Okay, 1st: in what world barries is anyhow part of IPC? Should I listen to the rest, or it is adequate?

[@mrunalsanghvi]

the whole thread playlist is awesome. great learning. but please add more videos to make the threadpool more robust whenever you get time. I am feeling that i kind of missing on something there. especially handling the slow reading client or a thread pool with EPOLL could also be a great thing to learn and apply, i feel. big thumbs up for all the effort. Cheers!!.

[@olegkuzenko]

Thank you. Great video!

[@intheshell35ify]

I have to google ever other word and splice the definitions together to try to understand what the hell he just said. Might need training wheels. And a beer.

[@eddaly4160]

Great video. As usual. Seems like there could easily be a case were I can post repeatedly with out a wait happening because my "consumer" is busy or unresposive...do I need to guard against an overflow on a semaphore...if so, is there a proper way to deal with this scenario?

[@CheesyAceGameplay]

I use semaphores for when I'm writing to a queue in one or multiple threads... and another thread is dequeing (removing) items from the queue.

[@jonathanluo0]

In my ubuntu 18.04.2 system, in semaphore reader, "sem_t *sem_prod = sem_open(SEM_PRODUCER_FNAME, IPC_CREAT, 0660, 0)" failed, but "sem_t *sem_prod = sem_open(SEM_PRODUCER_FNAME, O_CREAT, 0660, 0)" succeeded

[@sidneysherrill4870]

I'm experiencing the same thing

[@maryjonson9692]

Yeah, changing IPC_CREAT with O_CREAT solved the issues for me !

[@konulaliyevaa]

Thanks for the awesome video.

[@demonking1101]

wait this guy is managing memory using c language 🙄🙄🙄whoa

[@gabrielkolletalves493]

2:20 strange... you're presenting the code as if it's a kind of busy waiting (thread locks in while loop untill the variable is incremented by another thread) - but semaphores are not a kind of busy waiting. I'm having a hard time trying to differenciate that from a simple "N" variable who counts the buffer size.

[@carl-johanlandin1419]

Not sure why but I fail to run this code on my machine. It seems it exits here at this segment: ``` sem_t *sem_prod = sem_open(SEM_PRODUCER_FNAME, IPC_CREAT, 0660, 0); if (sem_prod == SEM_FAILED) { perror("sem_open/producer"); exit(EXIT_FAILURE); } ``` And the terminal output looks like this: ``` ❯ ./readshmem.elf sem_open/producer: No such file or directory ``` Any ideas why? I'm running Linux Debian and have tried to compile with both clang and gcc. I noticed you didn't need to link to pthreads library in your makefile which I had to do for a build to work.

[@carl-johanlandin1419]

It seems the directory /dev/sem did not exist on my OS, creating that folder solved the issue. Really like your videos btw! The best C content on youtube.

[@马哈一二]

Thanks, it's very helpful,

[@caesar104]

Could you make videos about library and its functions and also Annex K functions ? Are these functions necessary? Btw your videos are awesome and very informative. I learned more than my college lessons :) Thank you very much for your great effort sir ,please keep going :)

[@abhisheksa6635]

Why not discuss all other stuffs like spin lock, barriers, monitors and usage case basis, will be a great add.

[@dieTbshow]

Can somebody explain to me what exactly the parameters for sem_open(...) express?

[@benjaminmatanovic748]

Thank you so much!

[@ankan2088]

matthew mcconaughey on chemo

[@thanakornpasangthien5590]

Please do OS videos

[@xplozia5276]

my girlfriend would like to know about mutex please, thanks

[@JacobSorber]

Tell her I'll see what I can do.

[@rabiraju9414]

pipes please

[@huyvole9724]

bad

[@thangbui6383]

Thanks for the explanation, your video is much easier to understand than grinding at my college's book.

[@abdelrhmanahmed1378]

should it be convenient to use cv(conditional variables) in consumer producer case ?

[@af9466]

Thank you very much! This really helped me understand semaphores and apply them in my Unix lab work.

[@sumaiyatasnim102]

Very helpful. Is there any way to manipulate times (like arrival time or wait time in FCFS order of something) using semaphore and multiple threads.

[@lewiscole5193]

FWIW, the "semaphore" that Professor Sorber describes is not the one that Dijkstra defined in 1968 and then slightly extended in 1969, at least based on the pseudo-code Professor Sorber gives for the two primitive operators "wait" (what Dijkstra called "P") and "post" or "signal" (what Dijkstra called "V"). The semaphore that Professor Sorber describes has been called a "weak semaphore" because it cannot guarantee that process starvation cannot occur if it is used. See "Semaphore Primitives and Starvation-free Mutual Exclusion" by Eugene Stark in the October 1982 issue of the Journal of the Association for Computing Machinery, page 1049 thruough 1073.

[@00killerix]

Which sw are u using for your videos?

[@socat9311]

If the Dark Knight was a C expert

[@anelemadonda6191]

fanks a lot Mr Sorber

[@MazharKhan-jm7pj]

Hi Jacob, Thanks a lot! your videos are very informative and helpful. I have a request, can you make some videos on Operating system basic concepts like what are user and kernel modes? What are system calls and what is a Linux boot up process etc.

[@benjaminshinar9509]

can you do one about microsoft zune?

[@kakerlakenmilch1155]

I love this stuff

[@techtechlearn3438]

Great video !!!!!!!!!!

[@subhasishnathdev]

Thanks a lot.

[@othmanekaramat1962]

Insane, Thank you !

[@migtvill]

Can someone here link non-youtube studies of atomic studies of semaphores outside of programming, as in physical or physics applications?🎉

[@kartikeykushwah3926]

Really needed to know!

[@l0_0l45]

Please make a video on lock free programming! Cpp con has a talk on it but it was way too difficult to understand!

[@lewiscole5193]

Well, if he does do a vide on lock free programming, I would suggest that he might as well deal with wait free programming either in the same video or in it's own.

[@cricket9282]

hey! im not entirely sure where to ask you questions like this and i apologize for the off topic-ness, but, how do you feel about nim?

[@jaminyah]

Hi Professor! I noticed that in the following code using a named semaphore: ```code int main(int argc, char* argv[]) { sem_t *sem = sem_open("/ex_sem", O_CREAT, O_RDWR, 1); if (sem == SEM_FAILED) { printf("Failed to acquire semaphore "); printf("%d ", errno); return -1; } ... return 0; ``` On the second run of this code segment sem is assigned SEM_FAILED. Is there a way to check for an error and still have the code run any number of times after it is compiled? Without error checking there are no issues. Thanks.

[@sanketgandhi3139]

Thank you for such nice video. One small request if possible please make videos on mutex locks and condition variables

[@honey-py9pj]

when we write in the producer function : ```sem_wait(sem_cons) ```, is it because we don't want to write anything , until a reader is avaible to read it (consume it?) ?

[@ShravanBandaru]

Can we expect a separate video on Mutex???

[@JacobSorber]

I sort of already have one. Check out my thread safety video. It talks a bit about mutex locks. I'm happy to add another just mutex video, if you think it would be helpful.

[@SteaM992]

Hello. Thank you for sharing your knowledge with us. Could you recommend a book or 2 that goes deeper into the subject of semaphores, memory allocation in unix like OS (mmap), file descriptors, etc. Thank you!

[@raymondshawn3864]

Check Linux programming interface

[@dduartez99]

you literally saved me, I have a project due on the 2nd of sept on System V shared mem and semaphores and didn't know a thing

[@JacobSorber]

Glad I could help.

[@dibbyabarua9499]

Very good description of semaphores... very clear and concise! Could you suggest me a good book for Operating Systems? I am using two books right now: (1) OS by William Stallings and (2) OS Principles and Practice by Thomas Anderson and Mike Dahlin. I like the one by Anderson more than the Stallings one so far but I was wondering if there is any better suggestion?

[@JacobSorber]

I use Anderson & Dahlin in my class. It's probably my favorite, at the moment.

[@dibbyabarua9499]

@@JacobSorber ya. I like that one too haha

[@GEhehloopf]

Thanks for the video! I was wondering if something can be done using mutexes, can semaphores be used as an alternative for every case that requires a mutex? What's the difference between a mutex and a semaphore? Could you make a video about the difference? Thanks again!

[@adityakadam210]

4:29 exactly explains the difference. Semaphore can always be used as a binary semaphore which will act as a mutex lock the only difference being that a mutex lock once acquired can only be released by that particular thread which makes perfect sense whereas a semaphore once wait() is called can be incremented back to 1 with post() call by any other tread. This of course depends on the way your code works so if you properly implement binary semaphores there wont be an issue but to be on the safer side using a mutex lock when it can be is a better option

Hello! I was able to make this code work in my mac, but when I try to run in a ubuntu machine, I cant. I added the flag -lpthread which allowed me to compile, but after that, I get the error "sem/open/producer: No such file or directory". What could it be?

[@vapula87]

Hey Éverton! Not sure if you ever figured it out, but semaphores on Linux are implemented slightly differently.

[@shobhitmishra92]

If possible it would be utmost helpful to see video just like semaphores(with an example) for inter-processes synchronization with mutex and monitors

[@shobhitmishra92]

I was trying your code and while compilation it started throwing undefined reference error for the sem functions, I am using a gcc compiler on ubuntu distro , i after adding the -pthread with -lrt flags it started to work

👍👍👍

[@artemiocabrillosjr.244]

Well explained. Thanks !

[@JacobSorber]

Welcome.

[@rothenbergt]

Clean code

[@DipietroGuido]

Wonderful explanation. Everything makes sense now!

[@jesusalanlopezaldaran1663]

Hello everybody! I had the following problem: "shm_open no such file or directory" I solved it by changing "IPC_CREAT" to "O_CREAT". To make this change you will need to import the library "#include " "O_CREAT" creates the semaphore if it does not exist. Here is the code snippet for the "readshmem.c" file ... //setup some semaphores sem_unlink(SEM_CONSUMER_FNAME); sem_unlink(SEM_PRODUCER_FNAME); sem_t *sem_prod = sem_open(SEM_PRODUCER_FNAME, O_CREAT, 0660, 0); if(sem_prod == SEM_FAILED){ dogr("sem_open/producer"); exit(EXIT_FAILURE); } sem_t *sem_cons = sem_open(SEM_CONSUMER_FNAME, O_CREAT, 0660, 1); if(sem_cons == SEM_FAILED){ dogr("sem_open/consumer"); exit(EXIT_FAILURE); } ...

[@tktktktktk826]

Wouldn't it be easier to use 1 mutex in the example shown? Or am I missing something?

[@marios2liquid]

I had an issue recently were i was compiling many files and linking them together that had globals in them and externed all over the place with a global .h file. The globals were also volatiles. During runtime I could see that some variables were not correctly initialized as they were in the source. Compiling with different versions of GCC (>6) fixed this issue but introduced other issues. My gut says that the compiler running at -Os was basically optimizing these variables out (but not triggering any warning with -Wall, -pedantic since they were used in other files). I ended up converting all the globals to statics in each file with API access functions if another file wanted to access them or edit them.. That solved the issue.. I searched countless hours about translation units, shadow variables, or any other things that could happen. Couldnt find any. Also tried removing -Os but since this project was embedded it was introducing other issues (speed of execution and interrupts was messed up and the binary size was too big). I was able to also compile this project to x86 in addition to ARM and this bug wasnt present, sigh.. Compilers that had issues were GCC 4.8-6.

[@JacobSorber]

That sounds rough. Glad you got it sorted out.

[@georgioziozas9031]

great video , bookmarked. waiting for a video which uses processes and semaphores , a full example ! gj

[@abdesamiesalama9408]

Hi .. _check(my_input,(int)_Env); Could you tell me what the second param means ?

[@sourabhkulkarni3124]

The concept of Semaphore is cleared from your video. But your code involving Consumer and Producer is not clarified. Please let me know. Thank you

[@sd_1989]

Wonderful explanation. Thank you for your time.

[@marcel151]

2 months earlier and I would have a better degree in my exam about operating systems. Thanks for that video.

[@jnr2349]

In spanish we call traffic lights "semáforos" which is helpful in thinking about them.

[@socat9311]

Bit irrelevant q: what specs and software you have for the screen capture? Quality is pretty good. Trying to do it from my laptop and ends up rubbish...

[@NajmaMiah]

Can you do a video on System V semaphores please? Without mutex

[@aravinds123]

The best video on semaphores! Wonderful!

[@prshntpnd00]

Hi Jacob , Can you make a video explaining about zero-copy buffer .. heard this term a lot .. tried some articles but still not clear . Sorry about my youtube handle name , I am Raj Niranjan (India) Tried to ping you on instagram .. was unable to reach you on LinkedIn

[@ReD4eva94]

Amazing. Best instructor on youtube.

[@justusmzb7441]

I have to wonder: Our professor presented semaphores as if they did not contain a busy wait. But your presentation clearly suggests that they do. Would it be possible eg. for the wait() function to put the process into a ready state so that it can try again when it is next dispatched, and the core it would elsewise be used for a busy wait is free for use?

[@Sir.NathanDrake]

Nice video. Thanks for the clear explaination!

[@friction5001]

this was very helpful and well explained cheers mate

[@shivaprasad4379]

Please post more videos on other IPC mechanisms and Synchronization concepts, bottlenecks & downsides around that code in production environment

[@khoaang5712]

Thank you, your video is very helpful

[@miserablepile]

Thank you for your very clear explanation!