How to Set up Shared Memory in Your Linux and MacOS Programs. (shmget, shmat, shmdt, shmctl, ftok)

Patreon ➤ / jacobsorber
Courses ➤ jacobsorber.th...
Website ➤ www.jacobsorbe...
---
How to Set up Shared Memory on Linux and MacOS. (shmget, shmat, shmdt, shmctl, ftok) // This tutorial shows you how to set up shared memory between two or more process in any Unix-style OS, using ftok, shmget, shmat, shmdt, ahd shmctl. Implementation example included.
Related Videos:
Shared Memory with mmap: • Simple Shared Memory i...
How programs get memory: • How processes get more...
More memory: • Keeping strcpy and mem...
Make: • Learn make in 60 seconds.
Cloning processes with fork: • Creating new processes...
Bit Fields: • Bit Fields in C. What ...
***
Welcome! I post videos that help you learn to program and become a more confident software developer. I cover beginner-to-advanced systems topics ranging from network programming, threads, processes, operating systems, embedded systems and others. My goal is to help you get under-the-hood and better understand how computers work and how you can use them to become stronger students and more capable professional developers.
About me: I'm a computer scientist, electrical engineer, researcher, and teacher. I specialize in embedded systems, mobile computing, sensor networks, and the Internet of Things. I teach systems and networking courses at Clemson University, where I also lead the PERSIST research lab.
More about me and what I do:
www.jacobsorbe...
people.cs.clem...
persist.cs.clem...
To Support the Channel:
+ like, subscribe, spread the word
+ contribute via Patreon --- [ / jacobsorber ]
+ rep the channel with nerdy merch --- [teespring.com/...]
Source code is also available to Patreon supporters. --- [jsorber-youtub...]

[@jojodi]

Prior to the C89 standard, identifiers had a max length of 6. That's probably a big part of the function names.

[@JacobSorber]

Good point. I sometimes think I have repressed those memories. :)

[@しげお-i1l]

Thanks for the awesome content, very well produced. I'm literally binge watching the entire channel haha

[@dkkogmaw1311]

Also a posix approach aka with shm_open etc would be cool

[@artemiocabrillosjr.244]

Hi Jacob, Good job in making these topics more simplier. I am a computer science instructor just like you and I highly recommend your videos to my students. Keep it up!

[@JacobSorber]

Thanks, Artemio. I appreciate the support. Let me know if there are topics that you think your students would specifically benefit from.

[@rajendrasinghnegi1764]

​@@JacobSorbermake a full c language course videos from beginning to advance level

[@belesiu]

Is shared memory actually faster than cached disk I/O? Cached I/O, especially if it's memory inside the L1 or L2 cache, seems to be as fast as it gets, right?

[@Nick-lx4fo]

What tfok!!??

[@natiiim6336]

in my operative system course they call this 'message queues' and for me, it's so difficult to grasp. They ask to solve problems with threads synchronization first with semaphores and then do the same but with messsage queues.... using this like blocking access tools! AAAAH

[@shrek1412]

Hi Jacob, For ftok system call, the man page written "Today, proj_id is an int, but still only 8 bits are used. Typical usage has an ASCII character proj_id, that is why the behavior is said to be undefined when proj_id is zero", but in the example you put the 0 there, I tried several alternatives, it seems fine with 0 or nonzero. Do you know why? Thank you

[@marcello4258]

great as always.. but instead of saying unix based you might consider the term POSIX :) ..may linux folks might argue otherwise :P

[@jaydenritchie1992]

imagine if you could share memory in the kernal and have daul or more internet connections, one std ethernet and this hotspot at the same time and you could direct your packets rx tx, i wonder what would happen if i could power the kernal down and still have windows running and then make a system image

[@randomman5280]

Very useful. Thanks

[@JacobSorber]

You are welcome.

[@rcmarco3281]

way better than my Chinese tutor

[@gregoryfenn1462]

How do you use shared memory on bare-metal applications (e.g. on an ARM Cortex-A15 core)?

[@hasanalattar9561]

hmmmm, @7:33 are you comparing result to casted char * of (-1) ?

[@hasanalattar9561]

shouldnt it be: if (*result == IPC_RESULT_ERROR) ? ..

[@oscarchivas900]

Hi Jacob. Thanks for the content. Would this same concept work to share memory between docker containers?

[@goksanisil9107]

What is the idea behind using a FILENAME and ftok to generate a key for the shmget? Why not hardcode a number that both reader and writer agree on?

[@smrtfasizmu6161]

I coded along while watching the video and I just wrote #define SEMAPHOR_PRODUCER "seemaphor producer" And did the same thing for semaphor consumer as well.

[@monzurulislam3860]

Hi Jacob, thanks for the Awsome tutorial. I have a question about the file that we associate with shared memory. If this file(name) is one of the process (executable file) itself, will that may cause any issues??

[@florianunterfrauner2041]

Can a bool function return NULL in c? 9:08 (line 52)

[@marbens]

no

[@Ty1er]

thanks a lot.

[@JacobSorber]

You're welcome! Glad you enjoyed it.

[@obeid_s]

i came from Java , No unsigned byte , No accessing Shared Memory, Going to C++. Thank you.

[@zxuiji]

I'm guessing they were supposed to be pronounced "shared get", "shared ctrl" etc

[@mihirluthra7762]

Great video, would really like to understand semaphores in next.

[@JacobSorber]

Thanks. I'm not 100% when the semaphore video will come out yet, but it's coming.

[@randomscribblings]

Maybe I'm an old UN*X hand, but the promise of unified VM/filesystem is that two programs mmaping the _same_ file effectively shared memory.

[@randomscribblings]

In a way, I would call this "how to play with SYSV shared memory"

[@funkykong9001]

I'm loving the new layout and graphic highlighting! This was much easier to follow.

[@JacobSorber]

Glad you like it! Always trying make things a bit easier.

[@jackeown]

Just an idea of something I'd love to see: make a video about sharing memory but between Python and C.

[@JacobSorber]

That would be fun. It would probably be basically the same mechanism, with some sort of python bindings on it. I'll see what I can do.

[@patricknagel4649]

@@JacobSorber Thanks for the great videos. Does one already exist that covers the request of @John McKeown?

[@sam777flynn3]

Mr Sorber, You are really doing a great job ❕👍🏻 Thanks You 🙏🏻

[@ramanujaamit]

In destroy memory block function you wrote get_shared_block(filename,0); Shouldn't it be get_shared_block(filename,BLOCK_SIZE); Am I wrong?

[@fisyr]

This is definitely very interesting, but how does one prevent memory leaks in case the program crashes? Does the linux kernel know how to free memory if none of the programs that interacted with it are running or does that block of memory stay inaccessible until reboot?

[@JacobSorber]

You're right. This can get a bit tricky. The memory needs to be explicitly removed with shmctl, so if the program dies, you will have a leak unless you have another process that can clean things up. Depending on the application, it is often possible for one of the involved processes to detect that a process crashed, and make sure that appropriate cleanup happens.

[@JoeKenyon]

A video on FFT/signal_processing or bluetooth would be cool.

[@JacobSorber]

Thanks. I'll see what I can do.

[@parker7721]

Loved the video! But I feel there's so much to explain here that a video might not always be enough, so for those seeking for more resources to understand shared memory (or system v ipc) like me, check the "Linux programming interface" book. Chapter are self contained so there's no need for reading the entire book if you have some prior knowledge.

[@ChristopherBruns-o7o]

Also a snapshot of Make file in key moments would be amazing!

[@nickhyland9497]

Pronunciation of “shm” functions is what’s made this video

[@Jose-tw9bl]

I find your videos incredibly well explained. The drawings and explanations in the begining really help before getting to see actual code.

[@yassinesafraoui]

shmdt, Shared Memory DeTach? duh?

[@johanronkko4494]

Hi! Love your videos. I would love a video of how to setup a testing environment for C. Is there a testing library for C and how do I set it up? I've heard that some people use C++'s Boost library to test their C code, but I'm not sure if that's the way to go...

[@flanker53]

Will this work in android ndk ?

[@10e999]

Great video Jacob. To continue in the same subject, I'd like to see some example where share memory is useful. Also, are multi process queue implemented with shared memory ?

[@j_mkg]

very difficult format for compiling as I dont know how to break down your method to compile alternatively - will check out your compile video **make: *** No rule to make target 'writer.elf', needed by 'all'. Stop.**

[@nagesha5539]

Ok , but how to implement that shared memory functions (shmget()) without using existing header files, I mean implement that header files by your own sir

[@pierce8308]

It just doesnt feel like in the spirit of "Shared memory". Like all this in effect is doing is allowing shared access of a file that MUST exist on a disk. Although I rewatched the video and the part where you describe that I must not confuse them with mmaping content of a file but from what I get is shmget() is basically doing the same thing as mmap() but guaranteeing that contents of file are not changed. Shared memory in essence or what I would normally expect is one process being the host of the memory and other just have access to it like it normally would. For example the earlier tutorials about shared memory for parent-child fit the definition of what is expected. But I think in this shmget method we are introducing uncessary mediators like the file that must access on the disk, and then the particiapting process would intearct with it like they would with a file (but without changing its contents). Seems like a very old and inconvineint shared memory api, because in theory, existence of a mediator (like a file) shouldnt be involved at all during shared memory. It should just work the same as they were in the parent-child scenario I would love to hear your thoughts on it and if i said something stupid.

[@LDS-Hellgineer]

This video saved my soul in a current project. Thank you very much, keep it up!! greetings from Austria

[@maverick7615]

It may sound dumb but if we are not using a common file as a means of sharing memory between processes, why do we need to specify a correct pathname of a file that must exist on disk?

[@rainfallen1064]

What’s the difference of using those methods versus using fopen(), and writing and reading from a regular file?

[@fennexagent8584]

however this doesn't work from a sandboxed app on mac (shmget gives "operation not permitted").

[@awaisansari3430]

What is the logic behind separating the creation of a shared memory segment and getting a pointer to that memory segment?

[@contactdi8426]

That's an awesome explanation! You made is so much easier to understand this :) Thanks a lot Jacob

[@hossein_haeri]

In the future, you may want to use a De-Esser on your audio file.

[@darshansrinivas6883]

Thank you for the explaination !!

[@ravisinnarkar9706]

Hi jacob . Great video, would really like to learn about epoll in next video

[@rakeshtm2750]

Great knowledge sharing. 😄

[@KalebeAlves-f7s]

8:16 that shmdt cracked me up.

[@kathiravankathir3089]

awesome, !!! could you please post a video about how to use semaphores in thread synchronization,,!

[@JacobSorber]

That's the plan. Not sure when it will come out. But, it's coming.

[@ORagnar]

That was excellent!

[@funhuninjastudio]

Can you make videos for remaking the functions of the standard C libraries. Like making setjmp,longjmp,malloc,free,etc.

[@JacobSorber]

Yeah, probably. I'll add that to the list. thanks.

[@prshntpnd00]

I can see OBJS=shared_memory.o but you don't have any shared_memory.o .. so this should give error no such file or directory

[@prshntpnd00]

Like if I decode it : I am writing filename different .small GCC -g -wall -c write.c shared_memory.o -o write.o GCC:error: shared_memory.o: no such file or directory Consider write.c as writeshmem.c

[@dduartez99]

So this is a implementation via the System V standard/mechanisms, right? POSIX way should be thru mmap and shm_open?

[@JacobSorber]

Yep. I have a few videos on mmap. Probably need another video on shm_open, though.

[@samikshahanchate3280]

I just wanna say..thank you..thank you so much..this is so helpful

[@JacobSorber]

You're welcome. Glad I could help.

[@jscorpio1987]

This video can still be helpful for Windows users who use WSL.

[@JacobSorber]

Thanks for pointing that out. I haven't spent much time with WSL, so I'm often hesitant to make claims about what it does and doesn't support.

[@jon-sw5yw]

wow I just completed my assignment a week ago involving concurrency and shared memory... and you just now post this haha I could've used mmap but I preferred using shmget and shmat for my assignment.

[@JacobSorber]

Nice. I hope it worked out ok, even though I was slow.

[@jon-sw5yw]

@@JacobSorber No worries it worked out perfectly fine! I actually plan to help my peers by referencing your videos! Keep up the great work!

[@60hit99]

What you mean by associating an address with filename?

[@JacobSorber]

I mean that the block of shared memory is associated with a filename. So, if a process wants to find the block of memory, it looks it up using the filename. It's simply the mechanism for finding the block. There is no relationship between the shared memory and the contents of the file.

[@ciph3r836]

I use /tmp/file as a shared mem

[@JacobSorber]

It's always an option.

[@michimarz]

You are the best teacher! Do you actually prefer using VS Code instead of vim, or is it just for those lessons?

[@JacobSorber]

Vim and I have a complicated relationship. Still trying to figure out exactly how to define the relationship. Let's just say I like both for different reasons.

[@abdullahbinjahed6900]

@@JacobSorber lol

[@halavich9672]

Outstanding!

[@JacobSorber]

Thanks!

[@productiveprogrammer4060]

Can you plz make a video series in tool chain of gcc or one video on elf file format?

[@JacobSorber]

What aspects of the toolchain are you most interested in? There are a lot of pieces in there.

[@productiveprogrammer4060]

tools that help us to find memory faults like Coredump

[@productiveprogrammer4060]

@@JacobSorber Can you also make a video on Program execution sequence , what actually happens when we enter ./a.out or (run any executable file)internally in O.S

[@ciph3r836]

How to detect sin wave in a wav audio file

[@JacobSorber]

Meaning, how to detect a particular frequency, additive noise (sine wave added on top of other signal), or how to detect whether the wave file is a single tone (sine wave)?

[@ciph3r836]

@@JacobSorber i am trying detect certain pattern . The sound pattern is a sin wave whose length is 0.01 seconds . The sound wave starts with a maximum amplitude but decreases gradually giving it a click sound . When i view the wav file in audacity I can clearly see the two spikes but when i read that file in to a numpy array things get messy . Tried finding the rms valve of 0.01 seconds of audio at a time . I thought I will get a abnormally large rms valve and i get that but i get multiple large rms value in a wav file with just 1 click

[@ciph3r836]

@@JacobSorber the sin wave is played by speaker and my mic records that so the sin wave is with some noice lke cpu fan,ceiling fan noise or tv noise

[@JacobSorber]

@@ciph3r836 One option would be to compute an FFT of your audio signal, and look for a spike at the expected frequency.

[@ciph3r836]

@@JacobSorber sir i just started doing this just minutes earlier. Fft seems to be promising

[@rafalmichalski4893]

Hello Jacob - great talk !. Could you make some content about Makefiles ?

[@JacobSorber]

Thanks. I already have some. What aspects of makefiles would you like to see more about?

[@rafalmichalski4893]

Hello Jacob, due to Makefiles I think following topics to consider: - %.c : %.o notations and alternatives if any - some defaults rules to write makefile as short as possible with some variables describing comiler, compile/linkage flags - how to mix C and C++ build in makefile - considering CMake to generate makefiles - imvoking one makefile from another if possible