How to measure memory usage inside my program? (getrusage)

Patreon ➤ / jacobsorber
Courses ➤ jacobsorber.thinkific.com
Website ➤ www.jacobsorber.com
---
How do you measure a program's memory usage? // As programmers we talk about memory usage all the time, but have you ever actually measured how much your program was using? This tutorial shows you how to measure memory usage a few different ways, including getrusage.
Related Videos:
mmap/brk - • How processes get more...
mmap2 (shared memory) - • Simple Shared Memory i...
strace - • Spying on Running Prog...
libraries - • How to write your own ...
***
Welcome! I post videos that help you learn to program and become a more confident software developer. I cover beginner-to-advanced systems topics ranging from network programming, threads, processes, operating systems, embedded systems and others. My goal is to help you get under-the-hood and better understand how computers work and how you can use them to become stronger students and more capable professional developers.
About me: I'm a computer scientist, electrical engineer, researcher, and teacher. I specialize in embedded systems, mobile computing, sensor networks, and the Internet of Things. I teach systems and networking courses at Clemson University, where I also lead the PERSIST research lab.
More about me and what I do:
www.jacobsorber.com
people.cs.clemson.edu/~jsorber/
persist.cs.clemson.edu/
To Support the Channel:
+ like, subscribe, spread the word
+ contribute via Patreon --- [ / jacobsorber ]
+ rep the channel with nerdy merch --- [teespring.com/stores/jacob-so...]
Source code is also available to Patreon supporters. --- [jsorber-youtube-source.heroku...]

[@GaussianBluff]

How much memory are you using: C: a little Java: some Javascript: yes Haskell: it's trivial to measure, first you prove Furgelton's 2nd theorem of algebraic manifolds, then simply construct an operator system over the field of... Prolog: true.

[@JacobSorber]

Love it.

[@simonmaracine4721]

Except that Java usually uses a lot more memory than "some"...

[@swarnavasamanta2628]

@@simonmaracine4721 exactly, i compiled an android OS from source once. My 8 GB machine with 8 GB of zram, just couldn't keep up. Had to decrease Java heap size.

[@shushens]

Memory is to Java what cookies are to the Cookie Monster.

[@user-mr3mf8lo7y]

Me, not much. MS? heck a lot..

[@roshanvemana8250]

Really, love the way you dig deep under the hood, coming back again and again

[@badwolf8112]

For some reason, learning about what happens at such a low level is very stimulating. Would be cool if there was an OS or computer with some interface to see in real time what's in your CPU registers, and what are the physical memory addresses of processes. Basically the under-the-hood details you wanna see.

[@smrtfasizmu6161]

You can use i r option in gdb. Run program with gdb and write i r in order to see the contents of the registers. I have used it before both for debugging and for understanding what's going on under the hood. You can use disass command in gdb in order to see the assembly code and wher you are in the assembly code. You can use nexti and stepi in order to go instruction by instruction. I find it very interesting that with C I can compile the program and then use gdb or objdump -d in order to inspect the assembly and the machine code instructions and follow what's going on. I learned the basics of assembly, how function calls, stack frames, stack pointer, base pointer, instruction pointer etc. work because of it. It is especially engaging because in C you can practice your knowledge by writing inline assembly (you can do it like this __asm__(assembly code goes here, put it inside parenthesis and put /n character after each instruction);) and you can even write machine code. You can write machine code instructions in a char array which you put on the stack and then cast it into a function (you can write typedef (*Function)(); Function f = (Function)charArrayWhichContainsCPUinstructions; f(); Just keep in mind that if you want to do this you have to compile your program by writing gcc -z -execstack... I don't have that option on Windows I don't know how to download it if it is even possible, but it works on my Linux. Another approach (which I have just tried out yesterday) is to use mmap with PROT_EXEC flag and put your machine code there and then do the exact same process that I have described earlier in the comment.

[@smrtfasizmu6161]

Another nice thing is that there are man pages which you can read which explain to you how different functions in C work, what headers you need to include etc.

[@papasmurf9146]

Computers from years ago (1960s) would have the register values displayed in a series of lights (representing bits) on the front panel. I'm not sure when the idea died out -- probably went the computers got fast enough that the blinking lights couldn't keep up.

[@disabledmallis]

Consider trying cheat engine

[@obiwac]

Could you make a video on how to/is code in memory executed? It's always been quite an obscure subject for me and I really appreciate the work you put into your videos. I think it would make for a very interesting video!

[@pranjalsingh9154]

Thank you so much! This was brilliant... subscribed! 😊

[@Nick-lx4fo]

You could also write wrapper defines for memory alloc functions, which also provides the benefit of telling you where the memory is being allocated and how much

[@physis6356]

great vid, thanks a lot!

[@sachinpandhare8737]

This is actually very very useful

[@alikhalidsalim4865]

You're a legend.

[@HimanshuSharma-sd5gk]

English is not my mother tounge , but when this man explains ,it flows like butter in my mind

[@mockingbird3809]

Love this video

[@adimikulic]

any chance you could do a virtual memory video at some point? i've always struggled with the concept

[@arashfatehi9971]

I guess the behaviour (reporting variable sizes in each run) is related to the implementation of libc library on your device, it is interesting to see other implementation`s behaviour like uclibc and musl in this context

[@pathayes2224]

Great work. Can you do a presentation on how the a multicore server allocates multithreaded processes to its CPUS and discuss how the efficiency of these CPUS are effected according as the number threads increase. Also with that how the modern scheduler handles timeslices in a multicore system?

[@JacobSorber]

Thanks, Pat. That would be a serious video. I'll put it on the list and see what I can do.

[@pathayes2224]

Great. Looking forward to it

[@Catbangin]

I hope you also have a video which compare circular buffer using power-of-2 and the traditional circular buffer.

[@JacobSorber]

Sounds interesting. Are you looking for a speed comparison? Or a space comparison?

[@Catbangin]

@@JacobSorber Yes both speed and space comparison. I want to know more about it and how the mask work in it.

[@grizzlyrideemwet1698]

How about a discussion of determining memory used/available on an embedded system?

[@krishnamurthymayya1904]

Amazing video Jacob .. Just a small question, can this be used only for the process or a particular function inside a process as well? Let's say there is a big process in production code which involves multiple functions out of which I want to find out which functions are time consuming and want to optimize the same, can this tool be used for that?

[@TheSpinTensor]

Hi, what you are looking for is a profiler. Usually they are quite expensive. Together with the NEC cooperation I work at an open source profiler for everyone to use. Soon we will provide memory measurement of codes. Please check out github.com/SX-Aurora/Vftrace for further details.

[@FightEverything]

Hi Jacob, I'm running into a problem of tracing the memory allocated on a thread (or a task on a thread pool). Is there anyway I could trace that?

[@smrtfasizmu6161]

It is so good that on Linux I have man pages for C, it is basically a cheat sheet. I don't have to spend the time memorising what header to include, how is which constant called etc, what arguments and flags which function has etc. I studied C for one semester in university back then I had no idea that man pages existed (and neither did my colleagues as far as I know), so I don't know if it was considered cheating to read a man page on a test. I wonder if it is considered cheating to look at a man page on a test.

[@evolagenda]

How many pages does a program get by default to use, if any before needing a new page or does it depend on the size of the program being loaded

[@manufg6366]

In windows instead of ps ux what is the alternate command?

[@user-hg1mn3qo8x]

I was wondering how to find out , what amount of memory OS gives to my program and how many pages are given without heap allocation? So is the vm max size the limit of the memory which is given to it?

[@afsalpa8719]

Can you take vedio on kernel module?

[@pajeetsingh]

So this is how load balancers are made?

[@kepler22b47]

today I was like yeahh I know now something about programming and some inside about it ;)... then I stumble upon this video and after watching, I read some comments that confirm my thoughts but.... ...then I realized hmm I know definitely nothing about computer science at all, haha WTH !

[@islandcave8738]

It gives you a page to draw on!

[@fxn]

Is this paging out what's known as "swapping"? If yes, when we say that a machine is noticeably slowed down due to swapping, do we really mean that swapping is happening way more often than usual?

[@dorquemadagaming3938]

Yes, the operating system is using the swap file (or a partition) on the disk to move out rarely used pages to make room in the physical memory - which means you are running out of it. It should be the last resort to keep your applications running, as the disk access is much slower than RAM. If the swapping occurs constantly, those "rarely used" pages are still being accessed a lot, forcing the OS to swap them in and out all the time.

[@gokulsvg7069]

Hi jacob can you do a vedio on object oriented programming

[@JacobSorber]

Just an OOP overview, or do you have a specific topic you're interested in?

[@Jonathan-di1pb]

could the variations be because of address space randomization from the os causing it to not align with pages, which causes pages not to be used fully?

[@unperrier5998]

user address space randomization aligns on pages. It's the base virtual address of the page that changes, not the offset inside the page.

[@unperrier5998]

No actually you may be right, I've tried with a sample program and addresses in the heap varies. It doesn't prove it's ASLR that causes it, as it may be libC initialization allocating a varying amount of memory each time (which is unlikely): $ for ((i =0; i < 5; i++)); do ./a.out; done maxrss=5824 main=0x40116b data segment=0x404034 stack:0x7ffcb27884b4 heap=0x1d782a0 maxrss=5824 main=0x40116b data segment=0x404034 stack:0x7fff060c5e74 heap=0x8392a0 maxrss=5824 main=0x40116b data segment=0x404034 stack:0x7fff0335ae04 heap=0x12f72a0 maxrss=5824 main=0x40116b data segment=0x404034 stack:0x7ffd87970e44 heap=0x22fb2a0 maxrss=5824 main=0x40116b data segment=0x404034 stack:0x7ffd220bb174 heap=0x22d02a0 Code: static int in_data_segment = 42; int main() { int in_stack = 42; int *in_heap = malloc(sizeof(int)); printf("maxrss=%ld main=%p data segment=%p stack:%p heap=%p ", get_mem_usage(), main, &in_data_segment, &in_stack, in_heap); }

[@Jonathan-di1pb]

@@unperrier5998 I'm not sure but I thought the offset into the page might be randomized too

[@Jonathan-di1pb]

Now that I think about it that would require a bunch of emtpy space inside the page leading to a bunch of internal fragmentation seems like too big a waste...

[@shushens]

Are you thinking of offering an advanced OS course for online users? I am talking about really hardcore things that theoretical courses do not go into :) If you do, I would sign up. It has been too long since university, and the OS course there was not much to write home about.

[@JacobSorber]

Interesting. Thanks. What topics would you be looking for in that course?

[@shushens]

@@JacobSorber Process scheduling, thread scheduling/synchronisation, things an OS does to be more reliable, I/O handling and interrupts, etc. :) Of course the textbooks have a lot of information, but this is mostly theoretical, copied from research papers, and mostly outdated.

[@eotcoldhymns2930]

Which open source IDE do you recommend for C/C++ development including embedded system development?

[@JacobSorber]

I don't usually use an IDE. Here's a video of me explaining why (a while ago). kzbin.info/www/bejne/hqLJZ3tnjLiVm7s

[@sumitbhosale3401]

Nice Video. Can You make video in fuse file system and data structure. #StayHome

[@JacobSorber]

Thanks. I'll see what I can do.

[@anikaitmakkar989]

What's the name of the font you're using

[@JacobSorber]

Which one? In the text editor? Video titles?

[@anikaitmakkar989]

@@JacobSorber text editor

[@JohnHollowell]

I would assume it is the default font family for Visual Studio Code which is "Consolas, 'Courier New', monospace"

[@TheYeahbaby2010]

what is your text editor?

[@Nick-lx4fo]

VS Code

[@Kenbomp]

Ps vz proc.psuedo file Getrusage

[@m4l490n]

Linux FTW!!

[@JacobSorber]

Indeed.

[@R4ngeR4pidz]

Last

[@shahnawazazam]

Not last

[@Nick-lx4fo]

@@shahnawazazam last as of the time of posting this

[@ie4242]

you're hot, just saying. Also thx for your help

[@JacobSorber]

You're welcome.

[@HassanAli-yk7ne]

Thora khaaya peeya kro0.