How to measure memory usage inside my program? (getrusage)

Рет қаралды 31,224

Күн бұрын

Пікірлер: 71

@GaussianBluff 4 жыл бұрын

How much memory are you using: C: a little Java: some Javascript: yes Haskell: it's trivial to measure, first you prove Furgelton's 2nd theorem of algebraic manifolds, then simply construct an operator system over the field of... Prolog: true.

@JacobSorber 4 жыл бұрын

Love it.

@simonmaracine4721 4 жыл бұрын

Except that Java usually uses a lot more memory than "some"...

@swarnavasamanta2628 3 жыл бұрын

@@simonmaracine4721 exactly, i compiled an android OS from source once. My 8 GB machine with 8 GB of zram, just couldn't keep up. Had to decrease Java heap size.

@shushens 3 жыл бұрын

Memory is to Java what cookies are to the Cookie Monster.

@user-mr3mf8lo7y 9 ай бұрын

Me, not much. MS? heck a lot..

@badwolf8112 4 жыл бұрын

For some reason, learning about what happens at such a low level is very stimulating. Would be cool if there was an OS or computer with some interface to see in real time what's in your CPU registers, and what are the physical memory addresses of processes. Basically the under-the-hood details you wanna see.

@smrtfasizmu6161 2 жыл бұрын

You can use i r option in gdb. Run program with gdb and write i r in order to see the contents of the registers. I have used it before both for debugging and for understanding what's going on under the hood. You can use disass command in gdb in order to see the assembly code and wher you are in the assembly code. You can use nexti and stepi in order to go instruction by instruction. I find it very interesting that with C I can compile the program and then use gdb or objdump -d in order to inspect the assembly and the machine code instructions and follow what's going on. I learned the basics of assembly, how function calls, stack frames, stack pointer, base pointer, instruction pointer etc. work because of it. It is especially engaging because in C you can practice your knowledge by writing inline assembly (you can do it like this __asm__(assembly code goes here, put it inside parenthesis and put /n character after each instruction);) and you can even write machine code. You can write machine code instructions in a char array which you put on the stack and then cast it into a function (you can write typedef (*Function)(); Function f = (Function)charArrayWhichContainsCPUinstructions; f(); Just keep in mind that if you want to do this you have to compile your program by writing gcc -z -execstack... I don't have that option on Windows I don't know how to download it if it is even possible, but it works on my Linux. Another approach (which I have just tried out yesterday) is to use mmap with PROT_EXEC flag and put your machine code there and then do the exact same process that I have described earlier in the comment.

@smrtfasizmu6161 2 жыл бұрын

Another nice thing is that there are man pages which you can read which explain to you how different functions in C work, what headers you need to include etc.

@papasmurf9146 2 жыл бұрын

Computers from years ago (1960s) would have the register values displayed in a series of lights (representing bits) on the front panel. I'm not sure when the idea died out -- probably went the computers got fast enough that the blinking lights couldn't keep up.

@disabledmallis 7 ай бұрын

Consider trying cheat engine

@obiwac 4 жыл бұрын

Could you make a video on how to/is code in memory executed? It's always been quite an obscure subject for me and I really appreciate the work you put into your videos. I think it would make for a very interesting video!

@Nick-lx4fo 3 жыл бұрын

You could also write wrapper defines for memory alloc functions, which also provides the benefit of telling you where the memory is being allocated and how much

@HimanshuSharma-sd5gk 4 жыл бұрын

English is not my mother tounge , but when this man explains ,it flows like butter in my mind

@alikhalidsalim4865 4 жыл бұрын

You're a legend.

@pathayes2224 4 жыл бұрын

Great work. Can you do a presentation on how the a multicore server allocates multithreaded processes to its CPUS and discuss how the efficiency of these CPUS are effected according as the number threads increase. Also with that how the modern scheduler handles timeslices in a multicore system?

@JacobSorber 4 жыл бұрын

Thanks, Pat. That would be a serious video. I'll put it on the list and see what I can do.

@pathayes2224 4 жыл бұрын

Great. Looking forward to it

@roshanvemana8250 4 жыл бұрын

Really, love the way you dig deep under the hood, coming back again and again

@grizzlyrideemwet1698 Жыл бұрын

How about a discussion of determining memory used/available on an embedded system?

@adimikulic 4 жыл бұрын

any chance you could do a virtual memory video at some point? i've always struggled with the concept

@sachinpandhare8737 3 жыл бұрын

This is actually very very useful

@smrtfasizmu6161 2 жыл бұрын

It is so good that on Linux I have man pages for C, it is basically a cheat sheet. I don't have to spend the time memorising what header to include, how is which constant called etc, what arguments and flags which function has etc. I studied C for one semester in university back then I had no idea that man pages existed (and neither did my colleagues as far as I know), so I don't know if it was considered cheating to read a man page on a test. I wonder if it is considered cheating to look at a man page on a test.

@manufg6366 3 жыл бұрын

In windows instead of ps ux what is the alternate command?

@evolagenda 10 ай бұрын

How many pages does a program get by default to use, if any before needing a new page or does it depend on the size of the program being loaded

@Pranjal-AI 4 жыл бұрын

Thank you so much! This was brilliant... subscribed! 😊

@physis6356 3 жыл бұрын

great vid, thanks a lot!

@arashfatehi9971 3 жыл бұрын

I guess the behaviour (reporting variable sizes in each run) is related to the implementation of libc library on your device, it is interesting to see other implementation`s behaviour like uclibc and musl in this context

@krishnamurthymayya1904 4 жыл бұрын

Amazing video Jacob .. Just a small question, can this be used only for the process or a particular function inside a process as well? Let's say there is a big process in production code which involves multiple functions out of which I want to find out which functions are time consuming and want to optimize the same, can this tool be used for that?

@Catbangin 4 жыл бұрын

I hope you also have a video which compare circular buffer using power-of-2 and the traditional circular buffer.

@JacobSorber 4 жыл бұрын

Sounds interesting. Are you looking for a speed comparison? Or a space comparison?

@Catbangin 4 жыл бұрын

@@JacobSorber Yes both speed and space comparison. I want to know more about it and how the mask work in it.

@ДеянЦонев-ы7в 2 жыл бұрын

I was wondering how to find out , what amount of memory OS gives to my program and how many pages are given without heap allocation? So is the vm max size the limit of the memory which is given to it?

@kepler22b47 3 жыл бұрын

today I was like yeahh I know now something about programming and some inside about it ;)... then I stumble upon this video and after watching, I read some comments that confirm my thoughts but.... ...then I realized hmm I know definitely nothing about computer science at all, haha WTH !

@pajeetsingh 6 ай бұрын

So this is how load balancers are made?

@FightEverything 4 жыл бұрын

Hi Jacob, I'm running into a problem of tracing the memory allocated on a thread (or a task on a thread pool). Is there anyway I could trace that?

@ironmonkey1990 4 ай бұрын

thank you!

@fxn 4 жыл бұрын

Is this paging out what's known as "swapping"? If yes, when we say that a machine is noticeably slowed down due to swapping, do we really mean that swapping is happening way more often than usual?

@dorquemadagaming3938 4 жыл бұрын

Yes, the operating system is using the swap file (or a partition) on the disk to move out rarely used pages to make room in the physical memory - which means you are running out of it. It should be the last resort to keep your applications running, as the disk access is much slower than RAM. If the swapping occurs constantly, those "rarely used" pages are still being accessed a lot, forcing the OS to swap them in and out all the time.

@gokulsvg7069 4 жыл бұрын

Hi jacob can you do a vedio on object oriented programming

@JacobSorber 4 жыл бұрын

Just an OOP overview, or do you have a specific topic you're interested in?

@anikaitmakkar989 4 жыл бұрын

What's the name of the font you're using

@JacobSorber 4 жыл бұрын

Which one? In the text editor? Video titles?

@anikaitmakkar989 4 жыл бұрын

@@JacobSorber text editor

@JohnHollowell 4 жыл бұрын

I would assume it is the default font family for Visual Studio Code which is "Consolas, 'Courier New', monospace"

@shushens 3 жыл бұрын

Are you thinking of offering an advanced OS course for online users? I am talking about really hardcore things that theoretical courses do not go into :) If you do, I would sign up. It has been too long since university, and the OS course there was not much to write home about.

@JacobSorber 3 жыл бұрын

Interesting. Thanks. What topics would you be looking for in that course?

@shushens 3 жыл бұрын

@@JacobSorber Process scheduling, thread scheduling/synchronisation, things an OS does to be more reliable, I/O handling and interrupts, etc. :) Of course the textbooks have a lot of information, but this is mostly theoretical, copied from research papers, and mostly outdated.

@Jonathan-di1pb 2 жыл бұрын

could the variations be because of address space randomization from the os causing it to not align with pages, which causes pages not to be used fully?

@unperrier5998 2 жыл бұрын

user address space randomization aligns on pages. It's the base virtual address of the page that changes, not the offset inside the page.

@unperrier5998 2 жыл бұрын

No actually you may be right, I've tried with a sample program and addresses in the heap varies. It doesn't prove it's ASLR that causes it, as it may be libC initialization allocating a varying amount of memory each time (which is unlikely): $ for ((i =0; i < 5; i++)); do ./a.out; done maxrss=5824 main=0x40116b data segment=0x404034 stack:0x7ffcb27884b4 heap=0x1d782a0 maxrss=5824 main=0x40116b data segment=0x404034 stack:0x7fff060c5e74 heap=0x8392a0 maxrss=5824 main=0x40116b data segment=0x404034 stack:0x7fff0335ae04 heap=0x12f72a0 maxrss=5824 main=0x40116b data segment=0x404034 stack:0x7ffd87970e44 heap=0x22fb2a0 maxrss=5824 main=0x40116b data segment=0x404034 stack:0x7ffd220bb174 heap=0x22d02a0 Code: static int in_data_segment = 42; int main() { int in_stack = 42; int *in_heap = malloc(sizeof(int)); printf("maxrss=%ld main=%p data segment=%p stack:%p heap=%p ", get_mem_usage(), main, &in_data_segment, &in_stack, in_heap); }

@Jonathan-di1pb 2 жыл бұрын

@@unperrier5998 I'm not sure but I thought the offset into the page might be randomized too

@Jonathan-di1pb 2 жыл бұрын

Now that I think about it that would require a bunch of emtpy space inside the page leading to a bunch of internal fragmentation seems like too big a waste...