Linux Crash Course - Understanding Memory and Swap Usage

[@rudyleplane727]

Can't thank you enough for explaining the memory swap subject! Over 3 years using Linux as my daily driver I still found myself closing browsers, shutting down my virtual machines, even rebooting to address it (even with 32gb of ram). I now have the confidence to tackle this and really make use of my rig!

[@krernertok9686]

Hi Jay! Your videos are a godsend. As a career changer and a sort of a late-starter in IT, your content is pure gold to me. This video helped me fit some pieces better in place and I feel more confident with this topic now. Thank you so much!

[@hector9586]

it's never late to learn! keep on learning.

[@walter_lesaulnier]

I LOVE the way you and DJ Ware cover these kinds of topics- a great blend of more advanced topics that are useful for home desktop users as well as server and enterprise admins. I'm a compulsive tinkerer and videos like these help me learn ways to satisfy my tinkering/ optimization addiction.

[@christopherjackson2157]

I like this format where you go through tools and features individually with their own videos. It works better for me than longer more wide-ranging videos. Thanks for the helpful content!

[@4thatfilm]

This is the best vid I've seen on the science of memory and use of swap!

[@ferdwinfondong761]

All about data handling. The whole IT field rest on that. Thanks Jay for readjusting this in my brain. Your content is a brain saver.

[@kellingc]

I've always included a swap file on distributions I've installed. Now, there is a different type called a page file. In the Windows world, the difference is a page file is dynamicly allocated as needed up to the limit you set, whereas a swap file is a static size. Page files are more disk management efficient because it only takes up as much space as it needs. But, contiguous space is not assured, which can really hurt performance. My first experience with paging and swapping was in 1985 on a VAX 785, where idle procecess would be swapped out to allow higher priority and active interactive processes access to memory (all 2 megabytes of it). We were running VMS 3.4, and the system administrator made the swapper a lower priority than he should have. The VAX (which serviced the 28,000 undergrad and graduate student population - even if you weren't a student that had a reason for an account, you had an account) was freezing. After about 3 days of troubleshooting, it was found that the swapper was swapping itself out to the page file. When something is paged out, it is put in a sleep state until the swapper swaps it back into memory. Well, if the swapper is put to sleep, how is the swapper going to swap itself back in? The system admin pit the swapper back at the hiest priority so it would never be swapped out. Thank you for reading my war stories of my leading a time sharing system.

[@StrikerEureka85]

i've always wanted to learn how this works. thanks so much for uploading!

[@send2gl]

I have an old original Raspberry Pi with 256 MB ram and 64 MB allocated to GPU. It runs headless and has Apache server installed along with ssh, dlna server, network scanner server and a vnc server. A desktop is installed. I very rarely need the desktop so both it and vnc server are disabled and only started if required. I did enable 512MB of swap as I think this may be needed when I do run the desktop and vnc server. I do have more modern Pi devices but old faithful has server me well and continues to do so.

[@babhineethbhat]

Was looking for this. Thanks 👍🏼

[@progpogs]

In the scenario of hosts behind a load balancer, I would rather avoid swap slowness all together, and quickly remove that host as it stops taking requests reaching memory limitations, that I would see request latency increase, especially if your application is set to backoff/retry behind the scenes. This is the best argument against it imo, its not just about disk space being cheap.

[@tourdesource]

That sounds like a good strategy, thanks for sharing

[@devbites77]

Great intro to the topic, but would love a deep dive into memory.

[@janleroux7257]

Thanks. Nice clear explanation. Simple enough to understand.

[@testaccount-xl3ki]

oh my. You nailed it, good explanation

[@UmutAkkaya-s8q]

Great video thanks.

[@13thravenpurple94]

Great work 🥳🥳🥳 Thanks 💜💜💜

[@TomeOfKnowledge74]

I use swappiness on my machines to limit the swapping to my SSD. I'd love to hear your take on swappiness, and recommendations.

[@reptilicusrex4748]

Excellent content, thanks. In regard to allocating storage for swap-space, maybe you can elaborate on this: When a laptop is put to sleep/suspended I thought that the OS writes the contents of RAM to swap-space and back the other way when the laptop is awakened. So, in that use-case wouldn't a sufficient amount of swap be necessary? Sleep/suspend isn't relevant to servers/workstations but maybe it's worth mentioning.

[@SteveMacSticky]

@Reptilicus Rex yes that's true. if you wish to put your machine to sleep or suspend, you need swap enabled, generally 2gb bigger than your ram.

[@kavishaabeynayaka7359]

This relates to hibernation i think where we want power-off but also to start from where we left since we need to power off the device all the contents in the ram written to a non-volatile memory like hard disk(swap space)

[@Matt2010]

The reason too the free is lower than available is the buff/cache. The latter being alot like a RAM disk, but is changeable depending on memory demands.

[@gregorydayton5428]

Very helpful video. Thank You

[@Rbourk252]

I always benefit from LLTV. My knowledge has expanded ❤️

[@benjaminwharton6264]

Thanks my dude

[@cfg83]

Excellent explanation of the /proc/meminfo Cached parameter. My new slogan is "Unused RAM is wasted RAM". Now, where's the T-Shirt?

[@user-mr3mf8lo7y]

Even though it may cause slowing down the server, any swap is better than no swap, especially running a database aplication. 30+ years of experience suggests that. Cheers,.

[@LearnLinuxTV]

Agreed!

[@sandeepvk]

The issue with swap memory is also about how much to spare. Now is 1 GB good enough or should it 50% of the RAM ? like I have a 16 GM RAM and partitioned 4 GB for swap in my SSD

[@LearnLinuxTV]

It all depends on whether or not you plan to use hibernate, or "suspend then hibernate". If you do, swap should be slightly larger than how much RAM you have. If you're not using hibernate (and never plan to) then I usually just standardize on 2GB (4GB at the most).

[@Not-THAT-ChrisPratt]

Once again you have helped me to better understand a Linux topic, esp swap. Thank you so much!

[@treyquattro]

hey Jay, nice episode as ever - you've amassed a great body of work already. What are the plaques with Japanese (I'm assuming) characters behind you?

[@eliinthewolverinestate6729]

A m.2 at a good clock only moves 5000 mb/s at best and ram does 35500 mb/s. A sata ssd is around 500 mb/s. It's better to use the ram for video than hard drive. That's my theory is try not to use swap at all. But you need swap if you freeze your computer. The swap will slowly untangle it. So that is why the older thinking of twice the ram.

[@duncan-mcrae]

Keep doing what you're doing. Much appreciated

[@Richardpasquinucci]

If I am using Linux Mint. Can I do this from the Cinamon browser instead of the terminal?

[@drcemdede]

Thanks for the video. Unfortunutely, my server uses swap because of the analysis dependencies. I have more than enough space, ram and storage but the analysis sometimes oversteps to swap.

[@RobBCactive]

Then you have enough memory demand to write dirty anonymous pages to the swap file, there's no harm in having idle unused daemons end up with saved state in swap rather than retain memory wasting it. It's when you have a high rate, due to insufficient memory causing thrashing and have the OOM killer terminate processes, possibly due to lack of swap allocation that you have a real issue. This swap is bad mantra is over simplistic, it's there to use memory better.

[@ricks2061]

Hii please reply. I need help. I kinda set my swapsize to 2092 and now i can't remotely desktop connect to my Jetson Nano with Linux system. And I checked in it's SSH that the "system has been minimized". Is it because i set up it high? Did i broke my device???

[@toweliethetowel8280]

"free -h" is the best readable for noobies like me :)

[@brbbst]

I’ve got 15.5 GB RAM and the max thats being used is 3.8GB ( 2.1GB AVG). Swap is always used up to 3 GB ( 1.7 AVG). The performance of the applications I am hosting is not very good and I feel that is because it is not using enought memory even thou I’ve set 2.5GB for each ( total 7.5GB). For example right now in bpytop it shows the ram as 19% used ( 2.94 GB) ,81% free (12.6GB) , 79% cached (12.3GB) , 1% Free ( 163 MB) and the swap 328 MB used. So at this point I don’t understand swap since I am not running out of memory.

[@Anthony-Webb]

I come back to this video every time I need a swap refresher. One thing I will say is that by the end of the video, I hate the word "swap".

[@waynemorgan6714]

Wow thank so much. I just set up a Raspberry pi 4 8GB RAM, booting off a 1TB SSD for Ubuntu Server 22.04 I was worried about having enough RAM

[@petevenuti7355]

Is there a way, driver or something that would allow one to use storage as system memory? Say use a flash drive as part of system memory and a hard drive for swap. If I have 2 GB of ram and need 32+gb, setting a 64gb swap isn't going to help. How about a distributed shared ram?

[@AHMEDIDANI]

Thank you man

[@Dillinger86]

Any content you put out for System 76 is valuable to me, I own one & it's my first Linux machine.

[@muzikermammoth3995]

How do you calculate the total memory from the available memory, free and used, etc? It used to be you could do that in the past, but in this example there seems to be megabytes of missing ram.

[@DL-xf3ur]

Thanks as always for the great content. Much appreciated.

[@LearnLinuxTV]

My pleasure!

[@cheako91155]

3:20 You have a gig "free" on that server because you don't understand what that 0.5G swap and vm overcommit_ratio are doing. The default for overcommit_ratio is 50, that's where 2.6 is coming from it's close enough to 4.5 * %50. This %50 is where the twice ram(because half of 3(1 + 1 * 2) is a little over 1 or use all of ram and some swap) for swap advice comes from, when ppl stopped following this advice... the kernel devs never changed overcommit_ratio and thus the old advice is still current. It's not that you have a gig free.... it's that you have a gig of ram the kernel will never commit... to things like running applications.

[@mrri8403]

This guy is awesome 😎

[@LearnLinuxTV]

I know! He's just so amazing. How does he do it? He's just soooo cool 😃

[@frankkolmann4801]

When I installed a 2TB SSD 16G mem in an old Toshiba laptop running Ubuntu I got 15sec boot time I set up 16G swap file, everything starts in seconds I can run multiple applications and I never ever need to use Htop or other system monitors.

[@jc.2190]

1y old, but still be a great video.

[@Alex-rh4yt]

This feeling when you understand how it works😊

[@lucasrodrigues9766]

So now I will learn whether reducing swap to 10 makes Linux faster or not.

[@matthiasbendewald1803]

It is a compromise: It will make your system more "snappy", applications that didn't run will stay in RAM for longer and therefore be available more quickly. On the other hand, data of running applications will not be cached as well, so accessing data from disc will take longer and therefore your running applications will be slower. If you want to know more learn to use vmstat and how to interpret the values. This gives you the tool to fine-tune your applications.

[@The-Other-Mario]

What's your opinion on compcache / zram like it is used by default on fedora instead of a swap partition/file.

[@k.chriscaldwell4141]

Thank you.

[@mihai6564]

good video

[@ramyanarsupalli2109]

Nice video,, but the swap space is used even I have free and available memory,, what would be the cause?

[@АрсланГаджиев-ж5ж]

Please make a detailed overview of the pros and cons of your system76 laptop and not a big comparison with the MacBook Air m1 13 in terms of work?

[@armajroy]

How about making a video about open source BIOS. If I want to install an open source BIOS in my computer replacing previous closed source BIOS what kind of challenges do I meet? Or, is that even possible?

[@LearnLinuxTV]

It’s very possible to do, but depends on what model you have. Unfortunately, that particular topic wouldn’t translate well into a video because a small number of people would have the tools and appropriate hardware.

[@armajroy]

@@LearnLinuxTV Yup, that's true. Well, in near future, if it is possible explain these concepts with your amazing video.

[@mijaelviricocheaparra7474]

Well, it is stated that the swap is a bad thing because ot uses slow storage devices, but I'm not sure if it is still the case. We now have 7000 MB/s storage devices.

[@Gaius__]

Yes, and we also have DDR5-6000 RAM now (6000MHz / 6400MHz being considered the "sweet spot" for AMD / Intel CPUs), which transfers at 48000 MB/s.

[@waelelkholi3406]

Why not make the swap file very large ( for ex 20G) to have a lot of ram

[@aleksandartyankov1454]

test it out - assign 20GB of your disk for swap space, set the swappiness to 100 (use swap aggressively) and you will get the answer to your question very quickly :) basically you would think you have a machine with a lot of memory, while in fact everything will be extremely slow

[@TadakichiSan777]

Can you make a video on audit daemon since its running in your htop?

[@durangodave]

hi, question, what was the 60 swapiness, is that in GB ?

[@Gaius__]

No, that is a likelihood of use, on a scale from 0 (never use) to 100 (always use). However many processes or applications will simply ignore it, and still write to swap, even with lots of RAM available. The best thing to do is leave it alone (especially on desktop Linux) and let the OS do its thing.

[@Wardaug]

Why doesn't "disks" set up the linux file system when you are partitioning a external HDD

[@venkatesh4760]

Thanks for this Video and what is the shell used to get this kind of arrow prompt ?

[@LearnLinuxTV]

It's just bash with a custom config, nothing special.

[@DaveSomething]

I use the -h on things like free or df, h is for human readable...

[@siamhie]

Same for me.

[@predragnikolic1259]

Tnx , this is very helpfull and informative for people migrating to linux like me :)

[@abhimudaliar1064]

Thank you

[@alx8439]

I'll just give you this real example: you might have an app which is memory leaking, it will first consume all the ram, then it will make other processes memory to be pushed into swap. And then it's a matter of chance, what kind of error handler for memory allocation exceptions that buggy app will have. In vast majority of cases it will have something like "okay, we failed to malloc, we wait for a while and try again". Which in practical term means your system will become very unresponsive, because whatever memory will be freed, it will be immediately occupied by that single buster. And your chances to quickly ssh to that machine and kill the nasty app will be getting thinner and thinner, as now SSH daemon, and login/sh processess it spawns will be 100 times slower, as they'll be swapped to. So swap is bad not because of religious holywar. And to be 100% safe, have some resource manager set up on your system. And test how your system behaves, when some shitty process eats all the ram. It can be coded with just few lines of C code. And it will open your own eyes what a disaster it can be, if any of apps your server might be running will get some memory leak.

[@FeiBenZhu]

"How do we monitor memory usage without having to install htop?" … use top instead. It shows you mem & swap stats as numbers sans the fancy graphs.

[@pivot3india]

Can you do a Video on fio command

[@ohwow2074]

Someone please tell if it's possible to install GCC 12.1 stable version on Ubuntu 22.04 or not. I've been struggling to update mine from 12.0.1 which is experimental. I just can't find a way to install 12.1 which is the latest version.

[@marcin2x4]

Linux Crash Course helped me in improving Linux knowledge! Great thanks for your contribution and outstanding tutorials! On the other hand, I wanted to ask your opinion on WSL2. Did you have a chance to interact with it? Any thoughts from professional community on it's usage in day-to-day tasks?

[@kalaivanipalanivel]

I f I reboot the system . How I am always keep the swap memory on

[@roadkill9173]

-m is show output in mebibytes; while --mega will show output in megabytes

[@JoelTalom]

How to like twice a video? 😉 great content

[@LearnLinuxTV]

Thanks!

[@8bitoverclocking932]

I wish every distro, whether it be for desktop or server, came with vim and htop. These 2 programs are my first downloads for any distro I install. Great content as always, keep up the great work!

[@McMaxW]

I have +15GB of free RAM, how can I make the system use it by caching more?

[@tonymckeown5393]

Agreed, I have said myself "why argue about putting aside a 'tiny scratch' on a huge disk", I think some individuals just attach to ideas and 'get comfortable'. Swapping may be slow but it could be the best place to be when needed.

[@peterjansen4826]

I am naughty, I haven't used swap for years and it goes fine. :) 32 GB DDR4,

[@bertnijhof5413]

Excellent video, easy to understand explanation of the essentials. However I only use desktops, I trust the defaults chosen by the kernel designers, so I never touch swapinness anymore :) I moved my applications to VMs and my Host OS is a minimal install of Ubuntu 22.04 LTS, my VMs run off OpenZFS 2.1.2. My Host OS has 2 swap partitions; first priority a 2GB Swap on a nvme-SSD (3400/2300MB/s) and second priority a 16GB Swap on HDD as an overflow swap, but intended more for future hibernation. I often see swap usages between 100MB up to 1500MB and I don't care, since I don't notice any negative effect on the responsiveness. Remember many programs have large parts, that are never used or only once during e.g. initialization, closing the program or in case of errors or exceptions. I see it as "unused and wasted" RAM too :) Those parts of the program will be swapped first (least frequently used), so you will not see any effect with respect to responsiveness. I limit the amount of RAM I assign to VMs, because I want to be able to run 3 to 4 VMs in the 16GB of my Ryzen 3 2200G. The VMs share the RAM with a 0.5GB to 3GB ZFS memory cache (L1ARC). So for Windows 11 Pro I use 4GB and for Linux VMs I use 1.5GB to 2.5GB. I have no complaints about responsiveness. Also there I see the SWAP is used extensively, but that swap storage is part of the virtual disk and that virtual disk is cached by the ZFS memory cache (L1ARC) of the Host OS. The L1ARC is lz4 compressed, so currently it stores 6.12GB of compressed stuff in 3GB physical space. Looking at the size of Linux after booting say 0.6GB to 1GB of uncompressed storage, the Host OS stores most of the used disk content of the 4 VMs in the L1ARC, including any used swap space. Consequently my VMs run from memory :) :) and even on the 14nm Ryzen 3 2200G (4C4T, OC 3.7GHz) it is fast. The boot time and initial program load times depend on the speed of the storage, so Xubuntu 22.04 LTS boots from nvme-SSD in 9 seconds and Xubuntu 20.04 LTS boots from two striped HDDs supported by a 90GB sata-SSD Cache (L2ARC) in 30 seconds. Booting Xubuntu 20.04 LTS for the second time takes 11 seconds, since now it largely boots from the L1ARC memory cache. Note that e.g. ext4 installations also extensively uses caching, but by default the cache is uncompressed and the algorithm is LRU. ZFS uses a more sophisticated method of caching, mixing LRU and MFU, while dynamically adapting the boundary between the two based on the dynamic results of both methods. My VMs use ext4, so I have some caching there too, but the available memory is relatively small, so caching is relatively often deferred to the Host level and the superior L1ARC.

[@Pendrag0n]

I would love to see also tutorials that touch this harder topics (why its show less mem than actually physical mem, what are shared and cache plus free values differ from available, etc.). This concerns also other your videos/topics. As an Linux admin/user for years I already know basics, please consider to make videos for more advanced Linux user.

[@djamolmukhtarov3328]

Could you make the explanation of free and available more simplified than it is on the video

[@Gaius__]

Free = completely unused, just sits there, doing nothing at all, waiting for an assignment. Available = partly cached, but not on essential/current tasks, so it can be reassigned at any time, if needed (includes "free", of course).

[@pyrokamileon]

swap drinking game confirmed! 😂

[@binbashbuddy]

OK, I'm laughing my butt off. It looked to me like you used your mouse to quit htop, so I went to a terminal and opened it and clicked quit and it quit. I've used htop for I don't even know how many years and didn't know you could use a pointing device. I don't know that I'll do it again, but it sure made me laugh.

[@supperEisMan]

Thats a lot of saying SWAP 😅 I bet it felt weird saying it so many times. Very informative though!

[@peterjansen4826]

For me free shows 32087 MB total RAM, just over 32 GB of what I have installed.

[@Gaius__]

32GB = 32768MB. You are confusing GB/MB with GiB/MiB.

[@peterjansen4826]

@@Gaius__ Not really confusing it, just not knowing the abbreviation out of my head. I know the difference: 1000 vs 1024..

[@AIMJL-j2i]

GOATED

[@OveRaDaMaNt]

I use zram so my swap is in my ram

[@Yuri_Sh-911]

i've set swap = 0 but my proxmox dont care abt it ((((

[@luke421]

I love you

[@sanskarsingh9538]

this could be a 5 minute video!

[@_den_]

This is probably the same amount of water in your book.

[@djamolmukhtarov3328]

I really like your videos but this one is so complicated when you are trying to explain differences between free and available

[@kamrui.minipc]

Hey friend I'm Jiejie from KAMRUI. We sell all kinds of MINI PC on Amazon. We are looking forward to presenting it to your audience through your professional presentation. Can I have a chance to work with you?

[@purohitiyer2548]

Ig im the first XD, btw love your vids!

[@LearnLinuxTV]

Yay! Thank you!

[@MartinMllerSkarbiniksPedersen]

No swap is bad. You want the kernel to kill process that eats memory, not swapping.

[@georgehope5477]

You should upload your videos to odysee.

[@Bairyhalls47]

Too streched out to cover very little information. With 20 min, you could've went a lot deeper. The video has way too many filler sentences

[@mulimotola44]

You waste a lot of time saying the obvious and when it comes down to understanding the differences you just say "I'm going to stop my explanation right here". What's the point of making this video? Engagement only I guess

[@hrxy1]

learn to focus your thoughts, stop jumping around, one thought at a time. unwatchable

[@MartinMllerSkarbiniksPedersen]

No program is made to use swap. You are wrong. And you are wrong about swapiness