Testing File Transfer SMB vs NFS with CPU Bottleneck!!

[@SouravTechLabs]

5:12 With 90 MiB/S you would have written the whole 806 MB file three times over within the time you were talking. You started copying before 4:47 and you were 50% done at 5:17 so it takes a minute to complete the copy, that means you are getting around 13 to 14 MBPS. Thunar's copy operation was freezing but the downloading was active (look at your router, it's blinking if it has those lights) that's the reason of seeing 90 MBPS.

[@baumstamp5989]

exactly! it was just a buffer flush of the file that made linux file copy show that high speed for a moment, the last second before the video cuts, you can see that CPU jumps to 100% as well. also of course the time it takes for the 806 MB file... never 90MB/s... if he would test with a larger file it could be more easily seen after the initial burst of the copy.

[@bugs181]

@@baumstamp5989 This is why it's important to use proper testing tools, also. A great one to look for on any OS would be iPerf.

[@koush_b]

Good point Gaurav. I'm a regular linux user & I know the drill; in Linux it doesn't show the correct transfer speed. It's always a gigantic bump in the beginning & slowly decrease to the actual speed after a while if the file is big enough to copy.

[@maxheadrom3088]

My Samba transfers full Gigabit on a J1900 MB. I'm having problems now but it's probably from the CAT5e long cable. Interesting video and good comment.

[@epiqpwnage6989]

@@bugs181 using iperf3 to test my connection from my windows 11 to raspberry pi4 (ubuntu mate) and i get around 900mbps. both are connected wired gigabit ethernet. Using NFS and SAMBA I get about 40MB/s (320mbps) monitoring the pi with htop the cpu is not the bottleneck. Really can't figure out what is limiting it that much

[@L4X1K3]

that can't be 90mb/s, with the file being just over 800MB, this would take less than 10 seconds if that was the case and this takes around as long as the initial samba transfer at around 15-20mb/s

[@cocosloan3748]

I regularly transfer files from 1 comp to another via 1Gbps switch at speeds of 110MBps in Windows .Never had slow speeds (only when cable is low quality then it falls to 100Mbps) Some cables were bad I replaced them all and had no problem reaching 1Gbps ever ...

[@ryzenforce]

Although NFS (a file level access) is somewhat easier to setup than iSCSI, you need to understand how it works. With NFS, the device that actually read/write and make sure the data is written properly is the system running the NFS service - here your little NAS. It needs to have enough queues also to process stuff network traffic, good and fast disk pool, memory and a fast CPU, all things you dont have in your NAS. For iSCSI, the device that actually read/write is the remote system because it can access at the block level, and the NAS doesn't really do much in terms of data integrity.

[@thk4711]

I would recommend that you find out what is causing the high CPU load - is it samba or something in the kernel. If it is samba have a look at the network parameters there if it is the kernel there might be a problem with the driver of your network card. I am running a DIY NAS using a nanopi m4 with the SATA interface add-on . This board has a slower CPU that what you are using. There I get close to gigabit transfer speed in samba, ISCSI and NFS .

[@NovaspiritTech]

thanks good tip, i'll have to cehck that out

[@AndersJackson]

Yes, there are lot of things that could be trade of to get more speed when you are running NFS in both client and server. There are probably some things you also can do with SMB, but not that much.

[@GeertDroid]

I would love to see a follow up video on this topic !!!!

[@jonnykopp]

@@GeertDroid agreed

[@ran_red]

I don't know about people in the comments talking about the tests weren't done right, the numbers aren't correct, buffer this, copy that but I do know how to measure time. I just transferred a Ubuntu Server ISO (1.4 gigabytes) from OpenSUSE Tumbleweed to a server running Fedora using NFS and it took 11 seconds. I did the same thing from Windows to the same Fedora server and it took well over a minute, almost two minutes.

[@sashawinters2177]

i am using rpi 4, file trasfer over smb. hdd is attached to rpi 4 over usb3. If the hdd is formatted to ntfs speed is about 30.-40 (bc linux has a poor ntfs support). If the hdd formatted to ext4 speed is about 90.

[@Tumbzilla]

I was hit with same problem on Asustor NAS. CPU can't give 90MiB/s with SMBv2 or SMBv3 or any protocol that use secured connection. Only way i was able to use full network was with SMBv1 because it's unsecure protocol. I have my NAS direct connect to proxmox with SMBv1 and SMB server on Proxmox securing data connection to clients.

[@TheSecurityAgency]

Thank you very much, it was very interesting and absolutely unexpected.

[@TheInternalNet]

MORE! Really awesome DON!

[@Woodwerker]

Windows 10 slow transfer rate speed fix. I was getting 20 MB/s also when sending files to SAMBA share on Linux from Windows 10. After this fix I get up to 100 MB/s... step 1.) at administrator cmd prompt type: netsh interface tcp set global autotuning=disable step 2.) in network interface settings, advanced: large send offload disabled for IPv4 and IPv6 step 3.) Reboot and enjoy faster speeds

[@lawrencedoliveiro9104]

This is why they say, Windows is a great OS, if your time is worth nothing.

[@BruceBigby]

Maybe the data is going directly into the Linux kernel's buffer cache. NFS won't necessarily sync the data right away, but eventually, the kernel will sync the data to disk. There are probably NFS sync options. Apply them, and your timings might differ.

[@donporter8432]

Important to be aware of! Thank you

[@noenken]

NFS to NFS (Linux to FreeNAS) I can actually write to my SSD NAS with full 10GbE speeds on a Ryzen 3 1200. Protocol makes a lot of difference.

[@BrianThomas]

so which protocol is recommended for file sharing? SMB seems to have a lot of overhead.

[@raiscl]

That looks about right. I get better speeds with NFS than SMB at home, the only thing I hate about NFS is that all graphical file managers kind of freeze when transferring big files. Nautilus, PCmanFM, Elementary Files, all the same. So I tend to stick with rsync for big transfers .... =/ And it’s not a Async/Sync thing, unfortunately.

[@uibsen]

Do you have SMB 2 or higher enabled on the NAS ?

[@methanoid]

Don, what is that ITX NAS case? I wanna see if it's FS in UK.

[@HULCHUL1]

Looking for same, hope you find out what it was?

[@HentaiNat]

@@HULCHUL1 watch his video on the diy nas build

[@Smart-Skippy]

Thanks for making this video! Would SSH be an option for fast Windows 10 to my Ubuntu Raid 5 NAS ?

[@peachiekeen1565]

I have not researched different shares or file systems. I am currently using Windows to truenas based on Linux. Don't know what speed is. Doesn't seem slow would guess is limited by 1 Gb Ethernet ports

[@jarisipilainen3875]

0:37 very usefull.it only mean robber take whole case and leavy you monitor and power cable xD

[@lawrencedoliveiro9104]

In general, I would expect that file-access protocols like SMB or NFS would be inefficient for bulk file transfer. I was going to suggest you compare using rsync for this test, but given you can already max out your LAN bandwidth, that might or might not prove anything. ;)

[@VinaykanthK]

Can I possibly add network files either smb/nfs to local search index ?

[@MarkParkTech]

Windows NFS implementations have always been horrible.

[@rhfweb]

Epic, I just re-installed Freelancer the other day, what mods are you playing?

[@NovaspiritTech]

Hahahahahah same here!! I'm trying out discovery freelancer mod

[@rhfweb]

@@NovaspiritTech IIRC back in the day, the dedicated server binary ran better under Wine than it did Windows itself.

[@Tweak42]

I'd like to add comparison using SFTP transfer speeds and a better method to accurately benchmark throughput.

[@mikehensley78]

I only use sftp to transfer to/from my NAS via filezilla. I get 95 to 110MB/s. If I try to transfer from my server in Europe to USA via sftp I get 1 to 3MB/s. But if I mv the file to my http dir in EU and wget the file(http) in USA I get 14MB/s(my inet speed cap). Sftp adds a layer to decrypt because its basically ssh protocol. So a good CPU for both machines is needed for higher speed transfers via sftp. My EU server has a weaker CPU so it can't keep up. Some protocols be a harsh mistress.

[@nadeemm3536]

Have NAS share from Dell hw with centos and client is VM rhel7 having slow speed for read/write it happen on 6 diffrent nodes which have same MAS master share.. any advice

[@kirmistube]

Is there any performance difference for copy/paste speeds between NAS if you use NFS instead of SMB ?

[@MrJaz8088]

Windows 10 has Turned Off Read/Write Ahead for Attached Device, as people do not "Safely Remove Hardware" they just pull the unit It can be Turned back On and speeds are good again

[@lezbriddon]

win7, i can often download a file from the net to my network shared hdd faster than i can copy a file from the desktop to it.... which makes no sense. i just walk away as theres always time to make fresh coffee

[@empetor492]

whats the model of nas which runs webmin? i saw armv7 so it won't be so much powerfull as intel/amd would be

[@tommyb.6064]

So, i would guess that windows is giving the nfs server some shit to eat that it has to process? How is that possible? I mean, does when a file is transfered over ftp smb or nfs, other things that needs to be later converted such as the file system are carried over? How can windows send "bad nfs packets" that later on overloads the receiver cpu? I see it but, i need to have that understood.

[@doomicle]

NFS for Windows is a buggy mess. I installed it, and it stopped my SAMBA shares from working.

[@ngoquang2708]

In case of nfs, I think there is caching is going on, mean the linux system read the file you are uploading from disk to ram first but don't upload the data to the nas yet so the speed there is the read speed of the disk.

[@rjshaver1]

Would love to know the issue here. Many have suggested several things but I would like to have some of the suggestions tested. Poor CPU, antivirus, windows settings, etc. Thank you Don for sharing.

[@RoyHess666]

Network File System file system!

[@CINEMATIK]

very helpfull. thanks

[@AndrewAHayes]

Why does the NAS need a graphics card?

[@NovaspiritTech]

hehe, what he said =P

[@pepeshopping]

“Newbie” issue: This is due to the protocol (write sync) more than the cpu. It does not help to get a cheap cpu for your NAS.

[@lessevilgoog]

So.. are you saying the Windows implementation of the NFS protocol is doing write-sync differently than Linux, that seems like a pretty dramatic difference.

[@crazyrussianbot8012]

Linux/Unix is way better technology. only reason Microsoft exists is monopolies and chaining people to various systems like eg. DirectX, C#, Visual C++, and stuff like that. this way they have secured to be gaming platform #1 and apps platform #1 - at the cost of user friendlyness, surveillance, reboots, forced updates, etc...

[@rablumberg]

By my knowledge NFS does not use any kind of credential check, so for a small business would it be better to use SMB and CIFS to authenticate users? Is there a better solution to authenticate users using NFS?

[@danieldewindt3919]

Cool Comparison

[@cdoublejj]

so linux has a better nfs implementation?

[@stevenbell9589]

I'm on the lookout for a new server I'm using a hp proliant micro server n36l over 13 years old thinking of a new proliant and going down the Linux rout maybe you could test some different Linux server distros to see what's best for the need

[@WPGinfo]

Thnx for these observations.. Yes, this is interesting... however... is there a solution [or did I miss the clue here]? Speeds vary greatly, but can we influence the results in such a way that we get the best solutions. Next question imo would be can the linux open source experts find a work around and insert it in linux so that this is a thing of the past.

[@itaco8066]

Great Video !!

[@lucalac3329]

It is interesting and also somewhat unexpected. I have the feeling that the drive fs plays a role here...you are copyng from ntfs to ext4 I guess right? Can you try to mount a local ntfs partition on that Linux server and try to copy again? Thanks

[@AndersJackson]

I doubt that the file system are a problem here. It is probably network protocols (NFS vs SMB) that is the problems, and a bad NFS client implementation in the MS Windows kernel. Still, when it comes to NFS client in MS Windows, it can probably be a bit faster with some options, like async instead of sync.

[@davocc2405]

that's GOTto be something to do with the MS NFS implementation - also can you try running WSL on the windows box and in that bash session try to NFS mount from that - and see how it goes? Another good test of overhead is FTP - it's very light, allegedly the lighest of all but the servers aren't always great for it. I'd be interested to see your FTP speeds from windows, from linux and also ftp on multi-stream using LFTP on a unix box with the pget n=15 options.

[@AndersJackson]

No, please. FTP is an old obsoleted protocol. Should be droped in favour of Sftp ot SCP. It doesn't handle owners, rights etc in a proper way.

[@davocc2405]

@@AndersJackson FTP is still very much relevant on closed or low risk/impact networks - it's still in very common use in data centres where you have a closed or segmented off network and speed is of top priority. For the purposes of this exercise the test is worthy given the objective is to assess throughput and CPU load with a view to working out the overheads of the various daemons.

[@LiRaye]

So same fs would be more efficient?

[@AndersJackson]

Probably marginaly with different FS, but the speed different in NFS -> NFS with Linux clients. And not so much MS Windows -> NAS, whatever protocol and fs you uses.

[@darenthoman9962]

Wow is there a way to improve the speeds from windows to a Linux OS?

[@BrianThomas]

TCP windowing. You can check your TCP window size and adjust it to make the throughput more efficient. Windows 10 should be doing this for you automatically, but you'll know how that works out at times.

[@BrianThomas]

Oh you can also increase your MTU frame size as well. Sending a thousands of 64 byte size frames is way less efficient than sending 4864 byte size frames.

[@henryincalgary6959]

Did you try winscp and scp on linux

[@AndersJackson]

Those are not file systems, not even network file systems.

[@mikehensley78]

Those are protocols. Scp is basically using SSH protocol to transfer a file. Has encrypt/decrypt overhead. Its CPU intensive. I use sftp/scp everyday. If you have a weak CPU on either the sender or receiver you will notice a slow-down.

[@pdx2240]

Funny, I have a HDD attached directly to my ASUS AC68U router via USB, and via SMB, I hit a router-CPU bottleneck on 20MB pr second. If I then also use the built in open VPN built in and via open VPN and SMB, the transfer speed is about 2.5MB pr second, or 10 times slower. Im on a 1gbit internet line on both sides, so no internet bottleneck.

[@mikehensley78]

VPN adds a layer to be decrypted by your routers CPU. Routers usually have worse CPUs than an old cellphone. Makes sense it would slow down. Smb 1 ? Smb2? Smb3? Protocol matters.

[@KENMUOKI]

Very interesting, could it work with pi3...

[@ionescuig]

If this works with rpi3 and rpi4... is awesome

[@NovaspiritTech]

Yup nfs works for raspberry pi

[@AndersJackson]

You know that Raspbian is actually Debian, which is what also Ubuntu is based on. Yes, as long as you don't run MS Windows on your Raspberries.

[@ionescuig]

@@NovaspiritTech I tried but.. not even close enough: - eth0 raspberry pi4 with nfs server and usb3 hdd - wifi (next to router) ubuntu laptop with nfs client - max speed 16MB/s Maybe I am doing something wrong. Do you have or can you make a tutorial about this for guys like me? I am using rpi4 for a few things and NAS is one. Would be nice to have a good transfer speed... Thanks!

[@AcielGamingTech]

Hi Don, I really like your channel and I watch all your videos. Your videos have hooked me into the world of SBCs hehe. Here is a question I would to ask you. Recently I lost my RPi3B, which sadly died. Now I'm planning to buy another SBC keeping the budget friendly, here are the 3 I'm confused between: 1. RPi4B 2GB 2. NanoPi M4 2GB or M4V2 4GB I want to build headless NAS (non RAID for now) and other dozen services based on docker & non docker. Which one should I go for? Any suggestionz/recommendations? Thanks

[@anant0089]

dude speak in simple english what is transfer speed on smb and what is nfs, i get 20MB/s on smb i think i should get more.. using all gigaport etc..and all top notch stuff money can buy ..

[@williamrutter3619]

Can you put something with a bit more hp in it, that device does seem slow, you would struggle to watch a high quality video from it, interesting video though, good work.

[@AndersJackson]

Well, the hp in the server was enough to saturate the net with NFS - NFS between Linux machines (Ubuntu - NAS). Not so much with MS Windows on client side.

[@zilog1]

Sounds like windows being windows :p

[@jarisipilainen3875]

windows filetransfer is slow.it sure doing more than just sending file.it still use window filetransfer even use nfs or smb. try replasement filetransfer for window. teracopy? fastcopy? extremecopy?

[@JeffSmithInStJacobs]

Further proof that you should dump Windows and just stick to Linux...

[@johncnorris]

Windows is broke.

[@kaiserschnitselsr]

so is your grammar intelligence

[@ukaszzywczyk3855]

MTU and jumbo frames

[@jamess1787]

Weird. I'm perplexed

[@Diegura250]

Simba :-) haha

[@ХоўелРейес-з5в]

Never trust the rate calculation

[@lawrencedoliveiro9104]

So Microsoft makes you pay extra for an inferior implementation of a feature that Linux does better, for free? I’m shocked. Shocked, I tell you!

[@ukaszzywczyk3855]

jumbo frames are the answer

[@tommyb.6064]

Really? so windows is sending thousand of miniatures frames that are harder to process for the NAS and linux to linux simply has larger frames "or packets" being sent ? is that a setting that can be adjusted in the registry of windows? That's a good point only you lifted up!

[@NewsBroadcasting]

sshfs is so unreliable, and takes up so much CPU and locks up applications

[@souravbiswas-tk3se]

Interesting

[@dmbrv]

interesting

[@Tripnode01]

please stop you have a face for radio but not the voice !

[@nekilof-2363]

I feel like this video could've been much better edited - like take out the bit where you're correcting your slashes and I don't understand the beginning bit about the system build...that's not the purpose of this video.

[@barrykirkby9626]

File transfer to a usb3 seagate 12tb drive sucks... Don't matter the file system.