FreeBSD vs Linux: Which ZFS is Faster? 🚀

[@CyberGizmo]

For the blocksize I said bits, its not bits its bytes...yeah it was late so a recordsize of 128K is 128K Bytes not bits...

[@eugenesmirnov252]

I don't understand. Who uses 128 today? 512 for (old) ESXi 1M(up to 4M) for a "modern" storage.

[@Wandering_Horse]

I'm just here for the awesome graphic of the BSD dragon facing off with the Linux penguin!

[@MatthewHolevinski]

same, that thumbnail was pretty dope

[@snap_oversteer]

Nice comparison, I have always assumed that FreeBSD's ZFS is 'cleaner' as it is more integrated but I have been running ZFS on Proxmox for years without any issues.

[@ridcully]

Been wondering about this performance comparison for a while now… great topic!

[@CyberGizmo]

thanks!

[@mrb180]

everytime I watch your content and see you explain things, I'm filled with excitement and joy. may you live forever, old school wizard of Unix. I wonder how would Solaris 11 on SPARC fare if added into those benches.

[@CyberGizmo]

Might be interesting to try.

[@simondj8572]

I would love to see that test again with OpenSuse, CentOS included.

[@CyberGizmo]

Do you mean CentOS Streams? CentOS as we knew it is gone, no longer downloadable even. Closest to it is RockyOS, AlmaOS or EuroLinux...

[@dogzdontzbarkz6540]

Today is a very nice day to watch your video indeed

[@AK-vx4dy]

@13:00 It is not exactly DB load, databases have to kinds of activity: one is log or similar, quite sequential *but* with confirmation of write, second is data or indexes and here it depends on database, one can be quite random but also with confirmation on write (typical sql), quite sequential write w/c (copy in write or append only strucutured - newer databases) or sizeable chunks quite sequential (index reorganisation or commit pages from memory)

[@CyberGizmo]

“Great insight! You’re absolutely right that database workloads can vary significantly depending on the database engine and workload type. In my ZFS benchmark, I focused on random reads/writes, sequential writes, etc.]. This covers some typical database I/O patterns but certainly doesn’t encompass all possible database scenarios. Testing more specific DB-like workloads, such as transactional logging or index reorganizations, would add valuable context. Thanks for pointing this out!”. I can also think of about 5 additional workloads neither of us talked about, maybe if this video was about databases I would do that, but it isn't its about baselining two operating systems running the same version of ZFS.

[@WilReid]

Was the drive similarly filled during the tests? You may or may not know, WD does a neat trick with their drives, when they're

[@seedney]

Will you do what's new video on FreeBSD 14.2?

[@CyberGizmo]

the slides are wrong the tests for FreeBSD were done on 14.2

[@Vamooso]

You are doing the people's work!

[@DV-ml4fm]

I trust zfs on freebsd more.

[@simondj8572]

How so, does it has better support on hardware and security updates?

[@CyberGizmo]

I ran ZFS on FreeBSD initially for about ummmm 5 years I think, worked great no issues, I switched it over to linux to ease admin tasks on my servers i think about 6 years ago now, it was a little shakey at first, almost gave up and went back, the merge was still underway, but they corrected the bugs and its been smooth sailing ever since those initial bumps you always get with a major update. I have even exchanged zfs pools back and forth with FreeBSD and no issues.

[@DV-ml4fm]

@@CyberGizmo Thst is cool. I'm not a zfs expert as I use freebsd as a desktop. I just know it works. 🙂

[@georgH]

I tried doing something like this earlier this year and using Phoronix test suite, but it was taking forever to run. I was trying to measure how ZFS changed since the early OpenSolaris version until today's Illumos as well as BSD and Linux. How long did it take to run for each OS?

[@guilherme5094]

👍Thanks DJ!

[@CyberGizmo]

welcome!

[@danwilhelm7214]

I'm confused. Were your tests done on bare metal or in a vm?

[@CyberGizmo]

baseline test was done on bare metal

[@haystackdmilith]

I wonder how better for FreeBSD is on 14.2. A lot of performance improvements for ZFS in this release

[@CyberGizmo]

uhhhh, I tested on 14.2 :)

[@haystackdmilith]

@@CyberGizmo Ok, then it's just 14.1 on all the benchmarks :) Thanks

[@CyberGizmo]

@@haystackdmilith oops, forgot to update the version thanks for the heads up, it was late when I put those together

[@haystackdmilith]

@@CyberGizmo No worries, the amount of work you've put into this video is clear. Keep it up!

[@haystackdmilith]

​@@CyberGizmo Also… Guys from Phoronix should learn from you how to make benchmarks. I like your approach very much.

[@jholloway77]

It always blows me away how well FreeBSD compares to Linux when you consider the resources behind them. Even when Linux can do something better, FreeBSD isn't far behind. They've got some great developers behind them to do so much with comparatively little backing.

[@nmihaylove]

With a COW filesystem you never have file rewrites. Each write is a new write. That's the whole idea of COW. However, with a recent version of ZFS (don't know which one exactly) when you overwrite a file with (mostly) the same content, it will perform a limited form of deduplication. If you are trying to write a block (i.e. record as in the recordsize dataset property) with the same content, it will reuse the old block rather than blindly writing a new copy as has been the case in the absence of explicitly enable deduplication. But there is a catch. It will only do this if you are a using a strong checksum algorithm. It's turned off for the default Fletcher. You need to set it to sha25 at the very least. So it's very important to have the same settings for both platforms. You need to provide the full list of the zfs parameters and make sure there are no important differences between them. Also, it's a good idea to have the tests for different record sizes. Do you have the scripts with which you test availabel somewhere for downloading?

[@CyberGizmo]

So first of all the benchmark is not completed, I said that at the beginning of the test. As for the scripts, not those specific ones, because they are not done yet. As for the rest of your comments thank you, I know all that already this isn't my first rodeo with benchmarks been doing them at the engineering level for large corporations for 50 years.

[@michaelheimbrand5424]

Thanks for the interesting topic. It would also be nice to see a similar test of network performance between FreeBSD & Linux.

[@CyberGizmo]

Yeah I probably will get to that at some point during the new Lab build, gonna be awhile will take me some time to get there.

[@tonywise198]

Interestingly, ZFS has been removed from the installer of Linux Mint 22.1 Beta (just released). It was not used enough to justify the effort of support.

[@nixtoshi]

Great comparison! I think linux should generally be a better OS for web servers and even application servers because more programs will run on linux without any translation or modification, because freeBSD is less common. Only very large organizations or applications with heavy write-loads would benefit from switching part of its system architecture to freeBSD for the gains in write performance imo. My only memory of using freeBSD was pleasant though, I installed it on an old computer with a GUI and it was very fast, it felt about as fast or faster than Debian with GUI.

[@tsiiphsycoii]

Auditioning for santa at your local mall? 💪

[@MatthewHolevinski]

I'm surprised you didn't use Oracle Linux

[@torsmork]

I love FreeBSD. ❤️‍🔥🔴💻

[@flow5718]

What's the reason behind FreeBSD being faster for writes though? 🤔 Sounds like tweaking some kernel parameters should fix it on Linux.

[@savagepro9060]

Possibly the reverse too. Meaning, what makes Linux faster on a certain matrix should be a kernel tweak for FreeBSD, right?

[@CyberGizmo]

Could be a number of things, I am planning to do a post performance tune video so will be diving into why the difference between them, could be just a simple thing like drivers for the nvme

[@MaxUgly]

Do btrfs on both oses! I feel like this concept (bsd vs linux) could slingshot your channel like it was strategically flying past Jupiter!

[@glynnec2008]

My takeaway from your video is that Linux *reads* are faster, while FreeBSD *writes* are faster.

[@CyberGizmo]

Yep, that's about right.

[@gshingles]

That FreeBSD fwrite result must be hurting it's mean. That just crazy. I was changing my mind constantly watching along with the results. It definitely depends on the use case, which is unfortunate...

[@mikejakubik]

You want reliability, performance, years of testing and OS integration, choose FreeBSD. It's included out of the box and supports root on ZFS.

[@Mitsumata]

I use NixOS with ZFS btw

[@ivanmaglica264]

Doesn't ZFS do it's own scheduling and memory management? For Linux I'm (pretty) sure it bypasses OS buffer and cache and uses it's own... It often competes with memory with VMs on Proxmox, it does not want to relinquish the memory as fast as Linux's buffer and cache does.

[@CyberGizmo]

In Linux, ZFS manages its own block I/O operations at a high level, but the actual low-level block I/O operations are ultimately performed by the Linux kernel.

[@AlmeuP5Dro]

Thanks for conclusions. But for a beginner into Unix ?! - Mac...( so expensive 😀) , I would choose Linux ( I am a newbie ) . Tried to install FreeBSD but was dumped at a CLI terminal ... 😄 Ok, I remember that I found a BSD distro , who has GUI ( installer + DE ) , GhostBSD ? ( not sure ) , which also could boot the laptop , with UEFI Secure Boot activated ! , well the App Store was not so big comparing with Linux ...

[@darukutsu]

why would you need uefi secure boot?

[@CyberGizmo]

A valid point

[@lunisamazigh4386]

so, BSD needs GNU ???

[@koskos758]

Not fair comparison - ZFS is not native for Linux - it is not even in Kernel!!! And still doing well.

[@StaffyDoo]

This, word by word 👌

[@hariranormal5584]

yeah but that's assumed, how else will you run ZFS in Linux, if there's a better way or a implementation then it should be obvious

[@eugenesmirnov252]

Bummer. I also for FreeBSD with arms and legs, but Linux is faster. Look to FreeNas. Who will use Core today? I won't. Anyway if all it was under PVE, what's a story about CPU? (should be host, in my view, we don't want to measure how good KVM or q35 handles I/O, right?) I just want to ask if any chance to see configs? And too much possible interferences hard to avoid. So here is like (as always) and build up your ceph cluster please, it's time for me to study this thing. Be well for all of us!

[@gdotone1]

so, linux is faster over all. I wonder does MAC OS suffer the same problems as FreeBSD?

[@CyberGizmo]

Good question and I do not know, I was going to try installing 2.2.3 on MacOIS sometime today just to see what happens

[@gdotone1]

@@CyberGizmo I wonder if the scheduler harms that speed by a good bit. I look forward to any results you find with MacOS. good luck.

[@Mitsumata]

@@gdotone1Apple has a closed ecosystem, the point of pulling ZFS into macOS? Just use APFS, lol. You'll only make things worse, you won't get any sane support, including from Apple.