Nix explained from the ground up

[@zyansheep]

I sure do love it when a random yt channel with under 1k subs and a celeste speedrun posts an in-depth technical video explaining a complicated subject intuitively!

[@vikingthedude]

I think this is surma from google! Hes one of the hosts of the http 404 podcast. Jake archibald is another. Looking forward to his videos

[@CoDEmanX47]

​@@vikingthedude It's HTTP 203 but yeah

[@tejing2001]

I like how you presented this. Building up from the basics without getting too bogged down in the details. I do have a few notes, though: 8:49 This is actually (unfortunately) not the case. Antiquotes of path types do expand to a nix store path, but builtins.toString actually returns the non-store path as a string. I consider this broken behavior, and have argued that it should be changed, but it is the current behavior. As a rule, just never use toString on a path-type. If you think you need to, then you're probably mixing eval time and run time in a way that will come back to bite you at some point. Also, it isn't exactly when a path-type is evaluated that it gets copied to the nix store (in particular, if you import a path type, it needn't be copied, and isn't), but I'm pretty much fine with glossing over that. 13:07 Yes, you did a bad thing, but perhaps more importantly, on linux, this build would have completely failed because there, builds are run inside a container without access to the host filesystem at all. Might have been good to mention. 21:12 You did a subtle bad thing here, by overriding phases without calling the pre and post hooks. This keeps the hooks from firing, and can lead to hard-to-debug behaviors as a result. This is one of the biggest causes of frustrating confusion I see for people writing their own derivations. Some construct that depends on hooks is supposed to work, but doesn't, and they can't figure out why. It's cause they overrode a phase and didn't use runHook to run the pre and post phase hooks.

[@dassurma]

Oh cheers! Really appreciate the thorough review. I’ll add those to the video description. The remark about the hooks is interesting. That was something I definitely misunderstood and I suspect did cause a lot of headaches lol. Wrt containerization: I actually had that on my TODO list to check if you can have your build run in a container, and I never got around to it. Can derivations opt out of the containerization? Or how does bootstrapping work on Linux?

[@tejing2001]

@@dassurma Containerization is a nix option, controlled in nix.conf or on the command line with --sandbox or --no-sandbox. It's off by default on darwin, I think because the sandboxing tech available on darwin causes too many problems, but on linux it's on by default. The sandbox only has the closures of the store paths mentioned in the .drv available in it. Nothing else. As far as I know, there's no way for a derivation to skip containerization; if your build can't work in a container, then it really isn't a proper nix build in the first place, so it isn't really viewed as sensible to give that option, especially considering the security implications of building other people's (not necessarily trusted) derivations. Bootstrapping stdenv in nixpkgs (in linux or darwain, afaik) is done without reference to the host system, because it instead starts with a precompiled set of tools, downloaded much like source code is. Those precompiled tools are very rarely updated, and extra care is taken to ensure provenance when they are. I believe several separate groups build them independently and check that they get bit-for-bit identical results, among other things. They don't really need to be updated until you reach a point where the latest version of the compiler can no longer be built with them, anyway.

[@shvmoz]

Have been digging into nix/h-m over the last few weeks, awesome to have the algorithm send you to me pal!

[@dassurma]

Flattered that you spent your time on my vid :D h-m has been a game changer for me. It’s like NixOS with having to give up your normal OS. The weirdest corners for me so far have been to not be able to quickly edit my config files and having to do a special dance for nvm, rustup and similar tools.

[@TheSast]

​@@dassurma if you don't use configure those directly with nix, but simply use home-manager to place them in the appropriate location, you can use home-manager to symlink them instead, allowing for rapid iteration. You can then turn that on or off with a custom option.

[@opheon]

​@@TheSastif the files in the appropriate location are manager by h-m, wouldn't you need to rebuild with every change?

[@badluckprophet9103]

Always love a "I made the video I wish I'd had." Thanks for putting this together, it's very well done.

[@arnaudbarre6006]

I loved the content, the tempo and the story: the end result is a consequence of logic steps, not something you take for granted. I just found that the music was a bit too loud.

[@dassurma]

Thanks for the kind words. You are not the first person to point that out. I’ll keep that in mind for next time :D

[@alexpage8085]

Learned a ton from this video Surma. Thanks for summarising all of this!

[@pederpbsds]

A good balance between handwaving nitpick details and drilling down into how it all comes together!

[@mr.daniish]

The editing and explaining of this video is insane

[@bjeanes]

Excellent video. I'll be sharing this one when people ask me to explain Nix.

[@WeshalbDennNicht]

What a great and helpful video for anyone trying to get a first, or a second deeper grasp on what Nix is and how it works. Thank you!

[@Rypervenche]

Extremely well done! You probably don't use this, but if you ever do a video on NixOS, I'm here for it!

[@tyler...]

This is such a great video! I've never really understood the reasoning behind Nix until now, let alone how it works. Saving this one to my "most important videos" playlist

[@sirpiplin9948]

The video we needed, not the video we deserved! Amazing Job!

[@theutz]

I needed this a year ago! But I’m happy that it’s here now. :)

[@wildwestrom]

This is superb! I think this is the best tutorial I've seen yet.

[@DipsankarMaity]

Loved It. Keep the videos coming Surma.

[@TotallyURGrandpa]

Antichamber, Celeste, and now Nix. This guy knows ball.

[@xsharawi]

This video is so well made, I'm glad I watched it!

[@xorlop]

I just gave my first talk that explained similar concepts… but dang! Your explanation is amazing! This is just great!

[@hhhnnn9195]

Thank you soo much! This was the Tutorial new users need, and I felt was kind of missing by the official documentation.

[@At-Dawn-We-Ride]

Thank you for this very informative, nicely balanced video. The documentation situation for Nix is indeed not ideal, so I appreciate your work of collecting various distributed pieces of pertinent information and combining them into a "digestible" format. 👍

[@ivanicguildmember2844]

Elegant explanation. Much tanks for making this video. Could you also make a video explaining the machinations of NixOS too?

[@ejiek]

Thanks! That is an amazing learning experience! Pleasant motioncanvas graphics, well pased explanation, relaxing music and colors. I'm happy that KZbin got me here. Subscribed and patiently waiting for more =]

[@Mendez_84]

I've been hearing a lot about nix recently but haven't bothered to check it out. Maybe I will take a look now. Thanks for the video

[@kylemoran455]

This is great! Do flakes or home-manager next!!!

[@SinhNguyen-tz6us]

Great video. I loved your OTMT podcast.

[@JordanShurmer]

Thank you Surma. excellent video

[@meyou118]

best description about nix - great video

[@Majiy00]

Great video! look forward to the next one.

[@BGroothedde]

This is a really good video with a lot of care, thanks!

[@oskarboer1511]

Thank you for making this super clear explanation!

[@povilasn8239]

Awesome video !!! Wish i had it when i just started learning, but you still managed to fill in some knowledge holes for me.

[@YaroKasear]

Make more. With content of this quality you deserve more subs!

[@bryanhonof]

Cool, loved the visualizations!

[@ramonrames]

More videos please. ❤

[@hidayattaufiqur3232]

thank you so much for this video! such a great explanation, you're awesome!

[@roycrippen9617]

Dude sick video!

[@someonesalt5084]

Top tier vid

[@luckydye]

great pacing 👍

[@mathiswellmann6858]

Nix is truly the best thing that happened to the linux ecosystem. Been using it for 1.5 years now and loving it. But it has a pretty steep learning curve I must admit, harder than Rust or CUDA IMO.

[@carneloot]

I would love to have more videos on this series (this is now a series, right?). Personally I’d love to know more about home manager, but I’m sure there is a lot more to talk about on the nix ecosystem

[@abhishek.01]

Thank you!

[@leandronicolas1]

Someone please ask for this to be added to the wiki and awesome-nix repo. Awesome video!

[@SpidFightFR]

Hey! Awesome video !

[@EmmanuelMess]

If this is actually able to compile the Qt frontend, it would be incredible, as Qt is very difficult to compile replicably, even with Docker.

[@luv2code]

I'm curious about how it replaces docker on your servers? I understand that you would run some bit of nix software that is sort of pinned to a specific version - like a docker image. But docker is more than that. Does nix also provide filesystem isolation and a networking layer?

[@luv2code]

Thanks for the video, BTW. I appreciate you.

[@gungun974]

Personally I liked both Nix and containers. The isolation of files systems and network of docker is a pure joy and the reproducibility and sandbox of building of Nix is perfect. That’s why I used nix to build docker Image that I can just run with Docker. Nix is not a Docker killer. Nix is a Dockerfile killer

[@tejing2001]

Nix doesn't provide the security benefits of docker, no. But honestly, that's not the main reason people use docker most of the time. Also, nixos and similar systems can certainly set up containers with those security benefits, if you want, and the nature of the nix store tends to make it rather easy to create such containers.

[@leoschafer5172]

It's possible to use a nix derivation to build a docker container (so there are no problem is someone used :latest and the docker file fails). I don't remember where the docs for this are

[@pythonBlender7]

I'm assuming your a sysadmin who uses docker like I do. There's a thing called nix she'll or something that makes a temp env for a package. That package could be your software. When you exit, the shell and envelope are gone I think and it's basically ephemeral like a container. Also if you think about it, that docker networking layer is only there because it has to be. Even if you are good at it like I am it's still an extra thing to remember and EVERYONE gets but by it the first time they try to send something to local host right? We'll if it's just on your system then you suddenly get to use local host again. That's nice right?

[@danser_theplayer01]

Let me guess, nix making new users is the equivalent of me compiling and testing a program on a VM for OS that I don't have.

[@janedoe5877]

Love this

[@__vyre]

beautiful video 🙏 keep it up bro🫶

[@miquelvazquez4544]

Great video! Can you elaborate on how it compares (pros and cons) with docker?

[@MrAzulay]

Great content as always, Surma! I’m also genuinely curious to how you find it work as a replacement of docker on your servers.

[@RonaldoArch]

great ! Thanks !

[@khanra17]

9/10 -1 for placing the nix language syntax in the beginning part. Felt overwhelming and weird!

[@richcole157]

Really great video. It is great that nix controls file system dependencies, but it isn’t controlling as far as I can see for the OS systems calls or for variations in processor architecture. I guess a set of build machines in vms is a way to control for these differences, but leads to a massive proliferation of different configurations.

[@acriliqueofc]

Great video!

[@ZackMitkin]

Very good video, the music is very disturbing though. Any change you can re-upload without such loud music?

[@user-zk5hh4kq3k]

I really liked the video but can you tell me what did you used to create such a good quality video.

[@dassurma]

That’s the wonderful Motion Canvas :)

[@rakeshvenkat9971]

Brilliant!!!!

[@vipierozan99]

i video about flakes would be awesome

[@Kotz_en]

I love how you animated the code throughout the video. Is there a tool that performs these transitions automatically, or did you create them manually when editing the video?

[@cinderwolf32]

Motion Canvas

[@Kotz_en]

@@cinderwolf32 Thanks!

[@cinderwolf32]

I think we were sorely lacking effective presentation and communication techniques for highly technical information and concepts, and I'm continually impressed by how everyone utilises Motion Canvas.

[@andreujuanc]

Amazing, quick question regarding "placed docker on my servers". Do you do cgroups by hand? how do you handle sandboxing?

[@illdieanyway7865]

If only programs were ALWAYS static, just like Plan 9.

[@AvisekDas]

I love you Surma

[@yoyonel1808]

at 5:49 the error message in not related to the code: /* f = */ {a, b, c}: a + b + c f {a = 1; b = 2;} # error: function called # without requirement argument 'b' It's not 'b' but 'c' (i hope :p)

[@BboyKeny]

That's a bold claim indeed. Can it install Crisis on my Tamagochi?

[@cokoala5137]

I do love that you're putting out content on Nix. I just really don't like the 'trust me' part and would've rather you just summarised with 'it follows these installation steps' in rapid 3 word dot points just so there's a grasp of what's happening. The mindset for people even looking at this is probably to know what's actually happening rather than magic. Other than that, I just love more and more nix content. Appreciate it

[@cokoala5137]

btw I know that there's a lot to go through, so understandable if you just cba breaking down that installation process

[@dassurma]

@@cokoala5137 what it does is quick to list, what takes a lot more is *why*. And in fact, do get even half of the explanation as to why, you'd have to have watched the video, because it's about the remote caching and the security implications. In the end, I figured the installer is not core to understanding how nix itself works.

[@CristianHeredia0]

Are you building nix containers instead of docker , or using nix to build slimmed down docker containers, or something else?

[@sillystuff6247]

this is valuable information. please post a version of this video that does not included the distracting annoying background music.

[@xSkyeHD]

Did you try this with a devcotainer or nixpack deployment?

[@Sean_neaS]

Nice work. This might have changed my mind about nix. I can see some good use cases, now, such as building llama_cpp and pytorch for my intel cpu. As long as I don't have to shave my head and hand out nix pamphlet at airports.

[@SirSomnolent]

Is it mandatory that I dye my hair blue before installing nixos?

[@peppidesu]

Utrecht University W

[@ZackarySmith-x2j]

Davis Matthew Robinson Joseph Hernandez George

[@NullViewer]

In some way all this looks really similar to Bazel build system.

[@toxicore1190]

I really enjoyed the video, however, found the music repetative and annoying.

[@zungaloca]

I love nix and keep using nixos, and probably will never leave, but I truly believe nixos need a nix gui and Better defaults, flakes still experimental wtf, no normal user should touch nix code, the amount of degrees of freedom nix allows is why to much for 99.9% percent of people, that's why most people get burned, everything should be achievable with a sane default config

[@BogdanSerban]

Isn't nixos that distro linked to extremist political activism?

[@ICopiedJohnOswald]

Read Nix Pills

[@valcron-1000]

Would be great to have a video on Nix Flakes and how they're useful for reproducible dev environments.

[@danko95bgd]

Funny that they also pipe into to bash to install 🤣

[@jirkasvitil2762]

Firefox is not xlosed sourced

[@dassurma]

@@jirkasvitil2762 I never said it was. But it *is* distributed mostly as a binary.

[@sodiboo]

​@@dassurmaThey way you brought it up very much sounds like you meant it was, but yes the literal content of your words can also be interpreted as "Firefox and closed source software" as in two distinct things, when it does sound like you mean Firefox is within that group.

[@Tomyb15]

All that and you still bother using a Mac?

[@naughtrussel5787]

The concept of bit-exact builds seems a bit wasteful in my opinion. I come from Python background, where versioning is somewhat relaxed, and relies on author's POV. It can lead to bugs and lack of reproducibility sometimes, but simultaneously, if a code has changed, but does the same thing anyways, it simply keeps working for a range of versions. In an ideal world, I imagine the comparison of code-"exactness" by not how it is typed out in the source file, but if it still does the same thing and does not break an interface. Having "x_plus_three = x: x + 1 + 1 + 1" and "x_plus_three = x: x + 3" to be strictly different things is somewhat too much for me, and this is not a specific Nix issue to me, rather general nagging.

[@MarthinusBosman]

Working with Python a lot as well I kind of agree, but it depends highly on what you're building. Python code is usually for a lot more "dynamic" projects. But if you're distributing code to millions, or building infrastructure that millions build on top of. I don't even want to risk an extra space messing anything up.

[@naughtrussel5787]

@@MarthinusBosman hehe, I would not pick python for that then :)

[@lukafireman]

And then you build the OS as it's also a variable... This is such a paaaain to maintain. They could have just went with docker-like logic.

[@xyangst]

Not pain to maintain just maybe not for everyone 🙂

[@roulzhq]

Damn! full length tech videos by surma. 2024 turned out great!