Implementing (parts of) git from scratch in Rust

[@yotubecreators47]

Jon's channel is my favorite place on internet and earth thank you Jon please do more of these

[@laurenlewis4189]

Not only the best resource for learning Rust on the internet, also one of the most clear and easy to follow programming tutors on the internet. Thanks for all you do, Jon!

[@driedurchin]

The cross-platform gymnastics git has to do is truly spooky.

[@DotDager]

Hey man, just wanted to say I love your stuff. This kind of stream actually sold me on streaming CodeCrafters myselft, and on playing with Rust as well. Keep it up!

[@MohitAgnihotri5]

Thank you for the video. I am writing in the Rust from last 2 years for commercial company. It's a coincidence that my name is previewing in your video @01:09 "mohitagnihotri", I have implemented this challenge own self (no external reference) and now comparing my solution with yours. I learn not only from your way of Rust code implementation but I also take some lessons from your thought process. Really very beneficial for me. Thankful to you!!!

[@jordanyates3349]

Another neat thing to implement might be an Entity Component System like FLECS / shipyard / etc

[@angelstojanov2346]

I second this

[@flwi]

This blows me away. I thought there is more "magic" in git. Nice to learn how it works. I'm currently learning rust and had a good time following your explanations. You have a very good style of explaining things!

[@cooldude3010]

**british DJ voice** one of the best one of the best more of these building complex systems long form videos please bossman

[@clubpenguinfan1928]

Wait, you're one of the missing semester folks? Holy crap you're the goat

[@kenzo3477]

Jon, a walk through the actix framework would be nice for someone like me trying to understand the actor model

[@draakisback]

Actix's actor model is not a great representation of most other actor models. It's got some really weird limitations and it's difficult to build proper supervision trees with it.

[@codingcoyote]

Haven't watched it yet, but as someone still getting acquainted with rust and craving a deeper understanding of gits internals, I look forward to seeing what you've done.

[@danygagnon8446]

I watched all of your videos and they're so well made. Thanks so much for this content. You are amazing!

[@Benabik]

3:19:10 The write-tree ordering was bugging me, so I looked at what git.git does. The ordering of the tree contents is just based on comparing bytes (strcmp), but they are sorting the entire index which has entries like "src/commands-rs" and "src/commands/write_tree-rs" (using dashes instead of dots to avoid becoming links). So the ordering in the tree object includes a phantom "/" at the end of a directory name. Ignoring the index probably makes the examples easier, but make actually working like git much harder.

[@Benabik]

3:33:04 I think the comparison logic you looked up is trying to emulate that without building the index. But you can see the place new entries are added in add_index_entry_with_check in read-cache and that is just using strcmp. You can see the result with `git ls-files --stage` which shows that ordering.

[@adamszalkowski8226]

Oh, thank you! Just found your comment. Had the same theory - thanks for looking it up!

[@SimonBuchanNz]

Thanks for looking this up! Implicitly ending with a slash actually makes a lot of sense... though I'd prefer that the UI showed it!

[@Joqer88]

This is like crack to me, I can't get enough! Not only do I learn more Rust, I learn more stuff about git! I wish this challenge would continue until a fully featured git clone (😉) would be implemented!

[@JaLikon65]

Great episode! Not only did I learn a bit about Rust, but I also learned quite a bit about git! Nice :)

[@puolisointu]

I was inspired by this and did the redis challenge. It was super interesting and I learned a lot. It would be awesome to see Jon have a stab at it the "correct" way. I have a little bit of trouble of finding correct mental model of TCP and how to parse it incrementally without excessive cloning, it's hard to search youtube or google for good information. If you have any recommendations for material I would be interested!

[@LesleyLai]

Nice content! My implementation used a lot of in-memory strings compared to your streaming implementation. This is definitely very educational!

[@microcolonel]

I love to send this kind of material to people. I'm thinking of doing my own spin on these types of challenges, where we start with a discussion of the problem and datastructure and dataflow design, and then build in a way that is *fast* by the time the tutorial is done. For example in this or the bittornent one, architecting for io_uring and mmap, instead of high overhead POSIX stuff.

[@steffenuhlig1970]

My brain already hurts after watching just about a third of the video, but it _is_ amazing. Thanks for this!

[@dbarenholz]

Another voice in the aether: these videos are awesome. Thanks for making them!

[@drvanon]

I wonder why your abstractions returned out so awesome. I notice that often when I make my own abstractions they often do not benefit me as much as i do see n this video

[@TheAulto]

That may also be your case, but he doesn’t seem to abstract until needed. Your first guess it probably gonna fail, so just inline everything and wait for a clear second candidate to pop up, evaluate and implement. Especially with an already working state, it should also be easier to pull off.

[@SimonBuchanNz]

The general answer is unfortunately "get good" - there's lots of rules of thumb and guidelines and blog posts and everything else telling you how to do it well: and none of them seem to really help all that much because you don't know which to use and when. For me, it's all been hard-won experience over to decades, and I think I'm still not as good as Jon. Ugh. 😂

[@dantenotavailable]

Interesting note: not sure about the rest of the plumbing but `git cat-file` itself will ignore the size value and print the entire rest of file. I used the same basic idea you used for your type (i.e. read one byte extra and compare how much was read) and created an object with a mangled size for testing. Tried it on both, mine worked, proper git printed the entire file.

[@ruijieyu9035]

3:40:22 it seems that ultimately the comparison rule just says append / to directory names while sorting, and what the git function does is to ensure that no new memory allocation is needed

[@driedurchin]

AFAIK Windows has no concept of an executable bit. On windows, under git-bash, it has a series of heuristics that it uses to guess if a file is executable, but they don't work well. So much so, there is a specific git subcommand to directly set its file-mode flags because you have no chmod on windows. P.S. One of the heuristics it uses when you run ls -la on git-bash is to actually check for shebang by reading the file.

[@adamszalkowski8226]

Maybe the directories in write-tree have a silent slash at the end?

[@Gh0stwrter]

great video man, it was the first time I came across your channel. I've went back to C to try and gain a better understanding for Rust which is helping and I go back and forth between the two a lot. Kinda like Neovim and Doom Emacs, I use both for some reason and both are great but your neovim theme is a little different then the grobox type themes I've been using. I like that comments are orange. Would you mind sharing what theme your neovim is using and is that feline status bar? also what lsp are you using for rust in vim? I ask becasue you are not getting tons of errors outputted recommendations when your first starting out and that always bothers me.

[@jonhoo]

Hey there! I discuss a lot of those things over in kzbin.info/www/bejne/rp7Uoa2mr8mdf5o :)

[@Gh0stwrter]

@@jonhoo awesome thank you for the response.

[@f3arbhy]

Great content! Couldn't help but notice that you have your browser search bar on the bottom - can't say i have ever seen that 😅

[@AnubhavAndReena]

Awesome

[@8ack2Lobby]

dude you are amazing! 💝

[@SimonBuchanNz]

Surprised the tree sort didn't just use collect(). I suppose you can't distinguish the errors?

[@damoon_az2465]

Thanks!

[@timvw01]

Great stuff

[@leob554]

where is he running vim on?

[@pathakvivek7865]

Hey, what is this theme? Can i get it on VSCode?

[@svenyboyyt2304]

Could you maybe try to implement the basics of Vim/Neovim in Rust?

[@georgiossmyridis5088]

What does he mean "this allocates"?

[@SOMEWORKS456]

What are you using for tab completion in terminal ?

[@SimonBuchanNz]

I assume it's at least close to the default fish config, which is pretty nice.

[@iam.pastel]

can you share your vim config?

[@mpete0273]

What colorscheme is that? Love it.

[@jonhoo]

Gruvbox dark hard :)

[@fabioc9142]

Please do a decrusting rayon

[@Dygear]

3:31:07 - 3:31:15 --- Clip it. Meme it.

[@thehady1]

I'll also wait one day with zig

[@MrEnsiferum77]

Just fix Git, with this alternate version, on the way...

[@DrOptix]

I stopped to comment when you arrived at the sorting issue. I'll make a guess that the answer is that they first list the elements of a directory, then the directory itself. Now that I commented this I'm looking forward to see if this is the answer to the problem.

[@DrOptix]

OK I did not expect them to have that approach. I'm still trying to understand why go to such lengths just to sort stuff

[@Dygear]

God I hope Code Crafters adds a dark mode. Getting smacked in the face with white backgrounds when swapping between the editor and the browser is giving me the squints.

[@Dygear]

Also, never realized that std::fs::create_dir_all was a thing. That's very handy!

[@minatonamikaze2637]

the darkreader extension

[@jimmylarsson5425]

What is the inline hints for function arguments called prepended with a panda and the defeninition above?, for example at kzbin.info/www/bejne/q2G5oKerfN93ldk

[@jonhoo]

You know, I honestly couldn't tell you 😅 I think it shows the current argument type, but no idea what has enabled it!

[@jonhoo]

Ah, looks like it's github.com/ray-x/lsp_signature.nvim specifically the "hint prefix"!

[@jimmylarsson5425]

@@jonhoohuge thanks!! also thanks for all the videos, filling the void for us above beginner level but not professional rust users :D

[@selvarajs.7494]

Those who are struggling in Windows os let mode = if meta.is_dir() { "40000" } else if meta.is_symlink() { "120000" } else if is_executable(&path) { // use is_executable::is_executable "100755" } else { "100644" };

[@chaitanyasharma6270]

please do git clone

[@AidenPearce-i6d]

implement udp next time

[@Greemjou23]

8:40 - actual start of the video thank me later :)

[@jonhoo]

The chapter marks already have that time code in there (and other useful ones!) :)

[@regisk7]

high level, zero cost abstraction and yet we are reading and writing in buffers like cave men... doesn't feel high level at all, other than useless verbose keywords and method calls...

[@SimonBuchanNz]

You mean passing the output to read calls? The whole point is these are the low level io interfaces used to reduce memory usage and improve throughout, if you want the high level APIs just fs::read(path) and you get all the bytes, or fs::read_to_string(path) for UTF-8; or use Read::bytes() to get an iterator you can use all the goodies there on. It's not like this is any different to how, say, Node does this, either, with the exception that you don't have a resizable buffer type yet (they're adding one for WASM memory), so if you, say, wanted to read to end you need to end up doing all that reallocation and copying yourself, or keeping and array of buffers and concatenating then at the end. Not fun. Can you say what verbose keywords and method calls you're referring to? Do you mean individual names or just that there's generally a lot more used than in other languages? There's definitely an argument that Rust can get too verbose; but in my experience that's tricky to actually compare; for example adding .with_context(|| ...) to each call seems like a pain in the butt until you try to do the equivalent in other languages: most of the time it's a wrapping try .. catch throw of an error, and it's often tricky to get it to give readable output. JS did recently make this much nicer with Error accepting a cause option, which I really appreciate, but it still is missing expression throws and trys.

[@damoon_az2465]

Thanks!