Making the (partial) Rust BitTorrent client more reasonable

After doing the BitTorrent CodeCrafters challenge in #rustlang ( • Implementing (part of)... ), I had an urge to do some re-organization to better fit what a real implementation would look like when it has to handle files, multiple peers, multiple pieces, concurrency, etc. So, that's what we did!
If you want to give the whole challenge a try, please consider using my refrral link: app.codecrafters.io/join?via=...
0:00:00 Recap
0:11:18 Writing what we want to end up with
0:21:28 Writing the interface outside-in
0:36:56 Grabbing peer info
0:42:44 Interacting with a single peer
0:56:10 Decoding which pieces a peer has
1:34:11 Moment of hesitation
1:39:15 Ordering pieces
1:53:11 Connecting to a pool of peers
2:05:10 Grabbing pieces one by one
2:20:09 Cooperatively download all the blocks!
3:11:23 Handling choking
3:28:06 Putting together all the pieces
3:35:29 Fixing the one bug
3:45:23 The many things that remain
Stream diff: github.com/jonhoo/codecrafter...
Live version with chat: kzbin.info8sEdv0G1VIE

[@twistedsector]

Your videos are invaluable to the intermediate Rust community, Jon!

[@elgusanito6991]

nice thumbnail

[@HowDoYouUseSpaceBar]

😩

[@elgusanito6991]

​@@HowDoYouUseSpaceBar😩

[@sifiso3223]

1:53:13 - the thumbnail

[@tjoey6663]

really fascinating stuff, hoping to see more like this

[@CompletelyCovered3]

A sequel to the famed classic!

[@haydenridd]

Thank god we replaced all those nasty single digit "8"s with a scoped constant and two nested casts

[@drvanon]

One architecture optimization (dont really know what to call this) i would make is to have a function called download block range and have that be where you deal with all the complex logic that you dealt with in downloaded. Then download all would just call the entire range and for any specific set of files you would figure out what blocks you need. Now, becuase of the possible overlap of pieces across blocks, it is possible that you download a block multiple times in a download all. Edit: or maybe I am misunderstanding how files/pieces/blocks work.

[@JakWai01]

Just wanted to say thanks for having made my Rust learning experience in the past really enjoyable. Totally unrelated question, but what kind of zip neck sweater are you wearing?

[@pdd9366]

The thumbnail lol

[@driedurchin]

How would you go about testing this implementation in a robust manner? Seems difficult to create all the edge cases of disconnections, retransmissions, etc.

[@illeatmyhat]

gut feeling tells me you want property-based testing, but agreed that the unknown-unknowns can be a real headache here.

[@freemasry-gr8hw]

could you make a video on implementing the Bittorrent DHT Protocol ? I would really love to see that on your channel :)

[@vaporif]

What did you mean by referring to async_channel as smoll-rs based one? It doesn't depend on smoll-rs runtime.

[@Nunya58294]

Although I don't code in Rust this is really fascinating!

[@codeplay4947]

Your colorscheme/theme is really niceeeeeeeeeeeeeeee!!!!! what is it, (it's not gruvbox) i checked your config files but cannot implement the same in mine, how can i get a theme like yours....

[@alphabitserial]

I'd love to see what a 'from scratch' impl would look like as opposed to using (mostly) just the crates from the Codecrafters challenges. One example I could think of is using nom for bencode instead

[@julienleclercq6809]

Isn't nom quite slow for a production software? Always read that it was more of a quickly prototype, don't care of parsing kind of thing.

[@alphabitserial]

@@julienleclercq6809 Not that I'm aware of, it's zero-copy by design and usually almost as fast as a hand-written C parser

[@julienleclercq6809]

@@alphabitserial Ah. yes indeed, I thought this was slower than serde, but it's actually quite the opposite. Thanks for the heads up!

[@alphabitserial]

@@julienleclercq6809 Absolutely!

[@raidensama1511]

🥤🍿

[@Dorumin]

5:12 what was that voice

[@simbiont777]

self.offset in DownloadedIter equals to 0 and still unchanged or I missed something?

[@driedurchin]

Saw that too, only half-way through though.

[@jonhoo]

Good catch! Fixed in github.com/jonhoo/codecrafters-bittorrent-rust/commit/27aeebfb1d9ff2e9294406b332a76ef71e38f4c1. Not caught by the CI I think because CodeCrafters only has single-file download at the moment.