"Type-Driven API Design in Rust" by Will Crichton

Rust is a unique programming language that blends imperative and functional concepts to make low-level systems safer. However, compared to other modern languages, designing APIs in Rust requires a fundamentally different mindset -- for example, designing without classes or inheritance. In this talk, I will live-code the design of a simple Rust API. Through the evolution of the API, I will demonstrate how Rust’s type system (especially traits) can be used to design interfaces that cleanly compose with existing code, and that help API clients catch mistakes at compile-time.
Will Crichton
Stanford University
@wcrichton
Will Crichton is a 6th year CS Ph.D. student at Stanford University advised by Profs. Pat Hanrahan and Maneesh Agrawala. His research is about applying cognitive psychology and programming language theory to understand programmers and to design better programming tools.
Recorded at Strange Loop 2021
thestrangeloop.com

[@paytonrules]

The fact that the audience could actually diagnose the issue at the end proves how well you taught it.

[@rafeu2288]

Also, how complete the Rust compiler messages can be ;)

[@G11713]

also, clear and captivating the presentation was.

[@markday3145]

Evolving the API step by step was much more educational than how documentation often shows a fully evolved interface and then trying to explain why each piece is there. Even though I have some beginner experience with Rust and iterators, I still learned a bunch (and some of what I already knew a little was solidified better). Thanks!

[@arisweedler4703]

The ANSI escape code at 9:00 is 2 commands: clear the screen & put the cursor at row1,col1. The grammar of these commands are: , literal open bracket, arglist (numbers and commas), letter. The letter is a function name. J is clear. H is “move cursor to position”. The 1,1 arg to H is self-evident. The arg “2” to J… idk what that does lol. And ofc the \x1B is just sending an ASCII escape character. It is the \xNN syntax where you can send a byte numbered by NN. (\x31 would be 1, \x41 would be A) (I think haha plz correct me if I’m wrong)

[@martinbecker1069]

This actually cleared up why using Enums for state machines (or state machine like things) aren't a good idea, because you can't implement a method for one of the variants. You implement it on the Enum itself which wouldn't solve the problem of the method being available in a context for which you don't.

[@dealloc]

Wow, this opened up my eyes even more to Rust's type system. I've gotten fairly familiar with Rust's type system already, but have barely scraped the surface. This talk is very well done and I learned a lot from this; especially the Unbounded/Bounded part is super helpful.

[@m.sierra5258]

Same. I thought I already know Rust well, but this opened my eyes to a new programming paradigm.

[@overlisted]

this talk is so well thought out, you seem to know every question that the audience may have

[@matteozampieri6392]

Fascinating: both the talk and the presenter, he’s a true teacher!

[@superscatboy]

"usize is a bit of an unwieldy name but that's what they picked" *[laughs in size_t]*

[@cblair1353]

Yeah, usize always seemed pretty clean to me.

[@davidvomlehn4495]

In C, you've got size_t. I'm thinking usize is at least as obvious. After all, to know size_t is unsigned, you almost have to know there is a ssize_t that's signed. Having ssize and usize is more consistent with the rest of the language.

[@Cty1011]

holy crepes Mr. Crichton, please do more of these talks. I have learned more from the first 25 minutes here then the last couple of weeks of hacking away at code. wish you would teach the basics (and advanced) with real world applications. My only minor issue (which I know can't be helped) was the pace but its YT... I can rewind my heart away :). Awesome!

[@enesgee]

so confident , that all the code is in /tmp - What an awesome talk

[@jRsqILVOY]

This is a great talk. Another great feature of the API presented here is that it works ergonomically out of the box for operations that need references, mutable references and ownership just by changing the Iterator that you send in to Progress - i.e. .iter() vs. iter_mut() vs. into_iter(), the Progress implementation itself is completely agnostic of T. The compile-time error messages can be an issue if you use crates that use this approach a lot though (as well as understanding all of the "helper" types like Bounded/Unbounded here, when there are dozens of them). It'd be nice if there were a way to add a custom compile-time error to a type in your library crate, matching against specific error types. e.g. if someone tries to call with_delims on the Unbounded case, it could print an additional final error message mentioning the difference and a link to the relevant docs.

[@SimonBuchanNz]

The standard library does this a lot, attributing lots of methods and types to have "did you mean..." type messages. Not sure if any of that is stable or on track to be stabilized.

[@lisovyy]

This is an awesome talk. I'm learning Rust while being myself a JS developer, and so far this is one of the most useful talks with code design in mind. Thanks

[@Notoriousjunior374]

You’ll never go back to javascript for anything else except only solely for the front-end usage.

[@notoriouslycuriouswombat]

Wow thanks Will, excellent communicator! The type state thing in relation to rocket for starting up a web server... can't tell you how many hours I've seen people (myself included) waste in web backends accidentally messing up order of things

[@flippert0]

Terrific talk, both in terms of subject matter and in the actual presentation. Mr. Chrichton is an awesome presenter and the fact he churns out so effortlessly code while talking to a crowd with great eloquence stuns me.

[@nirmalyasengupta6883]

For a moment, I thought I was in a _Scala_ session! Fantastic talk. Helps a newbie in Rust, like me, immensely. Things begin to fall in place. ☺

[@reecemcmillin6570]

Very cool talk! Feel like I’ve got a much better idea of how to utilize traits and leverage the type system in a more-than-superficial way now.

[@heliusuniverse7460]

Rust's type system blows my mind, im worried i might not be smart enough to use it well, but it does seem really useful

[@katopz]

I love his energy (also audience) and that missing Bound really illustrated how Rust is hard (in a good way). If that would be me on stage I would have been panic! and get kill by demo god already. Overall this talk is perfect (thanks!), please do more of this!

[@nilshaberstroh2705]

Fantastic talk! Really pedagogical indeed. But now to the real question: must one be a Crichton to be fierce in Rust?

[@elijahshadbolt7334]

Compared to C# and C++, it takes a lot longer to learn and fix problems at the start, but once you're over that activation energy hurdle, it provides a much cleaner way of writing performance intensive code, and the package manager (cargo) is way better than the C++ ecosystem.

[@Barnaclebeard]

25 years ago, when I was a studly Python developer in my prime, I laughed at people who tried to tell me that statically typed languages would be the future. I laughed!!

[@davenirline]

That's awesome! This is the future of API design.

[@ricardospear922]

Thanks for this talk. Barely getting into rust and while this lets me know I have a journey this was really educating and exciting

[@riebeck1986]

Thank you for taking this talk! This was extremely helpful. I have watched this video multiple times, and always end up learning something new.

[@anssietelaniemi3397]

Very good hands-on trait usage.

[@markday3145]

It would be convenient to have .progress() automatically do .with_bound() for ExactSizeIterator. Before adding Bounded and Unbounded, this was sort of possible by calling .size_hint() and making sure the upper and lower bounds were the same (which is required by ExactSizeIterator, but may be true for other iterators). I couldn't figure out how to do it with Bounded and Unbounded. I tried impl ProgressIteratorExt for Iter where Iter: ExactSizeIterator, but then I get an error about conflicting implementations of trait ProgressIteratorExt. Apparently, you can't use a negative bound (!ExactSizeIterator) to separate them. And it doesn't seem to know to use the tightest bound (ExactSizeIterator when possible, or Iterator otherwise).

[@EnjoyCocaColaLight]

Isn't putting exclamation mark before a variable something you do when dealing with bools to suggest a not-true of said bool? Rather than the negative (of a number, I'm assuming here. Because it's night time)

[@AbelShields]

@@EnjoyCocaColaLight it's also in nightly (I think) for negative trait bounds, for example you can do `!Send` meaning any type that doesn't implement `Send`.

[@sanderbos4243]

This talk is nuts, I learned so much!

[@MsHofmannsJut]

fantastic talk. new perspective opened. thank you, Will.

[@greyblake]

Very good and helpful talk! Thank you!

[@bovinovayoung1550]

Great talk! articulate and well organized! I totally enjoyed it!

[@michalbotor]

this is an outstanding talk. not only is it interesting but also puts a ton of rust features into practice.

[@eukaryote0]

A great speaker and a great talk. Enjoyed.

[@yvesm5455]

Great talk, thank you !

[@ZoltanTemesvari_temy]

Probably the only Rust video you need

[@mhadrouj]

Awesome talk from a brilliant teacher/coder! I wish he could have a series of videos that teaches beginners to advanced programming topics in rust/python/whatever

[@giuliocasa1304]

The problem with_bound() is that - for example - even chars() on a string does not implement an exact size iterator, which looks quite odd since a string has a bound number of characters

[@giuliocasa1304]

The ExactSizeIterator trait requires that the iterator can report its exact length in constant time. There is a subtle difference, the iterator would reasonably seem to be bound from a common point of view, but for the Chars iterator, instead, computing the length is implemented by iterating over the string to determine the size of the unicode characters (because the size of a Unicode character can vary depending on the character itself).

[@yourdadsbestfriend7101]

this is an amazing way to present and solve the problem

[@tanuvishu]

What an amazing talk

[@oxereviscerator]

What an excellent talk.

[@doomguy6296]

Love your explainiation! Much thanks! Let me just mark that calling a generic T type as Iter, was quiet confusing to me at times 😅

[@n8style]

awesome talk! thank you!

[@ShinigamiZone]

Fantastic talk 🤩 So much was covered in under an hour

[@LukasPukenis]

Great talk, amazing presentation:)

[@busydying]

`cargo check` would emphasize that errors happen on compile time

[@kavehtehrani]

Amazing lecture and very well taught!

[@vijaysinghbisht3016]

This was awesome. Thank you

[@detaaditya6237]

Rust is insane! I'd love to try it later

[@naveendavisv]

Rust trait makes more sense to me now

[@ajithpanneerselvam5070]

An awesome talk!!!

[@WilderPoo]

Wow, super talk!

[@ribhi7491]

Awesome talk!

[@spencerj.wang-marceau4822]

Mr. Crichton message: "protect API users from unintended usage at compile-time with Rust" He demonstrates example unintended usages then follows up with code tweaks to protect against those. He grows the value of designing APIs with Rust step-by-step. In the example code, Mr. Crichton incrementally adds different Rust grammar keywords surrounding Rust's "trait" keyword usage to level up the API unintended usage protection levels. He ultimately delivers on a designed API that fully protects the API user from unintended usage. Building reusable libs or crates keeping in mind Mr. Crichton's message will definitely increase lib/crate quality and ease-of-use.

[@blob4000]

Fantastic talk!

[@tofraley]

This was great!

[@EnjoyCocaColaLight]

19:30 You got rid of the hashset and I can't stop wondering why and what that was about. Also I love how close to C# this is. Very pleasant language.

[@user-pj9qk9ki2d]

very good talk! btw which this IDE/editor did you use?

[@funkdefied1]

Excellent

[@spinthma]

Amazing!

[@adamduvick]

This is cool! I believe you could make it have implemented this more cleanly using an enum to represent Bounded/Unbounded.

[@0xccd]

What a talk!!

[@ShakthiMonkey]

19:25 at line 24, is that a recursive call to the function he's writing? (Is self.iter.next() a call to the exact function that he's writing that in?) Or is that the next() function of the iterator stored inside Progress.iter?

[@mattbradbury1797]

The latter, he's calling the encapsulated iterator that was given in the ::new() static method to create the Progress struct in the first place

[@ShakthiMonkey]

@@mattbradbury1797 great thank you

[@quertyv12]

This is basically a simple application of RFC PR 445 ( Trait Extension) + Builder Pattern + Trait Bounds. Nothing new. However, do we have any other option in Rust besides of designing our APIs based on traits/types? In other words in Rust everything revolves around traits. Whatever you do is kind of "Type-Driven".

[@fiazsyed5621]

Good 👍

[@empeaTV]

How did you manage to get the type definition within that for-loop to not bail at you at compilation? Is this even supposed to be working?

[@swapode]

Did you mean the ": &i32" in "for i: &i32 in v.iter() ..."? Everything in grey isn't actually in the source code. It's just the editor displaying the inferred type in an attempt to be helpful.

[@VivekYadav-ds8oz]

Why can't the with_bound() and with_delims() methods be shifted into the impl block with Iter: ExactSizeIterator?

[@samuraijack7295]

I think the idea is that you cannot set delimiters on a type without a bound being set first. The typestate pattern ensures that this invariant is respected by only making with_delims() available to iterators on Bounded types which are also only available after calling with_bound() on an Unbounded type. I think it's pretty neat tbh.

[@turun_ambartanen]

If the "impl Bla for Blubb" part before the prenthesis "{}" is the same you can combine them and implement multiple methods under one single "impl Bla for Blubb" line.

[@modex98]

i think for Bound & Unbound states we can use just an Enum

[@fubupc]

Using an enum will not work here (if you mean something like `enum Bound{Bounded, Unbounded(usize, ..)}`), because you cann't use `Bound::Bounded` as generic parameter in `impl Progress {...}` because it's a Value instead of a Type.

[@sampathsris]

Rust error messages are freaking great. All languages will tell you that "no method `with_delims` was found". But Rust also tells you where you can find it as well. How cool is that?

[@PaulSebastianM]

3:00 functional composition!

[@akarshjain7141]

How inline virtual text is being shown in the vim editor? @11:41

[@-..-_-..-]

he's using emacs

[@user-wj8rh9dq4u]

First, it's emacs. Second, it's called "inlay hints" and it's a feature of Rust's language server. As long as you properly configure your vim/emacs's LSP client, it should work.

[@manuelplank5406]

Instead of introducing the states, wouldn’t it be easier to implement with_delims only for ExactSizeIterator with a where?

[@andrewoswald9114]

Arguably so, yes, however, I believe his intent at that point in the talk was to try and demonstrate how "type state" can potentially offer a friendlier compiler error message.

[@andrewoswald9114]

Actually, I re-watched it and it really doesn't have anything to do with error messages, but rather a quick example of how adding state capability prevents ambiguity in the API. If you haven't already seen them, check out some examples that use std::marker::PhantomData.

[@maximinliebkne9375]

What's the color theme in the video?

[@raretapes8057]

might be modus operandi.

[@DJenriqez]

Imagine scooby doo unmasking meme where where is "trait keyword" and behind it there is "interface",.....

[@Dorumin]

16:16 what would have been a better name for it?

[@swapode]

I can't think of any. Rust nailed the primitive type names. usize/isize certainly beats size_t/ssize_t and fits perfectly within the u8, u16, ... schema. My guess is that it may seem a bit unwieldy if you come from languages that live further away from the metal, like python.

[@clarionisige6124]

Thanks for the talk . Which code editor did you use while working on the API from the CMD?

[@alaa3278]

@Xentatt No. Emacs.

[@alaa3278]

It is Emacs. Please check kzbin.info/www/bejne/mJ_RkpaimtaabM0 while switching applications Emacs is the one.

[@sytranvn]

Can we call with_delims then with_bound?

[@karangirigauswami]

No, I think you will encounter the same error as at 38:25. As Will mentioned at 36:21, the with_delims method is only implemented for the Bounded state. In this example, by default, Progress will be in the Unbounded state and can only be converted to the Bounded state using the with_bounded method.

[@bibi2900]

22:30 wow! We have this for years in c#… this is called extension method

[@bangonkali]

Same thought. I think flurl library brings this to insane levels. I sometimes see string suddenly become a get request in 1 line of code. 🤣

[@Adowrath]

Not necessarily/equivalently. The difference is that in Rust, you have a way to talk about that fact generically - akin to "Whatever this T is, it needs to have this method (from that trait)", but each specific instantiation of T can have the implementation be different. In C#/with extension methods, you don't generally get that. You either implement it for a specific type (or interface), one by one, but then can't use that method as a generic interface over that set of types, or on a generic type (with optional constraints), in which case you have a one-size-fits-all solution. Plus, with Rust's orphaning rules, his ProgressIteratorExt trait can only be implemented in two places respectively (simplified view): Alongside the trait definition, or alongside the type definition. That way, the compiler knows exactly where he'd need to look for any particular implementation, and with some built-in traits it can even guide you if you make an error, and the imports are very clear. If the type or the trait is in scope, the implementation is as well. Extension methods are generally unconstrained, which is both good in that they can be placed anywhere, and bad because they can be placed anywhere. All in all, they're comparable, but far from equivalent.

[@jit_rs]

The fact that you can extend types in Rust is merely a side effect. Rust traits are different from C# interfaces in three ways: 1) trait methods can refer to the type that is implementing the trait using keyword "Self". Think about type of "this" in C# interface definition 2) trait impls can be generic which means they can add a method to a swath of types at once. C# extension methods only extend one concrete type 3) traits can require the type to implement a static method. Static methods in C# interfaces have to have a body and cannot be overriden

[@yapayzeka]

this is gold. also very talented person

[@JuanGarciaBlanco]

Great talk!

[@kirillt9009]

Really good talk.