Improve your Rust APIs with the type state pattern

[@letsgetrusty]

📝Get your *FREE Rust cheat sheet* : www.letsgetrusty.com/cheatsheet Corrections: 10:00 - PhantomData isn't needed here. The Locked and Unlocked structs are already zero-sized types. 11:40 - The lock method should return PasswordManager

[@kennethbeal]

That was great, thank you! Amazing how it catches so many errors before even running; and, the hints are dynamic as well. Nicely demonstrated.

[@xokelis0015]

There is a bug in your final lock() method on lines 27 and 31, since it just returns an unlocked password manager. You should change the state to locked in both lines. Other than that minor issue, great vid.

[@RoyaltyInTraining.]

I instantly fell in love with this pattern as soon as I figured out what it's all about. It's the perfect use for Rust's amazing type system.

[@jeffg4686]

I like the idea of using "types" as "markers" of sorts (regarding using generics on a struct just for differentiation). Better than using a field as that's runtime data.This was a great little video. Helped me see things a little different. A "rust patterns" mini-series would be good. Definitely some distinctions from other languages.

[@shinobuoshino5066]

Lmao, this was done in OOP for past 50 years.

[@LibreGlider]

@@shinobuoshino5066What does OOP have to do with the topic of the video? This is about encapsulation verification (access control) at compile time via Rust's type system. How does OOP do the same via types?

[@VictorGamerLOL]

You mistyped the lock method it still returns an unlocked password manager

[@letsgetrusty]

Good catch, it should be locked.

[@mihaigalos279]

@@letsgetrustyI suggest to always return Self to simplify refactoring and guard against such cases for the future. One can also construct and return a Self {}.

[@brdrnda3805]

@@mihaigalos279 Isn't Self the wrong type? If you call unlock then Self is PasswordManager but you want to return a PasswordManager

[@FryuniGamer]

​@@mihaigalos279 can't use Self when changing the state since Self represent the type with the current state

[@Galaf]

Thank you, I thought I was going crazy.

[@EngineerNick]

Brilliant thank you! PyO3 (the rust crate for python extensions) uses this method to confirm the user has the GIL (Global interpreter Lock). At the time I gave up trying to understand the phantom data, but thanks to you I understand now! Now if only there was a way to make the macros that crate uses less mysterious...

[@jonesmartins]

This is the first time I understand PhantomData. Thanks!

[@diarmaidmac2149]

Amazing video. Thank you! This solution is very elegant. Thank you for exploring the initial, non-optimal solutions first. It makes it easy to see the benefits of the final solution.

[@embeddedbastler6406]

As an alternative implementation, we could have the unlock method return an owned token that is a required parameter for the list_passwords method. Upon locking, the token is consumed again by the PasswordManager.

[@GrzesiuG44]

This is definetly the proper way to solve this specific problem. Not only you get compile time checks for API usage, but your API is now closer to supporting being unlocked independently in two different functions.

[@tubebrocoli]

@@GrzesiuG44 however, this requires allocating extra data, which isn't what the demo was trying to demonstrate.

[@oleh1]

@@tubebrocoli cannot this token be a zero-sized struct too? "struct Token;" If it can, how does passing a zero-sized struct as a function parameter affect the function's call stack?

[@SundersBurrito]

Also you would only be able to list once. The impl in the video lets you use the unlocked manager as much as you want.

[@vanish3408]

I'm pretty sure you don't have to specify the type of PhantomData since it would be done by the compiler, aside from that very small thing, this is one of your best videos so far. It's providing information on an intermidiate level while being explained very well!

[@julytikh]

Indeed, usage of `PhantomData` seems redundant since `State` is already a zero-sized type. (BTW, there is another error: in the third solution, the method `lock()` should return a `PasswordManager`, not `PasswordManager`.)

[@divandrey-u3q]

@@julytikh Yeah, I also noticed both errors. But the video is still great!

[@tri99er_]

I'm new to Rust, but I felt weird, when he said, that just using "State" would take up memory, since Locked amd Unlocked are zero sized types, just like PhantomData. I'm glad to see, that I wasn't mistaken. What are idiomatic uses of PhantomData then? I know, it can "own" nonzero size types (e.g. PhantomData), but when is it useful compared to using just plain unit structs? The only thing I can see, is if somebody by mistake created PasswordManager (which wouldn't be prevented by compiler), they'd be wasting some memory in case, where "state: State" ("state: i32": size is equal to size of i32), as opposed to "state: PhantomData" ("state: PhantomData": size is 0). But there's a way to restrict the State generic to only include the right types, which would prevent you from creating idiotic types like PasswordManager, which would be even better, than using PhantomData to deal with it. You can create a (possibly private) marker trait yourself, let's call LockState and have Locked and Unlocked implement it, and then restrict generic State: LockState. At that point there's no need to use PhantomData in this case, and it's absolutely impossible to create idiotic types (e.g. PasswordManager). I guess PhantomData is useful in cases, where your intent is for your type to be able to be marked by any types user of your library wants (e.g. YourType, YourType, YourType). But I don't know of such scenario (because YourType would not actually contain values of those types, it's only marked by them).

[@MrVulfe]

@@tri99er_ One case where this comes up occasionally is when you need a type dependency for the working of your type (e.g. for a return type in one of its methods, for example) but the type isn't actually used in the struct itself. In that situation, you can use a PhantomData to satisfy the compiler that you are "using" the type in the struct so that you can use the parameter in your methods.

[@julians.2597]

@@tri99er_ e.g. the implementation of 'dyn Trait'

[@kajacx]

Nice, saw this once before in a "chaining builder" pattern that prevented setting the same property twice. I personally would make UnlockedManager a newtype for &mut PasswordManager, that way, do don't need ownership and you get the re-lock for free when it drops.

[@codebycarlos]

Wow man. First 12 hours with Rust and it has already blown my mind so many times.

[@varantavers]

Recently I have came across a situation where I had to do something similar and this video immediately came to my mind. Nice work!

[@kelechiukah8025]

Nice video. Would love to see a split screen or have one pane zoomed in, one pane zoomed out, because i found it hard to do global reasoning when we move around with only 10 lines of code. Or maybe a mini-map

[@flippert0]

Very interesting and illuminating ! A while back ago, I was heavily into generating code from UML state diagrams. Generated state machine code either would use a traditional state transition table or state classes. Now duplicated code isn't that much of a problem, if the code is generated anyway, however to avoid duplicatation I could also a use traditional inheritance. So "LockedPasswordManagerr" and "UnlockedPasswordManager" would both inherit from "PasswordManager" which would implement common code (here: "version()" and "encryption"). It never occurred to me, that I could do the same with generics.

[@tri99er_]

There's no inheritance in Rust. And traits don't have access to implementor fields, so you'll end up with the same amount of duplication.

[@minciNashu]

Would be interesting to expand on this further by implementing auto lock on drop, raii style.

[@kajacx]

Just make LockedPasswordManager a newtype for &mut PasswordManager. That way, it makes the locked manager avaliable after it gets dropped automatically thanks to Rust's ownership system.

[@MrTrollland]

u can just impl the Drop trait cant u

[@antoniong4380]

Wtf. You can actually do this? So much potential! And head wacking because I need to make sense of the structure

[@kwinzman]

I've been doing that instinctively for a long time now. But great example and explanation nontheless! The Rust specifics with the zero-size types was new and pretty informative!

[@banniball]

I liked the general concepts of using types to represent the state. But wouldn't one problem be that after locking the password manager one could still have a reference to the unlocked one. Since lock/unlock returns a new instance

[@banniball]

I.e it's not like an FSM with one state but rather a struct where you can have multiple instances all with same content with different state.

[@MatthiasBlume]

That's the whole point: lock and unlock take their self argument not by reference but by ownership transfer (i.e., move). So the caller loses ownership, and there cannot be any references to the old instance or the whole thing does not typecheck.

[@leonardogomes5121]

Great! Great! Great! The best explanation of the Type State Pattern on KZbin! You rock man! How can we support your work?

[@CYXXYC]

how would using just state: State waste memory? Isn't it also 0-sized?

[@koonoho]

I'm also confused about this. I can certainly see PhantomData being useful if the struct holds some data, but in this case there really shouldn't be any additional memory used.

[@pawe460]

I checked it on goldbolt and both structures with :State and :PhantomData takes 72 bytes for me, so I guess it is optimized out in both variants

[@Baptistetriple0]

What he is saying is very confusing, if T is a ZST you don't need PhantomData to make it ZS because it already is, PhantomData exist primarly for lifetimes: for example there are structs that owns a pointer for optimizations, but still need to hold the lifetime of the backing data, so you add somewhere a PhantomData

[@letsgetrusty]

It wouldn't, I made a mistake. Will point this out in the pinned comment.

[@nirmalyasengupta6883]

Zero-size Types is new to me. I have learnt something. Thanks. I have used similar approaches in Scala / Java earlier, even though the effort was more. Moreover, in Akka-Typed, the approach is very similar, even though the implementation is cumbersome. Thanks again, for uploading this.

[@schred]

Your best video yet in my opinion, thanks for your work! :D

[@slippinchillin]

I like TypeScript exactly because it provides similar functionality to implement this 👍

[@jonnyso1]

That's the exact feature I needed the other day and I didn't know yet !

[@lukasz_kostka]

Fantastic. I love short, to the point tutorials like this.

[@Matt23488]

In the third solution, you didn't update the lock() method to return a locked password manager. But it doesn't matter, you got the point across and I bet many people didn't even notice.

[@Jiftoo]

I use this pattern for data sanitisation in my backend. Works great!

[@ShreksSpliff]

In some other languages like Haskell they call it the Indexed Monad pattern. Feels pretty similar to me. The Rust library called Graph uses this in their builder pattern graph constructor, so it can infer at the type level if your graph is directed or undirected, and if the edges or nodes contains values. Also, thanks for this. The non generic example was a nice touch. Should it be put in the lock impl, as it defaults to lock or is this better as it changes with the default state?

[@chessai2121]

Indexed Monads are an extension of "plain"-er GADTs with DataKinds. Before those were usable, people used open phantom types. GADTs + DataKinds are more common than indexed monads because a lot of operations can be represented with simple non-monadic functions.

[@Chastor97]

Till the end I have been waiting for raii cause I think it fits here well

[@Artentus]

Instead of zero sized structs you can use empty enums here. They are not only zero sized but actually not constructible.

[@diadetediotedio6918]

This would solve the problem of user building the wrong state by itself

[@ErikCampobadal]

The issue is that you can't implement different functionality on a struct based on an enum variant. Therefore, we only know if the manager is locked/unlocked at runtime, resulting in panics or results like the first solution. As he showcased, generics allows multiple implementations. The only concern I have is restricting the available State variants, since his implementation allows arbitrary types to it. Perhaps having a sealed trait would solve this as a constraint => pub type PasswordManager {}

[@Artentus]

@@ErikCampobadal You didn't understand me correctly. I'm saying turn `struct Locked;` into `enum Locked {}`, an enum with 0 veriants which is a non-constructible type in Rust.

[@ErikCampobadal]

@@Artentus ahh interesting, indeed. This combined with a sealed trait could be a great solution

[@Wodziwob]

Would you say this is the better approach? Is it less common?

[@Charls93xx]

That was super straightforward, thanks for sharing!

[@FlooferLand]

Another solution would be to use Enums. You could have an enum called PasswordManager that has 2 states, one "Locked" and one "Unlocked", both of them containing a ManagerInfo struct. You could have a lock/unlock method for the enum which could do `let manager = PasswordManager::Locked(manager.0);` Doesn't really solve the main issue, just adding to the pile XD

[@tri99er_]

Enum variants can't have unique implementations, so it doesn't really solve anything at all.

[@heret1c385]

great pattern. Man, I just love rust.

[@dirty-kebab]

I was thinking of the double struct idea before you described the first one. 🤠

[@dirty-kebab]

Okay that was better than I expected. Answer me this.... Why did it default to locked? I couldn't see any detail mentioning why it knew to default to locked? Because it was higher in the file? I feel like rust is deeper than that right?

[@sunofabeach9424]

@@dirty-kebab 8:45

[@debuti]

I think this is maybe your best video

[@martinbecker1069]

This perfectly explains Phantom Data types, I never really grokked this but this instantly made me understand and Now I can see how it can be used in other places! It kind of reminds me of how Two structs of the same Type but with different life-times are treated as two completely separated types so you can't return something with the wrong life-time.

[@julytikh]

Actually, `PhantomData` is redundant in this particular case. The types `Locked` and `Unlocked` are already zero-sized (because they have no fields), and `PhantomData` does not improve upon that.

[@LukasCobbler]

@@julytikh can you explain how not to use the phantom data in this example

[@julytikh]

@@LukasCobbler just use `State` instead of `std::marker::PhantomData`.

[@TadaHrd]

This was relay helpful. I couldn't come up with such a smart system. However, you should have a State trait that is implemented for the two state types.

[@skyeplus]

This example is consuming previous state, which is fine. Can you make a proxy type holding reference instead?

[@julians.2597]

it intentionally consumes self to prevent dangling references to previous states

[@rsalmei]

It does not need both the default generic type and the last impl block. Just define `new()` in whatever state you want it, e.g. in the `impl PasswordManager` block.

[@alexandersemionov5790]

interesting, looks like a builder pattern, but returns an instance of same struct with just a state 0 size in memory and template reference thing. Great combo

[@sinushkin]

SRP. the first responsibility - unlock, the second - all after unlock features. You even don't need state pattern

[@JUMPINGxxJEFF]

Nicely explained, thanks

[@Echiduna]

Enjoy this video so much!

[@JohnDoe-ji1zv]

Nice video, really useful, didn’t know about this pattern and state before. Cheers

[@chillyvanilly6352]

Wow, that was genuinely an awesome vid, thank you!

[@fumseck]

How feasible would it be to do it the way Mutexes work, with the lock being held only until out of scope ?

[@inx1819]

this is what I thought the video would be about

[@Turalcar]

This API prevents calling lock twice while for mutexes calling lock twice (usually from a different thread) is the point. I suppose you could create a Mutex API such that each client gets issued a reference wrapped in a way that prevents lock being callable the second time on the same reference.

[@irlshrek]

this was a really good explanation!! thanks for this

[@AceofSpades5757]

I haven't seen this one before. Very cool!

[@AlwaysStaringSkyward]

I learned a lot from this. Thank you!

[@soberhippie]

The problem I ran into is when I try to implement a trait for such state-parameterised struct, I either get "trait not implemented" or "duplicate definitions with the name " errors.

[@xavhow]

This is so useful! Thank you for sharing.

[@danielcollin4012]

Good video. My only comment would be to use &str for passing in strings and then take ownership internally instead. That makes it user side a bit nicer to use.

[@kajacx]

How? If I have a computed String, I have to pass a reference to it, and the code just copies the value for no reason, when I could have just given the ownership instead.

[@danielcollin4012]

@@kajacx If you pass &str you can just pass refs like "my string" instead of "my string".to_owned() so there is always a trade-off in this case. If one wants to support both refs (without copy) and passing in owned string using Cow

[@AmirHosseinHonardust]

My head hurts with possibilities!

[@CalifornianViking]

Thanks - it is nice to see these concepts being applied. Is it possible to achieve this without having to create a new manager when switching from locked to unlocked?

[@qexat]

The third solution inspired me from making a Python equivalent(-ish) that uses a boolean literal generic trick Although I'm not posting the link because I'm unsure if my comment would get deleted for spam or something...

[@mustafazakiassagaf1757]

i like this kind of video. keep up the good work man

[@setoelkahfi]

This been really useful. Thanks!

[@haydn.murray]

This was great!

[@loganhodgsn]

Since Locked and Unlocked are Zero-Sized Types (ZST's), why do we need to use PhantomData? What if our state needed to carry a little bit of information? (such as who unlocked it)

[@ianknowles]

Always handy Bogdan appreciate the share!

[@wumwum42]

I think in the final solution, you can merge the first and 4th impl. PasswordManager and PasswordManager is the same

[@rad9587]

This is correct, but I think it's clearer, because we are not creating a locked manager, but just a manager(Even though it will be locked anyway)

[@mmssus]

Question here though, why PasswordManager implementation for the constructor doesn't need a generic like the common methods implementation "encryption" and "version"?

[@aniketfuryrocks]

The third solution is good but u could have just used Into and From or enums

[@electrolyteorb]

- We have Quadrilateral. - We need a Trapezium, which is a Quadrilateral with atleast one pair of parallel sides - We have a Parallelogram, which is a Trapezium with two pairs of parallel sides - Now we have a Rectangle, which is a Parallelogram with 90* angles - Again, we have a Rhombus, a Parallelogram with all sides equal - FINALLY FINALLY, We have a square... Which is a Rectangle + Rhombus??? How do we model this in Rusts type system???

[@sunofabeach9424]

6 structs

[@nerdbot4446]

Can you also create an impl block that defines methods for multiple states? There might be methods that shouldn't exist in all states, but in some.

[@GAGONMYCOREY]

This video is one of your best

[@AndrewErwin73]

This is why I prefer OOP! I feel like all three solutions could be simplified with polymorphism.

[@illker.]

bro 💀

[@khai96x]

Locked and Unlocked are already zero-sized, why PhantomData?

[@laurentle-hebrard544]

It gets rid of the "unused" error on the parameter State.

[@ryan-levy]

@@laurentle-hebrard544 It's a bit more complicated than that. There's a good, albeit technical explanation of PhantomData's purpose in the Rustonomicon. The easiest way I can explain it is that it allows the compiler to properly handle the type parameter.

[@khai96x]

@@ryan-levy You can still use Locked and Unlocked directly, and the compiler would still distinguish PasswordManager and PasswordManager. PhantomData is only necessary when either: 1. Locked and Unlocked aren't zero-sized, which isn't the case here. 2. Locked and Unlocked cannot be initialized, which also isn't the case here.

[@ryan-levy]

@@khai96x Ah okay. I guess my only argument for it here would be... it helps highlight the purpose of the type argument in a self-explanatory way? I dunno, just a stretch to make some sense out of that decision. In the context of a tutorial, it's not that bad of a usage, since it helps highlight why someone would want to use PhantomData.

[@porky1118]

1:40 It's pretty obvious at this point, that lock needs to return a type, which borrows password manager, which automatically unlocks, when it goes out of scope, and only this struct would implement the functions which requires the password manager to be locked.

[@porky1118]

A solution like in your video is something I thought about just after posting this, but didn't consider it to be a good solution, since it still requires the alternate variant of the password manager to be owned, which doesn't work well if you want to store the password manager in another struct.

[@Archepter]

This was very cool. Thanks !

[@samansamani4477]

Thank you, this can become handy

[@r31527]

Awesome! Is there a way to prevent users from passing a random zero-sized type that we don't handle?

[@sailormouse4749]

You can define a sealed trait in your library and then implement it for only the state structs you want. Then you can do and now only the types that you wanted the user to pass in as generic can be passed in.

[@r31527]

@@sailormouse4749 Thank you so much

[@juan0338]

Would an enum work?

[@rez188]

If you try it yourself, you'll see that there is no way to create an 'Account' struct without using new. new() always returns an 'Account' and the only way to get a different type is by logging in or out.

[@r31527]

@@rez188 True! Completely looked past that. But that's still just in the context of this specific password manager example

[@_jdfx]

Really cool! thanks Bogdan!

[@timvw01]

In the last example, if you switch states, does rust have to move all the data to the new struct? So its basically allocating a whole new object when switching states?

[@ddystopia8091]

Incredible!

[@shufflecat3334]

So how do we handle the case where the client gets the password wrong? The unlock method should either return an unlocked manager or a locked manager...or panic I guess?

[@dmdeemer]

Bogdan, Rust neophyte here. I get confused sometimes when my variables get consumed when I don't expect them to be. How do you know quickly when a variable is going to be consumed, either by using it as a function parameter, calling an implemented method on it, or even iterating over a vector? Do you just have to be aware of the exact syntax of every function call? Also, in the Type-State pattern is there any way to avoid moving each member from one state of the type to the next? That could get tedious for a non-trivial number of states or members.

[@electra_]

In this specific instance, I would probably use soemthing similar to Mutex with a "PasswordManagerGuard" type class that borrows it mutably. Then, dropping this object would automatically lock the password manager. Overall this seems like a really cool way to do different states and I will try to use it going forward.

[@LtdJorge]

I believe the video is not about the implementation of locking/unlocking. It's about the methods a user of the API can use when in any of those states. So basically, moving runtime errors to compile-time errors.

[@MiterAlmeida]

Loved this video! ❤

[@nocodenoblunder6672]

This looks amazing. But is only suitable for state that changes infrequently right? Whenever we transform from one state to another, we are creating a copy of the structs fields

[@jeffjia106]

can we make Locked, Unlocked from struct to enum?

[@jaumesinglavalls5486]

why the lock method returns the state Unlocked? It should return the state Locked, right? Also, in rust should be possible to force the manager cannot be used after calling to lock or unlock? (to force do the switch of the variable?)

[@stephenreaves3205]

This was beautiful

[@ErikBongers]

PhantomData feels like a cludge. Hope they'll expand the language to allow for unused templates parameters.

[@sunofabeach9424]

check out rfc 738. unbounded types were there but they were removed and markers were added instead

[@tathagataroy5153]

Why not just use a trait PasswordManager with the common methods and then create LockedPasswordManager and UnlockedPasswordManager to implement Password Manager. What would be the difference between using traits and the phantomdata method? I fail to notice any duplication that may take place in the trait method? Thank you.

[@flippert0]

I did exactly that in the past (see my comment above)

[@szotsmiklos8549]

What stops users from adding a new impl for any random State? It looks like you can't enforce the state type to be some private trait for Locked/Unlocked? Otherwise I think the approach with different structs are better since it's not possible to add weird variants of PasswordManager that way, and for shared methods you can still have a trait and impl for both locked/unlocked manager

[@minimalhostage]

the video missed one approach to the generic that solves the issue you mention. Use trait bounds instead of setting State to a default struct. create a trait in a private module, impl it on your state structs. 1. trait Lockable; 2. impl Lockable for Locked {} 3. impl Lockable for Unlocked {} 4. pub struct PasswordManager {…} 5. pub fn get_password_manager() -> PasswordManager you can also use the builder pattern to create a password manager. but as long as you keep your marker traits and marker structs private,there is no way for users to add impls. trait bounds and builders are extremely powerful when you want to control apis.

[@YuruCampSupermacy]

Video starts at 8:00

[@kirillgimranov4943]

Nice stuff, it'll help a lot

[@fhdiaz87]

Can we pass any type as parameter? Not Locked or Unlocked

[@adambutler2646]

great video. thank you

[@HaydonRyan]

This was tricky and a smart use of types. I really like this :)

[@xDeltaF1x]

I don't think PhantomData is needed here since Locked and Unlocked are already ZST

[@Chalisque]

What happens if you unlock with an incorrect password? I presume unlock actually needs to return a Result, containing an error of the password is incorrect.

[@officialismailshah]

Best vid❤❤❤❤

[@ThePythonicCow]

Starting at the 2:06 mark I think you accidentally got backwards whether the lock should be held or not when calling list and add password.

[@erlangparasu6339]

thank you so much 🙏

[@n0kodoko143]

Sweet!

[@sgq995]

Great video!

[@ramongonzalezfernandez8904]

I would have the unlocked one contain a mutable ref to the locked one, and when it drops, its automatically locked.

[@ГеоргийТимофеевский-х9х]

How do you forbid to pass something except Locked and Unlocked into the State? What can prevent user from creating PasswordManager?

[@ГеоргийТимофеевский-х9х]

Also it would be nice if the PasswordManager could be only created from PasswordManager and not from outside code