Rust Lexer : Finished It!!!!

LIVE ON TWITCH: / theprimeagen
Get in on Discord: / discord
Get in on Twitter: / theprimeagen
Got Something For Me to Read or Watch??:
/ theprimeagenreact

[@excelfan85]

Buying the book and coding along is the best thing i have done in a long time. I have chosen Rust as my first language to learn, and it's beaten me down so many times. I feel like i am having some wins atm. I got stuck trying to get this to work on my own, but was super happy to see where i had gotten it right. I am having some ah-huh moments along the way, best feeling ever. Being new to this, its easy to get caught up in the frustration of how long it takes to do something. I have a project that i am working on, its progressing soooo slowly it hurts, and that is because i am trying to get it right rather than getting it written aka failing fast. What is hard will one day be my warmup, but having the patience and fortitude to keep going is hard. That said, I have never complained after sitting down, writing some code, and learning, like i have after sitting down and gaming.

[@TheVimeagen]

yeah! i will also be doing some upgrades, adding lifetimes, etc etc to rust so as you follow along you should be able to +1 your skills!

[@JakobKenda]

I used the regex crate for my lexer. HEAR ME OUT! It's just because I want all Unicode characters to be valid, not just ASCII.

[@TheVimeagen]

hey, we all choose our own destiny

[@excelfan85]

We all run our own race too

[@FusionHyperion]

@@excelfan85 that's racist

[@SimonBuchanNz]

Keep in mind most of the standard char::is_* methods (except is_ascii_* obviously!) use Unicode properties. If you just want to know alphabetic characters, that's enough. Full Unicode property testing would be nice though (especially ID_Start/Continue!)

[@dealloc]

Waiting for that Regex bomb to happen...

[@pixelstriko1642]

I made a lexer too, but when I started on the parser things got a bit rusty.

[@Shri]

6:10 trait and impl is like if Javascript had the functionality to manipulate the Base class instead of extending it. In most OOP style languages, class once defined is sealed and cannot be manipulated externally. Any manipulation requires you to go back to the class definition and make changes there. And it makes no sense to append a class to add "area()" if you are only using for a specific section of your code. Yeah can see why it is so powerful! This actually feels more OOPSY than regular OOP languages

[@TheVimeagen]

yeah, i want rust + javascript, which i hear is ocaml i am going to be going deep on ocaml soon

[@rogergalindo7318]

haskell and typeclasseeeeessssssss just do that!

[@dealloc]

There is (was?) a proposal for a trait-like system (more like a protocol since it also implements) in JS called First-Class Protocols Proposal. Of course never got any real attention.

[@MadaraUchihaSecondRikudo]

JavaScript *does* have the functionality to manipulate the base class instead of extending it. Promise.prototype.doWhatever = function() {} - I just added a method to all instances of the Promise class, at runtime. I could even change an existing method Promise.prototype.then = function() {}. The different between Rust's traits and that is that JavaScript's is global, if I made a change to Promise.prototype.then I affected every single promise ever, including those spawned in unrelated parts of the code and even in libraries. Rust's traits only take effect if the trait is in scope, which is a big difference in making such changes consistent and predictable, no guessing needed on which .then() I'm using now.

[@dealloc]

@@MadaraUchihaSecondRikudo Yes which is why I want traits/protocols in JavaScript that does not infect global namespace.

[@RVDriest]

Can someone explain me the difference between Rust Traits and C# extension methods?

[@blackbriarmead1966]

Not so long ago I was intimidated by your coding videos about lexers. But then I was forced to learn the theory of finite automata and implement a parser in Java in 3 weeks as part of my compilers course in college. And now im wondering what else I find that I find intimidating that I could learn in a few weeks. And i didnt expect to be so interested, but its like peering under the hood of the tools I use every day

[@DNA912]

I'm coding along with this lexer stuff, and man, prime really is right when he says you need to keep coding rust to not loose your abilities. haven't done rust in a few month and last week I started again and man in was hard to read, but now it start coming back and, damn do I love rust.

[@thekwoka4707]

I find it to be one of the easier languages to read for more complex things. What makes the forgetting it easy is just if you are coding in something else all the time your brain gets very wired for that, so then coming to something that is similar but different, your brain has some trouble

[@GeekOverdose]

I keep hearing "the book" mentioned. What is this book?

[@ErikBongers]

To avoid Ident(String) making a copy, perhaps have a Ident(pos, len) referencing positions in the source text? Would Ident (rust_string_slice) work in an enum? Probably not possible. But yes, I agree it would be better to avoid copying strings all over the place if you can reference the source text.

[@Jiftoo]

I miss him yelling "This is a marker" each time.

[@remrevo3944]

20:26 Rust supports just using "" to define multi-line strings. r#""# is only useful if you don't want to escape quotes.

[@Yupppi]

I realized watching very effective ADHD person code is like NTSC/PAL difference. My head's been spinning watching Prime breeze through code and only now I realized 0.75x playback speed makes his coding and commentary feel like regular stuff that I'll be barely able to follow. Fascinating how fast a person's brain and fingers can work.

[@theevilcottonball]

Why don't editors support querying top left functions based on their first argument, so in C: uint8_t a = 4; and then you could type "a"+( and the editors would provide provide functions that take a uint8_t as a first argument, so you select area(uint8_t a) and the editor would insert it. area(a); why does it have to be a dot, or a method?

[@sanjayidpuganti]

Is this a fan channel or is this also prime's channel?

[@mixed_nuts]

Is the trait system like in Lua fn foo:bar()? Which is like calling bar(foo)?

[@TheVimeagen]

that is how all functions really work with classes, but its more than that its typechecking + that

[@notuxnobux]

foo:bar is just syntatic sugar for bar(foo). You can implement traits with lua metatable (metamethods) but metatable in lua is very very different from how traits work since there is a lot of things you can do with metatables.

[@spottedmahn]

3:12 So traits… that example is equivalent to C# extension methods, no? I’m sure they’re different but same concept here? 🤔

[@TheVimeagen]

i hear they are very close, not a c# master / fan

[@metaltyphoon]

It is.

[@Liphi]

C# extension methods are just a static methods, that takes the first argument as "self". So the idea is pretty similar

[@Liphi]

Actually would be nice if someday m$ introduced namespace defined methods, so u don't need to create a separate static class with static method every time

[@metaltyphoon]

@@Liphi every language that has `methods` works this way.

[@naranyala_dev]

make the same code in rust and zig, love it

[@anonymous.343.14]

Is the song thing an inside joke? I never hear music in the videos lol

[@irishbruse]

It's removed in the yt videos for copyright lol it's there if you watch him live

[@TheVimeagen]

song thing is most certainly real :)

[@kein3001]

16:15 Mindy comming here with the NixOS BTW I tried it yesterday it is sooo good!

[@kein3001]

Ohh it is 15:15 BTW

[@TheVimeagen]

got BTW'd with nix... always a W

[@kein3001]

@@TheVimeagen i installed it on my steam deck and it works wonderful!

[@q1joe]

so uh Traits are... c# extension methods?

[@NexushasTaken]

maybe yes. but it can also be a class methods or static functions(?).

[@oumardicko5593]

so rust traits are like kotlin extension function 🤔

[@andrewdunbar828]

Swift too

[@metaltyphoon]

C# too

[@ccgarciab]

But also like interfaces

[@sacredgeometry]

5:41 meanwhile every c# developer in the world is like yeah ... we have had those for almost 20 years. Cute though.

[@sacredgeometry]

7:57 absolutely this. It's a gorgeous language (one of the best). I dont even use any MS products, windows makes me legitimately angry.

[@SimonBuchanNz]

Traits are mostly from Haskell's (confusingly named) type classes, circa mid 90s, so they technically precede C# extension methods by about a decade! They're actually a lot more flexible and powerful than extension methods (not *always* a good thing), they're something like an interface for extension methods, that you can use as constraints for generic parameters, like C# interfaces but you can implement then on types from other packages (although there are restrictions to avoid conflicting implementations) You can also perform a bit of type level programming with their associated types, a simple case is the Iterator trait has an associated Item type, so you don't need the duck typing C# does to figure out what GetIterator returns. They are a very cool feature that more languages should have!

[@sacredgeometry]

​@@SimonBuchanNz So they are exactly like extension method on interfaces in C# i.e. how Linq is implemented? I am not sure what you mean by: "you don't need the duck typing C# does to figure out what GetIterator returns". Where is the duck typing?

[@SimonBuchanNz]

@@sacredgeometry no, not exactly like extension methods. You can't use extension methods as a way to accept that you take an interface, you need a real interface that the type declares. Vice versa, a C# interface can't be defined on a type that you didn't define. Traits combine the best of both and more. So this is a bit down in the weeds, you don't generally have to worry about this sort of thing directly, but C# doesn't actually use the IEnumerable etc interfaces for foreach and linq, because it has no way to represent the concrete iterator type (not item type) that it will return *and* that it is an implemention of IEnumerator (if it had used another type parameter I forget the exact issues right now, but IIRC basically it doesn't actually work: those are input types not output types it's the call site that has to provide them and the relationship and the caller wouldn't know how to infer the type to use). So basically it had a choice between iteration using the dynamic interface type or using duck typing by just calling the GetEnumerator method and using whatever type that returns as an enumerator, which is often massively faster. Traits can exactly represent a type that you can iterate and the exact types involved and that they implement an iterator trait with a specific item, meaning you can directly and easily constrain your generic code to the correct API, and get the same full performance as a foreach. And again, this is one of the simplest examples of using associated types, which is on top of the much nicer way of defining interfaces and their implementations separate from the type.

[@carstenrasmussen1159]

Traits in rust is just like uniform function calls like in D and Nim think that C++ also have it now

[@ccgarciab]

Traits are about subtyping. People always compare them first with interfaces

[@c0ldfury]

Vim coders are so SLOW. They jump about and move about so much it looks impressive, probably feels impressive to them. But ultimate the make 10 key presses for a single mouse click. They are dumb. They are literally 10x coders.

[@Cbon-xh3ry]

A teenager stuck in an adult body. I feel like I’m watching a dumb TikTok video 🤡🤡🤡