rust macros are magic

Rust macros are magic.
I want to show you a real life application of a powerful rust macro I put together, that handles spell assets in the bevy game engine.
If you want to learn more about macro programming with Rust I recommend checking these videos out:
Rust's Witchcraft by ‪@NoBoilerplate‬ • Rust's Witchcraft
Declarative Macros in Rust by ‪@letsgetrusty‬ • Declarative Macros in ...
rust libraries used in this video:
bevy (game engine)
serde (serialization + deserialization)
ron (file format, similar to json. Ron integrates awesomely with rust)
Support me!
⁍ Patreon: / tantandev
⁍ Monero: 43Ktj1Bd4Nkaj4fdx6nPvBZkJewcPjxPB9nafnepM7SdGtcU6rhpxyLiV9w3k92rE1UqHTr4BNqe2ScsK1eEENvZDC3W1ur
My discord group:
/ discord
#gamedev #rustlang

[@atahanyorganci]

you make this a proc macro (derive macro) you can create loader etc names from the struct and mark fields using attributes

[@danielgysi5729]

This sounds like a much more readable and intuitive solution

[@antoninperonnet6138]

And that would be a gret ressource for viewer, since the documentation for attribute macros is quite sparse ...

[@bilalmahmoud1225]

Then again proc macros are quite a lot more complicated to set up and get right. So I understand why going you'd go with proc macros. You can also simply derive those names using the paste crate instead, remind the need for implementing a proc-macro yourself

[@narex45635]

@@antoninperonnet6138 "... the documentation for attribute macros is quite sparse" might genuinely be the reason tantan didn't do this in the first place. Perhaps he simply didn't know how to use such macros or didn't think of it since the documentation is sub-par. We as programmers reach for tools we understand and which have the safety net of looking up how you might be meant to use it.

[@valshaped]

Alternately, copy the entire syntax of an enum declaration into the parameter section of your macro, and cough it out with a million impls on the other end :)

[@omg33ky]

I think this is a case where i would argue against this macro, since it makes the code way harder to understand. I think this could be done way nicer with a bit of derive macros and some generics and impls, which should make this code actually readable for someone new to the code. I know it might just be your personal project, so no one else has to lokk at it but if you have not used this part for weeks/months and then come back to it i promise you wont understand it at a glance/withou having to look through the code again to see what those magic ; and ? meant again. Anyways, have a great day!

[@Elite7555]

But let's be honest: procedural macros are a pain in the ass and they are pretty ugly as well. So theoretically you are absolutely right, but practically... I'm not sure.

[@NexusGamingRadical]

@@Elite7555 Depends what your trying to do, I default to macros all the way but proc macros allow you to standardize a lot of what your trying to do. Its just that you do have to a lot of processing when you need them which feels like a pain but I've grown to prefer writing procs now I've got some libs.

[@i_am_feenster]

This is extremely specific, but also exactly the info I needed! Are you a magician TanTan?

[@okharev8114]

i think the macro is overkill, i think that traits and some generics could do this

[@charetjc]

If this is true, then show us. I'd genuinely like to see a better solution.

[@codingleo]

Easy to say you put a lot of effort on this video. High quality and pretty clarifying. Really appreciate sharing it. Thanks!

[@godofpro5767]

Could you consider doing a bevy tutorial ? i would really like to see that tutorial series

[@yankilo]

Nice metaprogramming, bro!

[@Skeffles]

Fascinating to see how these can be useful with Rust.

[@CheaterCodes]

Let me just say, I have no complaints for using macros here. Seems like a good use case. However, my first choice would have been using traits and generics. In fact, I wrote a bit of rust code that provides almost the same level of abstraction. You can have a generic RonAssetLoader and a RonAssetPlugin, which is only marginally more typing effort than wehat the macros provide. The only *real* challenge is to find a nice way of automatically loading nested assets. No matter what you do (both your solution and mine), having to specify again which field need reloading is annoying. Using derive macros is probably the only good option here (or maybe there's some serde magic). Was fun wasting an hour or two on this :)

[@TS-by9pg]

I also tried implementing this using generics, however with generics I've run into an issue where you can't specify a different extension for different asset type to make it work exactly like in the video. Bevy automatically tries to figure out what loader to use based on the extension. It can be fixed with a trait, that will cast ron value into a desired type. However, with how many problems there are, it really makes me think that the whole implementation is flawed. Like, do you really to use Bevy asset system for that? It seems that it was not meant to be used like this I mostly do web/microservices/desktop development in Rust and only scratched the surface of Bevy development. But I'm pretty confident that using macros for fixing this problem is a sticks and mud solution, especially when there is a single macro that does three completely different things. Still need more research though to figure our what is a best solution.

[@flyingsquirrel3271]

Tantan videos are magic :)

[@Clickificationist]

bevy bevy bevy

[@cchance]

Please recreate this as a derive proc macro on the strict, really feels like this can be optimized and made even easier to read ergonomically

[@deltapi8859]

Quite interesting how Rust turns a high level language into assembly level legibility.

[@AndrewBrownK]

git gud

[@stysner4580]

I don't like Bevy all that much. It makes Rust (which is already verbose) more verbose, which isn't a good idea. Bevy code is near unreadable to me. It also tries to do some classic OOP things (like storing handles to stuff all over the place) that are very unidiomatic for Rust. You have a fully fledged ECS, why would you ever need assets to point to other assets? That's madness.

[@christopher8641]

In all fairness... This could all just be done with generics and blanket impls. Macros have their place but quickly jumping to them as a solution is definitely a code smell. Macros are great for busy work, but can become not great for everything else.

[@CatDevz]

No, you just don't know Rust.

[@christopher8641]

@@anonymousalexander6005 that is entirely dependent on the kind of macro. For declarative macros (ie, macro_rules!) they are practically free as far as the compiler is concerned. Procedural macros definitely have a compile-time cost, but that is a tradeoff for quick and easy code generation. Just like everything in programming, it depends.

[@stysner4580]

I kinda get the feeling you're trying to do OOP things inside Rust, specifically inside a framework that uses an ECS. What makes a hero a specific hero might be the spells that it has, it would make way more sense to construct that hero as an entity by adding the components it needs. The spell functionality should then go into an ECS system. Then you don't need handles pointing to each other all over the place. Which is really un-Rusty.

[@Galakyllz]

I would love if you elaborated more on what the ideal implementation would look like.

[@stysner4580]

@@Galakyllz Ideal is a big word, it depends not only on the project but also preferred style of the coder(s). Rust works well with ECS architecture because there is a lot of decoupling going on. Stuff is "linked" through an Entity, which is just an integer (in my custom ECS it's an integer with a generation attached, so you know if a handle is old and not to be used anymore). A system goes over every entity that has components the system needs. There is no reference counting, boxing etc. required at all. "Ideally" in Rust you should minimize reliance on referencing stuff outside of "self" (the current scope), meaning you pass stuff to functions as a (borrow checked) reference when you need it. But of course there are many cases where an Rc/Arc makes more sense, but again, you minimize it. In short: decouple as much as possible, pass by direct reference as much as you can.

[@lixou]

5:38 I think it would make more sense to make asset_type from a real type just for some edge cases

[@mirkopassoli9418]

I maybe coded in rust one hello world and wrote 1 game in Unity, so I'm not an expert in rust nor game programming, but seams to me that the last macro, the one you use in the tower defense game, is kinda doing too many things and it's unclear from outside (from the caller of the macro) what all those args do, especially the last 3 list an the "put some simbols where and there to differenciate". Am I too much concerned about code readability?

[@stysner4580]

You're half right. The point of macros is to abstract stuff, so as long as the macro name is good enough and you provide good documentation for them it should be fine. In my custom Rust game engine I use some macros to define stuff, every time I have to use one of those I peek at the documentation I made and I instantly know what it does and what I should input. As long as the macro saves way more time than looking up how it works (which you can do in IDE if you have proper documentation) it is worth it.

[@mirkopassoli9418]

​@@stysner4580yes, but in this specific case I would split the implementation in more macros to follow "single responsability" pattern, and if this is really impossible/much more complex I would rename the macro to show "all" the stuff that it makes, not just "create amazing ron asset loader" (create and register loader... don't be afraid of long names). If you use really descriptive names you could save the wait of the popup in the IDE that shows the docs

[@stysner4580]

@@mirkopassoli9418 I already agreed on the descriptive name part; it's very important. Splitting it up more than Tantan already has is not really necessary in this very specific case in my opinion, but of course that depends on style.

[@omg33ky]

I definitely agree, that code readability is quite bad here which could probably be solved with some proc macros and some traits and generics which are easier to understand than just some random ; and ? sprinkled in there. And I would split the load macro into multiple versions too for multi and optionals etc., since this is currently very unreadable (and I really hope he has some documentation for this thing or he's gonna have a bad time in the future)

[@Ferret440]

Really good video, learned a lot from it! Confused about the symbols : ? at the end there, which you said were differentiators, can you pick any characters to separate parameters? Or do those have actual meaning to rust?

[@modular_warfare_remastered3657]

The symbols at the end can be of any symbol to identify the next group of variables ex. $($var: expr),* your macro would be test!(var1, var2, var3)

[@valshaped]

You can use any arbitrary sequence of valid Rust tokens. That includes identifiers, keywords, operators, matched bracket pairs, literals; anything you would reasonably see in a Rust program. Whitespace is also ignored in both the parameter list and invocation site, so go ham!

[@charetjc]

Excellent video. I love programming in rust, but avoid using macros, since I've yet to write a program with the amount of boilerplate you have. One thing that stands out as a problem for me is the use of symbols to split up your argument groups to the macro. I'd much rather use a named tag that describes the argument group (because self-documenting) since I don't work projects from start to finish, but a little here, a little next month, a little more three months later, etc. Is it possible to replace the symbols (: and ?) in your macro with names?

[@DrIngo1980]

Beautiful. Just beautiful. Nicely done macro programming. Loving it.

[@allxrise]

If you don't want to write HeroAssetPlugin, HeroAssetLoader, HeroAsset, etc... by hand , you can make it simpler with paste crate. You can convert this macro that just takes an asset identifier and extensions identifier.

[@ariandannemann4544]

So coming from other languages i am a bit confused, couldnt the multiple versions of the same asset loader just for different file types simply be done using generics?

[@kangalio]

Yes... Another case of reaching for macros to fast as the low friction way to fix the problem

[@robonator2945]

​@@kangalio I mean is there any real issue with using them here? It seems like that'd just be personal preference to be honest. I can see why people might not like it, but I can also see why having a flexible generic system that you can be confident isn't going to incur runtime performance penalties is something that people would prefer.

[@tathagataroy5153]

Hi! What tiling manager are you using? Thanks!

[@dylanwilliams9860]

Would the macro provided not be a security problem since it has arbitrary code injection in any asset you want, even fake assets?

[@LegoDinoMan]

When the world needed him most, he uploaded.

[@costelinha1867]

Oh, look who's back.

[@paulosantana9607]

Nice colorscheme bro (y)

[@japrogramer]

It makes so much sense 😮

[@tonyoallie]

Why is it when I look at your code. I can read some of it even though I haven't looked at rust in months hell almost a year. I'm learning bash right now with linux so your code makes some sense. The macro is nice. Glad to see you're still working on your game. I hope to see it complete.

[@CielMC]

1. Wow, I didn't know RON existed, that's cool. 2. What editor is that? I want to say NeoVim but I also get a bit of Emacs vibe. 3. You can make the ? and ; parts of your macro optional entirely by using a similar syntax to multiple: $(params)?, which you can then optionally have the macro generate with the same syntax, this way you don't have to end all loaders off with ;?. Same for the handler method, since some dont need it, it can be simply not generated.

[@pro_gemer]

your vidoes are magic

[@o_2731]

I love your energy😂

[@undersquire]

I've ran into the exact same problem in my game where I have dozens of asset loaders. However i found that creating 'finalization' systems for assets is actually not necessary in most cases. For your Hero example, where you need to store a handle to another asset, you can actually make use of Bevy 'asset dependencies' to achieve this. Inside of your Hero asset loader, you can utilize the path to the Spell asset and load a secondary asset using the AssetLoader 'load_context'. Here is an example of what I do: let mut hero: Hero = ron::de::from_bytes(bytes).unwrap(); let spell_bytes = load_context.read_asset_bytes(hero.spell).await; let spell: SpellAsset = ron::de::from_bytes(spell_bytes).unwrap(); let spell_handle: Handle = load_context.set_labeled_asset("spell", LoadedAsset::new(spell)); hero.spell_handle = spell_handle; load_context.set_default_asset(LoadedAsset::new(hero)); This completely eliminates the need for another system running to finalize/post-process assets. This also will be easier when Bevy Assets V2 rolls out in 0.12 (iirc). The only downside is that with my example you cant reuse existing asset loaders to load dependency assets. You can actually do this via LoadedAsset::with_dependency or LoadedAsset::add_dependency, simply by using 'load_context'. let spell_handle: Handle = load_context.get_handle(&hero.spell); load_context.set_default_asset(LoadedAsset::new(hero).with_dependency(hero.spell.clone().into())); On the plus side, with the first method you know that all dependency assets are loaded once the main asset is loaded, since the asset loader for the main asset has to load everything itself. If you used LoadedAsset::with_dependency etc, the main asset (in this example, Hero) may load before its SpellAsset loads, so you can end up having to handle the case where a Hero is loaded but its spells aren't. Excited to see where this project goes!

[@troligtvisme]

Magic macro :) love it

[@RenderingUser]

I've always saw macros as arcane magic. But it's looking usable now

[@stysner4580]

It's really not that bad. The syntax looks very daunting (and is sometimes annoying or badly readable) but the logic in itself is fairly straight forward. macro_rules! is basically just a way to have "automatic compile time for loops" with odd syntax.

[@SabeDoesThingsChannel]

Can we talk about how TanTan is not verified? What the heck youtube!

[@Tobiky]

Good video

[@NEO97online]

This is really interesting. Is there a reason why you're using bevy's Asset abstraction for all these files instead of just deserializing them into Resources in `fn main()` before App.run() is called?

[@xplorethings]

Looks to me like bevy loads them asynchronously along with other assets. While it's probably not a huge deal for these, it can save time on startup.

[@Snailsxd]

nice to see someone using the helix editor haha

[@codingleo]

And this is why people love rust

[@DooMWhite]

How maintainable is it?

[@fabricefromage2076]

good vidéo (i didn't see it yet)

[@shalevh]

Actually helpful though, especially to a person learning game dev in Bevy. Bevy is hard to learn

[@Otakutaru]

would it be hard to write Deserialize trait for the spellhandle?

[@valshaped]

Rust's "orphan rule" prevents this: you're allowed to implement traits on types *if and only if* you own the trait, or the type, or both. Only Serde or Bevy could implement Deserialize for a Handle. The only edge case is when a type implements Deref for your type. In that case, Rust will let you implement any trait on the foreign type.

[@johanngambolputty5351]

Just the example I needed to start using macro_rules, hopefully I don't end up doing anything too atrocious ;)

[@oglothenerd]

What is the difference between .add_asset() and . insert_resource()? (App::new())

[@Davi_Dash]

so you not gonna talk about the fact that you can do you own programing language in rust using macros ?

[@BrianBarrett0214]

nice

[@demenevdmitriy4175]

I also expected you to do include_bytes for extra performance.

[@iglobrothers645]

Witchcraft

[@francois1e4]

Why not use generics ?

[@foodstuff]

Just another person telling you to use the "paste" crate!

[@usedcolouringbook8798]

I would liked to know what the symbols did in the beginning, not find out till the end; any way, great demonstration of application but explanation feels backwards.

[@mirkopassoli9418]

Maybe because you know rust/macros in rust, I'm not familiar with neither of those, so for me makes sense that you explain first how to make a basic macro, than you explain the symbol shenanigans.

[@williamist]

rust macros are great

[@sinom]

I've seen some similar things done with C macros and honestly I don't think I like them. In general hiding the definition of things like this just feels like a readability nightmare

[@diadetediotedio6918]

nipah~☆!

[@Sergi_Ferrez]

Rust does not have generics? I think that maybe it would be more cost effective.

[@menaced.]

C++ macros are much easier more elegant imo

[@AlFredo-sx2yy]

and they are the same as C macros, only that you can put C++ code, including templates, within them. Only thing i dont think is elegant about C macros is needing '\' at the end of a line to concat the macro, but other than that, they are perfect

[@valshaped]

Rust macros operate on the AST, and can understand the structure of the code you pass into them; this is just a really *disgustingly ugly* use of them. Really, the syntax of a Rust macro is arbitrary. They can be beautiful, and hygienic, if you want them to be.

[@lixou]

I Never saw someone using pub use for a macro. Hek I didn’t even know that’s the base thing behind it. So that’s what #[macro_export] does, right? PS: nice bevy 0.11

[@stysner4580]

Macro export is to export it outside of the crate. Then you need a pub use inside another crate the use the macro. He's probably not sharing the macro to another crate but rather declaring and using the macro in the same crate.

[@christopher8641]

macro export would work fine and just "moves" the declaration to the root of the crate it was defined in. Both ways work but I don't think you need to pub use unless you are very concerned with the pathing that you source it from

[@mohamedimranechehabi5735]

Hello, I am new to bevy and I would like to know where I can learn more advanced things about it please. And thank you for your time.

[@colin_actually]

I like Bevy in principle but hate absolutely everything about it in practice.

[@stysner4580]

I don't like it either. The insane builder expression to just create a simple app quickly piles on, and most of the syntax is already verbose on top of Rust's verbosity. Whenever I encounter Bevy code I don't even try to read it anymore, just looking at it makes me tired.

[@mikapeltokorpi7671]

🤩 OTOH: saved 1028 lines of code? You would not have a long career in the Musk empire. 😅

[@windwalkerrangerdm]

My takeaway from this is to never step into rust for making games.

[@seanperry3667]

Nah, this video makes it sound complicated and should not be. See the other comments suggesting smarter solutions.

[@windwalkerrangerdm]

@@seanperry3667 I mean I really want to. I need something on the sideburner and this seems like good venue.

[@stysner4580]

Asset type should be ty, not ident!

[@OfficialViper]

Hey, this is funny! Bevy's syntax regarding the syntax loader sucks, just as my syntax using my own "BoxedFuture" type. Bevy devs and I actually defined the same type with the same name for the same problem. So let me explain why this syntax sucks and why the bevy devs did it this way. The thing is that Rust does not really allow traits to define async methods. While this has been addressed in the new Rust release and traits can indeed have async methods now, it is still breaking the rules for a trait to be object-safe. This means that a trait declaring async methods cannot be used as a trait object. For example, if the AssetLoader trait was defined this way, you wouldn't be able to make a Vec containing Boxed AssetLoaders with dynamic dispatching. You wouldn't be able to generalize multiple traits implementing AssetLoader as a list of AssetLoaders. To get around this, we have to understand what async means. It's actually just syntactic sugar for wrapping your return type in a Future. So by not marking your method as async and manually specifying the return type to be Future, you kinda get the same result. To make everything work, you also have to wrap your Future in Pin and Box. Let's say your return type is T, then your method's return type would end up as Pin = Box

[@Qohist]

thats just prefrence.

[@Qohist]

also overrated*

[@felixjohnson3874]

SEXTENSIONS Change my mind

[@0xpatrakar]

Ok but why would you make this mess? Most of this duplication can be removed using traits with default functions and generics. Perhaps costruct a simpler macro on top of that.

[@The6DAZ6]

So... I have wrote code in many different languages and Rust is in my radar but I still didn't have the time to study it and I know basically nothing of it, anyway... Couldn't you simply create a public generic function to handle that?

[@benjj280]

*promosm*

[@cayhe]

Dont copy me (Tantan)

[@heliusuniverse7460]

i did this using generics

[@charetjc]

Got a link to the code? How did you use a generic to work with structures that have dissimilar field names/types?

[@juliohenriquesantos2190]

Please, make it open source! :(

[@LetrixAR]

No thanks. I'm fine duplicating code.

[@zenshade2000]

Often the best choice, even if it removes some of the "fun" of "being brilliant".

[@LocalTrashyt]

bro this isn't even rust full power. Wait until you use proc macros.

[@nephisto2]

Chad memes are bad for you. :(

[@lovely-shrubbery8578]

ur keyboard rgb offends me

[@edhahaz]

Rust is elegant, simple and productive

[@casualuser5527]

your pronunciation of the parameter is not standard.

[@willjohnson6714]

your video is great, but rust and bevy are an absolutely shit show -- look at all this insanely ugly syntax -- rust SHOULD NOT BE THE "FUTURE" of systems programming :S

[@zenshade2000]

This isn't AT ALL typical Rust or Bevy. Tantan is off into extreme wizard land with this project. Rust is quite beautiful when it's strengths are used the way they were designed to be, but not when it's contorted to its limits to wring every bit of automation possible out of it in the name of shortening lines of code.

[@zeocamo]

this is where PHP or JS is better, this stuff is where Newbie misunderstand and add ca. 342 Bugs per day to the code, making your Job the debugger of the team, yes i try this it is not fun. in Dyn. languages this can be done with a few lines of code. and that is why some thing like Go is nice(if you want compiler) as it don't do this all of this Magic that only the good devs. understand, 4/5 of devs out there will make a lot of bugs and problems and you need to use the same time in review that you need to write it your self.

[@AndrewBrownK]

I can’t tell which things you are arguing for, and which things you are arguing against. Macros “better” in dynamic languages, but lead to a bajillion bugs that are hard to debug. But then macros can only few people can comprehend in compiled languages, so better to have no macros when compiled like Go.. Easier to write (dynamic) does not mean easier to write correctly. I’ll choose difficult to read compiler messages over difficult to debug runtime errors every single day of the year. And maintaining 100 copies of code with slight variation (Go) is WAY more work and error-prone than having the patience to cooperate with the compiler for one macro. In rust, that 1 out of 5 developer that can make a good macro can easily save the 4/5 other developers from a lot of headache.