A.I. learns to drive on a pipe

[@ArmageddonNerd]

I love these. Seeing all the generations of the AI is so visually interesting and you've got a great eye for cinematic shots.

[@yoshtm]

Thanks :D

[@Wilker_uwu]

@@yoshtm i agree too. i also like how the information on screen is easy to spot and see too. ^^ being a bit nitpicky, if you have anything to improve, is just a bit of the color scheme on the cars, specifically in the comparison, just to improve the accessiblity a bit. you could try blue-orange for contrast, maybe change the wheel colors a bit, or maybe use the car model without the spoilers to highlight one with the spoilers so it's easier to spot. other than that, you did a nice job with the visuals :3

[@yoshtm]

@@Wilker_uwu Thanks, noted ;)

[@enaftenvedsvanefossen6102]

@@yoshtm same

[@mooskoop]

Idea: Use the new tool to extract inputs from replays, and use wr runs as input for your AI. You could possibly set new TAS wr's

[@yoshtm]

It would be intersting to try !

[@TEC_Sage]

@@yoshtm i’d genuinely love to see how this would turn out

[@lucasfourneret58]

That would be interesting indeed

[@ventilo1825]

other idea : use TAS imputs to train the AI

[@boerhenkcb]

Can someone translate to analoge language? What are WR runs and TAS? Thank you!

[@donadigotm]

Awesome video yet again. Explained in an easy way with not too many details and entertaining as well. Shows the process one has to go through to train even a simple model, but it's also interesting the genetic algorithm found different strategies to pass the same track and that the final network is able to generalize to other pipe tracks. Excited for what the future holds.

[@yoshtm]

Hey donadigo, thanks !! And congrats for everything you've brought to the game lately, I've become addicted to TAS world record videos ahah ! Yeah even a simple model requires more work than one might imagine ! What I think is very interesting with the pipe map is that it shows that in some cases, trackmania environment can be described with a very limited number of inputs, so it shows potentiel for fast learning with genetic algorithm / reinforcement learning methods. And the supervised learning part also shows that a neural network can really immitate well the decision process of a human for trackmania driving : the A.I. really plays the same way I do in the end. So maybe A.I. could be an alternative to TAS in the future, even if I think it will take a long time :)

[@galanlefont1600]

Great video!! Those shots of thousands falling and hundreds running are incredible. What a work of art!

[@yoshtm]

Thanks a lot !

[@black_m1n825]

We need a series called "Can A.I. finish Trackmania Nations Forever"!

[@akatsukilevi]

With enough data, yes

[@yumeiraosu]

@@akatsukilevi It will be fun to watch

[@t34d1um]

i wonder how much generations it would take haha

[@akatsukilevi]

@@t34d1um Thousands and thousands....

[@abebuenodemesquita8111]

@@akatsukilevi and time

[@kgaming7599]

*The entirety of the video:* How to make a fluid simulation with cars

[@DigitalJedi]

There have actually been studies that show that traffic flows can be modeled by fluid dynamics equations.

[@kgaming7599]

@@DigitalJedi Wow lol

[@JorisKeijser]

Except that fluid can't change direction on its own and is purely subject to laws of physics like gravity. Water will ram into a corner, a car will turn with it.

[@kgaming7599]

@@JorisKeijser Well, now we know who's poopin' the party

[@JorisKeijser]

@@kgaming7599 Now we know who has nothing of value to say.

[@Merivio]

When I did a machine learning project for a fighting game, I realised that 'memory' or familiarity with a scenario played a big role in my AI's learning ability. In your case I would add another input for non-supervised runs: how far along the track they are. While this (on the face of it) defeats some of the point of a generalised learning algorithm, humans do a similar thing to get over difficult hurdles themselves. When a human plays a track 100 times, they're not just learning how to respond to what's in front of them, they're memorising what's coming next and preparing for that too. Furthermore, we utilise a kind of deja vu effect where if a particular encounter reminds us of a previously experienced encounter, we react the same way. We also chain together groups of inputs, like a combo or something like what you find in tutorials for solving a rubik's cube; if one block is out of place, it can take a dozen moves to fix it, but it's always the same dozen moves, and we learn to ignore the state (to some extent) when we're in the middle of the combo.

[@Narblo]

How did you extract from the game the inputs to the AI?

[@Merivio]

@@Narblo this particular game was designed to be played by AI and had a dedicated library for the AI to interface with. In cases where a game isn't designed that way, I'm sorry I don't have that answer.

[@shortzProductions]

seeing thousands of cars falling down the pipe was nice to watch, a good old reference to those k-projects. :)

[@florianlemysterieux1689]

Tes vidéos sont à la fois hypnotisantes, super intéressantes et très impressionnantes, toujours un plaisir à voir :)

[@yoshtm]

Merci beaucoup !!

[@an0maly404]

@@yoshtm Ah, je me disais bien que cet accent me disais quelques chose ! Je suis d'accord avec lui, très bonne video ! ^~^

[@uvuvwevwevweonyetenyevweug5884]

@@yoshtm Très bon accent ! Je me suis abonné

[@psyk00o]

Totalement d'accord.

[@kittencorp.3295]

Baguette

[@crylia7611]

Let this ai try the patience map "calm down"

[@Extramrdo]

Yeah, that's where I thought this was going, esp. after seeing Wirtual deal with it.

[@klaernie]

@@Extramrdo I had the exact same thought! That would be really fun to see - after all the AI does not know fatigue

[@AbsoluteHuman]

This map has not only turns, but some branchings also though, hasn't it?

[@klaernie]

@@AbsoluteHuman no, calm down only has a single route - but a couple hundred corners iirc. Essentially it could be optimized to a straight line without branches

[@AbsoluteHuman]

@@klaernie but the network doesn't have a map so it can't choose turns

[@7verdy]

Honestly, that's nuts! As a CS Student and a huge racing games fan, I definitely enjoyed this video a lot and you motivated me to start working on the concept of AI in Trackmania! Anyway, great video, keep up the good work

[@bs_blackscout]

I strongly suggest tackling the basics first to not get frustrated. I talk from experience. Python, then simple machine learning, then some algorithms, then specific algorithms, then learn to integrate the API with Python, then implement your ideas into the game using AI/ML.

[@sot8546]

I relate to your comment so much

[@neoblackcyptron]

@@bs_blackscout is there an api in trackmania that is open to users. I know Machine Learning, in used it to build a cnn to denoise ecg signals via supervised learning. From this video. 2 things are a challenge. How do I record driving data. To be used for training. How can the ai send inputs into the video game. Does trackmania have api calls for that? Thank you. If you do know any game that has apis exposed to the users. Do let me know. I need to experiment with ML + Reinforcement Learning using genetic algorithms. A video game with visuals like like a good way to see how the algorithms are doing.

[@tom-hg1ko]

always stoked when a new A.I. video of Trackmania comes up

[@Airblader]

I would love to see the combined approach where you use supervised learning to "seed" the model and then let it evolve from there.

[@eliangodrix26]

Yosh today : creating I.A for trackmania Yosh in 2022 : WR with E-sport I.A cars at the Z-events

[@aest4s]

Didn't expect that ending, thanks for the shoutout :D such an interesting video and very well made!

[@yoshtm]

Oh, hi :D I love this cut ahah, thanks for the comment !

[@rxn7]

O kurwa jjayjoker

[@-Siftn]

Wow, I was loading into Forza Horizon 4 and clicked on this video, as I get distracted by the video a F-Zero X song starts playing @ 16:50 and I come to the comment section to say wow such a throw back! I finally click through the load screen on my Forza and then you start playing the Forza load in music and I was like wait.. what! That absolutely blew my mind. 🤯 Can't wait to see what we can learn from AI in the future, keep it up! ❤️

[@ViktorVaughn135]

Bro what that’s sick I wouldn’t of noticed it without you

[@sonicschall]

I was like "oh do I have Spotify running?" when the music played LOL

[@satibel]

If you're interacting with the game automatically, you can use a directx interceptor/shim/wrapper to cull calls to the gpu and be only cpu bound. Then you can start more instances of the game.

[@asj3419]

Just realized that the straight pipe situation is basically just engineering a control system. The better the system is at keeping itself straight on the pipe, the faster it can travel, so the goals are aligned. Not sure how you'd make use of this fact, but its nice.

[@Wshaf97]

Is there a way to run multiple instances of the game and feed them into the learning? Also speeding up the game does not affect the physics so assuming the inputs could be entered correctly could this be used to speed up learning? I have some newer hardware and would love to be able to run AI to learn different maps and techniques.

[@MrJCraft]

I am wondering the same thing, like a threadripper is what I am thinking, why not just run multiple instances

[@Wilker_uwu]

at this point it would be better for the long term to rewrite the game's deterministic physics into another piece of software, so you could simulate multiple instances of the runs at once without actually having to launch the game multiple times (and logging in to let the drm let you play before you can set it up, which would be tedious and time-consuming). if there's any means to get the numbers right (e.g getting block IDs from the tracks' gbx files, and collision boxes geometries), you could, in theory, simulate any track made in the game multiple times at once without having to render any visuals (so lots of memory is saved which means faster simulations), but also, if any people are willing to do this, that means we get a drm-free, Free/libre as in open-source equivalent of Trackmania, which would be neat

[@MrJCraft]

@@Wilker_uwu I think it would be easier to just run the game multiple times, I know of a few people that have workstation pcs with 128 gb ram, 32 core thread-ripper cpu, and a 3090 GPU, so it would be possible to run many instances of the same game on the same hardware, but it would really depend on how he is doing it now, in fact if he really wanted to make it faster maybe use cheat engine to run the game at a faster rate so reverse cheating XD making the game do the physics twice the speed or more but you would still run into the issues with not having multiple instances and when you are trying to do thousands of attempts max speed is always best, so of course re making the games simulation might be faster, but it might be the exact same depending on how the API of the game works, there are many unknown factors in how it works because I have never tried to do something along those lines, though I have done some python coding, also there is the a possibility that the API may not allow for multiple games being run at the same time, but that should be able to be circumvented, but it depends on a lot of factors that I and you can only assume but it would be interesting to see what yosh's ideas are to improve the speed

[@adamnielson42]

Could you use the same cheat software Riolu used, and speed the game up instead of slowing it down?

[@galeniium]

​@@adamnielson42 Speeding up the game would indeed allow you to do more runs. But the problem is that you would need to also increase the frequency at which you call your neural net in order to keep the same number of adjustments by the neural net. This may not be feasible depending on the computation speed of the NN.

[@blackmagehedonist]

It is so fun to watch the mass of AIs move almost like some sort of odd flocking behaviour-though I know it is not flocking, it does still look of it!

[@danielreich5266]

This is super amazing and satisfying to watch! Thank you lol. Can't wait till AI tos speedruns become more possible.

[@nathan6192]

Super intéressant pour quelqu’un qui n’y connaît absolument rien en IA ! Très bien expliqué et super niveau d’anglais ( et top l’accent) ! Super taf !

[@yoshtm]

Merci beaucoup !

[@binary_dragon_]

12:16 that transition!! damn so good!

[@Lmjacks]

combining the two approaches seems like a really potent combo for AI in this game. from your testing, it’s pretty clear that getting it to start in complex scenarios like this is extremely difficult and far too slow, but with a bit of example data it can have a very solid go at maps like this. from there, i’d imagine evolution and mutation would be really strong tools to overcome the fact that the dataset was provided entirely by you

[@ekyo_stuff]

At the start It looks like the car is melting into a million cars and it's so satisfying... Great stuff!

[@fairweathertrains3029]

At 18:50 thats a great example of the 'Butterfly Effect' in chaos theory. Great stuff

[@chriscalderon1337]

I was wondering if you would release another video! Your track mania AI videos are always fascinating, keep up the good work. :)

[@j-l_rossyde6868]

I find it really interesting how they wobbleon a staight pipe, but i guess theyre just constantly correcting thier oversteering

[@ただAdam]

Just so good vid. I hope you'll get much more fans such as me.

[@t34d1um]

this is what i go on youtube for, i don’t play trackmania but your dedication and passion about what you are doing really makes the cut for me ! i can feel you worked hard on this vid and it pays off, great content man, keep it up :) btw : ça fait toujours plaisir d’entendre un accent français par-ci par-là, continue à faire ce qui te plaît mec

[@yoshtm]

Merci beaucoup ;)

[@АлексейТарасов-ц1с]

Man this makes me want to finally start studying machine learning

[@nick.3455]

You should definitely do more of these a.i things they're so captivating

[@marcmontero2260]

I really really love this AI videos in trackmania, genetic algorithms are fun to watch and combined with trackmania, one of my fav games, is such an enjoyable experience. I'll continue to watch all your videos, and i hope one day i'll be able to see some AI generalize all kind of diferent maps

[@yoshtm]

Thanks a lot !

[@FrostKiwi]

So much work, so much expertise, so much quality. I am in love

[@14Yuke]

crazy how fast some of these players for example Hazard or BlueFeed are on pipes!

[@decb.7959]

It would be interesting to use the supervised model as a starting point for a genetic algorithm. That would mean that it starts with the basic ability to complete courses, but it could keep making improvements beyond the initial training data.

[@Ender_Onryo]

This was very interesting, well done. I was expecting this to end with the Ai clearing the track called Calm Down. I think that was the title. It was designed as a patience endurance test and is a giant Pipe Maze in the sky with no check points.

[@santlattuf]

Great video! I really enjoyed it, you explain really well and the cinematic shots are really well done. Keep it up!

[@ryanehlers1081]

The production quality on this is superb.

[@wintlink]

Vidéo franchement incroyable, c'est cool a regarder, les plans sont très agréable, et c'est bien expliquer. Les musiques que tu utilise sont franchement très sympa aussi !

[@yoshtm]

Merci beaucoup ca fait super plaisir ce genre de commentaire ;)

[@benjaminkindle1841]

You're good at making AIs, and telling an interesting story about it. Keep it up!

[@pi-stone460]

J'adore tes vidéos vraiment, WOW ! C'est vraiment génial de voir à quel point la programmation c'est puissant. Tu as déjà pensé à refaire un petit trackmania (genre le recoder) pour pouvoir accélérer le temps et faire plusieurs itérations par minutes ou secondes ? Vraiment gg ;)

[@yoshtm]

Merci ;) Trop compliqué, pour moi en tout cas. Mais ca serait très pratique c'est sur !

[@RJnospace]

just here to say that i think you did a fantastic job on this video mate it was a great watch

[@aynyxamer]

bravo! incroyable travail, je n’y connais rien en programmation mais tu as réussi à me captivé encore une fois avec ton A.I. ! Bonne continuation, j’espère que tu es fier de ce que fais :D

[@yoshtm]

Merci beaucoup !!

[@CocoCode]

Great video - good job on sharing progress along the way 🎉

[@topphatt1312]

0:38 the most terrifying or maybe coolest clip

[@AwesomeNick94]

When the AI starts off it’s like you go to cut into a cool looking cake but it turns out it’s made entirely out of live spiders

[@etiennelamole9565]

12:15 This moment is so epic, I really appreciate your work

[@petersmythe6462]

Another thing that might help is value learning. In other words, make a higher level network which only does one thing. It attempts to predict which combination of inputs and outputs will lead to good performance. Then it can use backpropagation to modify the network live depending on whether it thinks the input and output combination is appropriate or not.

[@petersmythe6462]

The way the value prediction network learns is by looking at how close it's time-averaged prediction matches reality. If it thought a car was gonna do really well until the moment it dropped off a cliff, then backpropogate a strong punishment. If it thought a network would get 77th percentile and it got 83rd, then that's quite good and the network should be reinforce itself with a reward.

[@melvyn_son]

Super vidéo et très intéressant et agréable à regarder, tout comme les précédentes 👍 Merci pour ton travail 👍

[@adeo]

You can use cheat engine to play the game at 200% speed or more, given that it's a deterministic game the results will be the same, but you would save a lot of time

[@sibonyhugo5887]

Yes but in that case the AI would train his input to be 2x too fast. So you couldn't just apply the result in the game.

[@adeo]

@@sibonyhugo5887 after the training you can just slow down the AI

[@yoshtm]

I get game data ~60 times / second, so I can't accelerate the game speed too much if I want AI to get data / make actions with a sufficiently high frequency

[@adeo]

@@yoshtm I think that frame perfect inputs are useless in TrackMania, so if you speed the game to 200% while recording 60 inputs/sec the AI would then be able to perform 30 inputs/sec when slowed down, still more than enough. I would go as far as speeding it to 300% to get 20 inputs/sec, that's still an input every 3 frames at 60 fps

[@Unknowiii]

@@adeo So you want the AI to play with lag while it's already struggling ? What ?

[@dropthepress]

Please continue with this series! This is amazing! If you're up for game suggestions, the Trials series is really good (newest one is called Trials Rising). It is a 2d platformer with a dirtbike and the element of precise balance might be very interesting. Also, trials loads very fast and the relays would be easy to capture!

[@petersmythe6462]

I think the thing to do is start with supervised learning then change the reward function to something more objective-based. This way, the AI starts by copying you, but then gets better and can unlearn your mistakes and weaknesses.

[@ipokiller2g940]

Looking at the car liquify at the start feel like something new to me :p

[@wompastompa3692]

Love the F-Zero X music. Now set the AI loose on the giant Hilbert's Curve course.

[@__-bk6mm]

LMAO the cars that fell off the pipe right at the start, now that is a real track mania player there :)

[@mayz2572]

I didn't knew i would be this much interested about IA

[@kajacx]

I think combining the 2 approaches would be the best. Train a bunch of networks using supervised learning, and then use that as your initial population for genetic neat algorithm. I think that is how they trained Alpha Star in Starcraft.

[@trulyinfamous]

I love Ai learning projects. They are always super interesting. A bit of constructive criticism: it would be nice if you put a list of the music you used in the description, along with the times they show up. I hate it when I hear a cool song used in a youtube video, but don't know what it is.

[@Nerpson]

N.E.A.T video as usual! :D Your final step: create an AI to recognize the map style, combined with an AI which finds the route, combined with an AI which recognizes the surface types and finally combined with an AI which drives according to all this data. In a few years maybe haha

[@Pvpiesz]

amazing video! you really hit me with that Forza Horizon soundtrack :D

[@DreadKyller]

If you used the supervised neural network as a base for a genetic algorithm, instead of starting from random/scratch, and having the first generation consist of a few of that network along with small variations in that network, you could give the genetic algorithm a good head start and allow the small randomness between generations to slowly optimize the movement to become faster. Switching between supervised and genetic algorithms is a very useful tool that it used quite often when training AI's.

[@tainted_bread]

This man actually recorded this video in the fucking catacombs, bro. That echo's wild

[@GGenoce]

"But then out of nowhere, the A.I. got this legendary run..."

[@satibel]

I think taking the replays as a new data set, then using supervised learning to train a few decently good AIs, which you then use the values of to train a generic algorithm instead of seeding with random values might give a pretty good result.

[@ENCHANTMEN_]

Watching the car melt at the start freaked me out there

[@paxon57]

For something like that I've been thinking supervised learning at first and then using genetic algorithm to finetune and improve, would be interesting to see how much (if at all) better it would get. Also, great videos! Sub!

[@meneldal]

Yeah I think that could work too, use supervised to get it started so it doesn't fail entirely at driving then a genetic with slower mutations for improvement.

[@panmig]

Just found this channel and I love it!

[@gary13001]

You deserve a lot more subscribers, keep up the good work :)

[@NTVE404]

Everyone else: enjoying the video Me: noticing steeple of aquatic rallies music in the beginning

[@RyushoYosei]

I love watching these videos, and really helps me want to learn how to try making and doing some of this myself, though I haven't gotten to learning the programming from scratch, most Neural network stuff I mess with, has been more learning watching other peoples code, and seeing w hat happens if I change values the few times I get to try, or using the Unity learning stuff. Would you ever consider sharing any of this, if anything to potentially the idea of getting multiple people to run training on a few courses, and sending you the replays, or something back, or seeing how different computer trained AI's might figure out how to solve the same track.

[@idirhouari3878]

Trop cool ! Such a nice video ! I think you deserve a lot more subscribers ! Your content is interesting, your explanations are clear and it is really satisfying and original ! Keep up the good work man :D

[@yoshtm]

Merci beaucoup ;)

[@cass9929]

The first 3 seconds perfectly describes how I feel in the morning

[@cyb3r._.]

Great video! This is my second time watching this video after about a year, and somehow this idea didn't occur to me the first time: I just wanted to ask if it would be possible to redo this video and maybe also some of your other older Trackmania AI videos with TMInterface like in your newest video, allowing you to do many more iterations than before; maybe then, reinforcement learning will work even with turns! Please consider this if you see this comment, and thank you for your time! Keep making these amazing and also interesting videos!

[@zine-e]

J'aime la musique que tu utilises au moment des premières générations ! (Also, it's "funny" to see every time cars falling down at the start)

[@ReVDeatH]

First 2 seconds at 0.25 speed is TM version of those oddly satisfying videos.

[@itisrandomidk903]

0:30 all the red cars look like a bunch of ants attacking someone

[@aaronryanowo]

I love hearing you talk

[@HeyGuys_Hi]

Une nouvelle vidéo et toujours le même plaisir À la regarder tu fais vraimment du super boulot

[@samb9645]

Love the hades music in the background at 1:16

[@adaranlives8515]

Love the hades soundtrack in the background

[@CYXXYC]

TM has deterministic physics, it is possible to run multiple runs at same time headless (without real timer and without graphics), although might require a lot of work on programming side

[@ShcrTM]

Community: Look at this guy having flickering inputs, he must be playing in slow-motion to get such good runs. AI: Allow me to introduce myself.

[@Т1000-м1и]

7:27 the ai do be dancin tho

[@krokkoguy]

You should use transfer learning, using the neural net trained on your play as a basis for the unsupervised evolutionary algorithm. Transfer learning is a really effective tool to get over that initial hurdle

[@anesthetized7053]

your videos never dissapoint! very well done

[@nicob4232]

On apprécie la qualité de ce que tu fais, que ce soit dans la programmation ou dans le montage. J'attends la prochaine mais repose toi un peu tu l'as mérité ;)

[@lhuitre2644]

Great video, as interested as usual !

[@ziwuri]

You should put this AI through the infinite pipe maze and see if it can get a world record there, since it's just the same pattern of turns for a very long time!

[@wappa6914]

Une nouvelle vidéo de Yosh ça fait plaisir 🥰

[@mrmunchkin2181]

Why not have checkpoints after corners so the ai can perfect one corner and improve it's time there which would help it in later corners? Like in CodeBullets Q-learning racing ai if my explanation was bad :p

[@nootaboot7042]

i don't know why you used the snake eater theme but i am very happy you did

[@thegreatwebstar]

This is actually what life looks like

[@gugus8081]

Nice edits ! Super interesting to see what's possible with this :)

[@RainyMusic]

Snake Eater et A moment apart dans une seule et même vidéo un homme de goût

[@krebsebubrek552]

The first couple of seconds are my brain during a test

[@dec35]

Super tes vidéos !

[@lemoaj8315]

The Music in this vid is on point!

[@ovrsurge4689]

Just in case you didn't know (he didn't list the music sources in the description) Most of the music I've heard in this video is from the Hades OST. a lot of banger tracks there.

[@AraiKay]

VERY nice video! only one small problem would be the sound. The "S"s were peaking a lot and were a bit harsh on the ears. but I might just have sensitive earing idk. I'd recommend watching some short tutorials on how to improve audio easily! :) It's still a very cool video. It mixes together two of my favorite things! AI and Trackmania, what more could you ask for!! :D

[@PsYkedelick]

Excellent job, I love your videos. Keep it up, for science !!! :)