Training an unbeatable AI in Trackmania

I trained an AI in Trackmania with reinforcement learning, until I couldn't beat it.
I just opened a Patreon page, where you can support this KZbin channel with a donation (link below). Between research, programming and editing, these videos take a long time to produce. Any support will help me to spend more time on that in the future ! This one in particular took up most of my time over the last six months. I hope you like the result, and I'm open to any feedback to improve the quality of these videos :) I'll post more details on the Patreon page to explain what I plan to do next.
• Patreon : / yoshtm
If you want to try these maps, you first need to have both Trackmania Nations Forever (TMNF) and TMInfinity installed. Then, you can download the maps on Trackmania Exchange (TMX). Everything is free, and you can use the links below.
• TMNF : files.trackmaniaforever.com/tm...
• TMInfinity: unlimiter.net/download
• Download the maps on TMX : tmnf.exchange/tracksearch?que...
Contact
• Discord : yosh_tm
• Twitter : / yoshtm1
Special thanks to Donadigo, for TMInterface !

[@culleng.1653]

Showing the AI's multiple training runs simultaneously and color coded is one of the most gorgeous and elegant depictions of machine learning I've ever seen. Thank you for this video!!

[@kossboss]

Yeah about that. @yosh how did you do that just curious. Is it part of the game?

[@1000-THR_Benjamin]

@@kossboss If you mean the color coding, he most likely just made it so that the color goes from red to green, and the more rewards/points it has, the greener it will be

[@ifyoustaywithmeyouwillneve1678]

@@kossboss i think the game comes With a tool or there's popular plugins for it (it has a huges player base) to show a lot of 'ghost' laps, there's lots and lots o vids outhere of videos showing that. For the coloring, i guees is what the previous comment said, anyway the game allows to make your own liverys and some people use mod the models. To look like something else, so probably is a middle point.

[@bcbble]

​@@kossbosschanging the car skin based on how far it gets into the track.

[@Todesschleicher]

​@@kossboss at 20:31 he thanks "Alyssia" for helping automate the car skin changes, so I guess there's some sort of script involved

[@One_Stale_Donut]

Insane production value, the editing that went into this, and the time it must have taken.

[@fluffsquirrel]

Ai still can't beat that! (hopefully)

[@principeSZN]

it took 3 years

[@fluffsquirrel]

@@principeSZN But the thinking, planning, learning up to that point played a huge role in shaping how he figured these procedures out and how to best interest and reach the audience. This is the culmination of all his hard work. His legacy began 3 years ago, and he's still going!

[@rolandasgrigaitis708]

It's not like he's doing these videos for free, genius. You don't need to appreciate him for something that he got the money off of viewers like you.

[@tophmyster]

@@rolandasgrigaitis708 L take bozo

[@florrex4136]

That drift-overtraining and then training off the excess is a really interesting approach to let the Ai discover new mechanics👍🏻

[@SumGuyLovesVideos]

this is what coaches do when they train a new skill for humans, it's interesting to see it applied here, and then removed

[@nemou4985]

Similar pedagogical approach to human learning in general

[@teakfreeman3543]

@@SumGuyLovesVideoswhat do you mean that’s what coaches do? I’m not following

[@SumGuyLovesVideos]

@@teakfreeman3543 take basketball for example, the coach doesn't just have the players play the game and hope they discover all the different skills, he sets up drills

[@SumGuyLovesVideos]

@@teakfreeman3543 each drill focuses on developing a certain skill, like thousands of jump shots, then later in the game it's another option the player can use

[@rafaelnubile]

Now we want a race between programmers to see who makes the fastest AI!

[@zalmarzalmar3835]

You mean between AIs who makes fastest AI?😂

[@SumGuyLovesVideos]

there's several different robotics competitions that do this :D

[@KekoBit]

@@zalmarzalmar3835 I think he meant a general purpose AI which makes programming AIs that produce advanced driving AIs :)

[@vladimirslavik5230]

It would be cool on real tracks with real cars.

[@vsoriginalpoetry6696]

Should we put people in them?

[@darkranger116]

I always love these training AI videos. Its like the "wizard knows his apprentice will be more powerful than him one day, but only if he is guided there" vibe

[@AtticusKarpenter]

Like shonen training montage, but protagonist is mentor. Its cute in some way.

[@joklu6257]

I also recognized that I find this type of video extremly satisfiying and I think you perfectly described the vibe of it

[@Factoid135]

You know CodeBullet then ?

[@PaulOsbornesChannel]

The way you taught it to drift, then taught it to use drifting to go faster was brilliant

[@JMurph2015]

The AI got a whole _training_ arc if you follow my drift.

[@nemou4985]

@@JMurph2015 The topic is drifting from training to puns so fast that I can't catch up

[@theredraider3860]

Give me a brake from these puns!

[@MouhibBayounes]

Can we steer this conversation back to its normal path?

[@zizazaplayz3992]

These puns simply TIRE me

[@adkh2112]

As a an AI engineer who played lots of trackmania growing up i thoroughly enjoyed this video. Great editting too and great to follow your journey and thought process!

[@kootenpv]

The main problem with the final test is that this part of the map was still contained in the training data, and for that it means it likely doesn't generalize and instead was over fit at least slightly

[@falinoluiz5962]

Slightly? Bro the whole thing is overfit lmao

[@jolemackerel]

It doesn't seem like a problem, just a known constraint on this solution. Technically speaking: say the AI trains thousands of times on each track in existence. It has the capacity to store track-specific data and total, general experience training data. Would it be over fit or appropriately generalized for its purpose? Philosophically, is it that much different than a human player with "favorite" tracks, ones that gave the brain the best feelings to repeat and learn?

[@jsanchez23]

Pretty sure that wasn't the point of the final test, as proving generalized success was already determined to be a much harder task. The final test appears solely to prove if the ai truly was driving/performing faster than him or if it was just its consistency giving it an edge due to all the human errors and mistakes a person would make in such a long track. So the final test was realistically just meant to be the exact same test as the full map, but with the potential variable being changed being himself as the player. As it seemed more likely a scenario to further "perfect" or at least minimize as many mistakes from their own times on a smaller snippet of map.. Which wasn't exactly a viable option to avoid so many mistakes on such a long track, with many opportunities for mistakes. Which is also likely why he mentioned that there are surely far better players that could beat the ai on the final smaller track snippet, but he had doubts they could beat the ai on the full size track. As on a track that long any human is bound to slip up and make mistakes with that longer frame to go so far without any mistakes, giving the ai a chance to catch up. While still possible, it would take a lot more practice and luck. So the task of further generalizing and making the ai successful on any track, is a task and test he hasn't actually truly delved into and realized was fully beyond the scope of this video and will likely be tackled in the future. So the final test really did just serve to further demonstrate the ais biggest advantage was consistency and before training it to drift, it wasn't actually driving faster or "better" at all, it just wasn't making the countless human error based mistakes that a player would. Since once transferring to the snippet map and practicing out as many mistakes as he could, it began to close the gap and was being outpaced/raced by him. Then the drift training proved to again close the gap towards actually driving more effectively, but still appears could be outpaced if a player was able to pull off their best version of a perfect or no mistake run. Since like in speedrunning, its easy to have an incredibly skilled player that is well practiced and capable of playing the game quite well, but runners will often have countless attempts, given the numerous frame perfect or otherwise precise moves needed and they have to make all of these numerous separate and precise moves all in the same run. So if you have an ai that can flawlessly and consistently pull of those specific frame perfect moves every time, you could say the ai is faster, but the speedrunner is still likely playing the game as a whole better and the gameplay between those moves is likely faster and better performed, but its easier to make those individual move based mistakes and if they were removed from the equation, the player would likely be faster than the ai with their overall gameplay. So the final test was just trying to remove that same mistake based advantage to the best of their ability, in hopes of improving overall fitness/skill over mere consistency.

[@falinoluiz5962]

@@jsanchez23 thanks bro gonna use this for my machine learning class essay

[@jsanchez23]

@@falinoluiz5962 Not sure if sincere or bait, as I do realize how long in the tooth it all was and I'm not sure if i really said anything of merit. If you are being sincere thats cool, if not, I get it.

[@plebiain]

I would unironically watch hours of you facing off against this AI on different maps to see what it does, please make more!

[@Lee-One]

Same, this production value is insane too, I would love to see more of ai in video games, it’s really fun to watch.

[@saddlebag]

Yeah, maybe with some tracks that change elevation and tilt.

[@ZeddisDead]

Please stop using the word 'unironically'.

[@plebiain]

@@ZeddisDead How about please stop policing other people's words :P

[@cinorana]

@@ZeddisDead why would someone need to stop using a word that is used by many?

[@johnbrooks6243]

Watching your own creation grown and outperform you has such a paternal feeling its amazing.

[@ewaldlatreider5127]

Father of the AI uprising 😂

[@dreamz-plays]

i know right well your dad wouldnt feel that tho

[@zbou23]

...future generations of humans, if any, are fucked lol

[@sav22rem22]

@@dreamz-playsSounds like projection

[@LetsbeHonestOfficial]

For the AI to be able to reintroduce its new isolated drift skill to its arsenal after the reward was removed is so cool to me. I feel like that is such a massive way to teach AI new skills. Doesn't that imply that you could isolate a thousand different skills, and then have the AI incorporate it all into something massive?

[@haydentravis3348]

That is in fact how we learn. What I find interesting is we seem to be mapping out the process of learning in general, by having to create it from scratch with AI. Videos like this have taught me how to teach myself new skills, though creating a reward for random things is still not easy.

[@JeffreyBoles]

This is the third Trackmania documentary I've watched on KZbin. I have never played Trackmania. I do not play racing games at all. Yet these docs are incredible!

[@Bassalicious]

This is nuts. The sheer amount of work involved boggles the mind. Such a well paced and chill presentation too, love it!

[@wafflegmd]

Really like the video!

[@apocalypselemon2209]

I absolutely love the part where it becomes the drift king for optimal rewards. Just styling by doing micro drifts constantly.

[@TheTorbjoern]

Car was presented with a carrot and it went "bet" I honestly laughed out loud when it did it

[@Sagerz]

Suddenly started playing Mario kart xD

[@InTehFace1]

Really great work, and guys like you are going to be talked about in 10 years as the pioneers of training AI in various applications. Very inspiring

[@Sloppy_Glocky]

your editing skills are outta this world man, what a fun experience this has been. Thank you for this video!

[@benslater4997]

16:19 I love how the Ai found a way to abuse the crap out of the reward system 😂

[@useless8695]

it got too smart we gotta disable the breaks again

[@rushi7312]

it was to be expected. When you ask an algorithm to find the best way, it always will according to its capabilities and limitations. Not limitations = problems^^

[@benknoodling3683]

It's how humans approach problems too. Like in teaching jobs that incentivize pass rate over anything else, teachers will prioritize passing students over making sure they actually understand the material.

[@OldGrumpyGamer]

Same with support centers. You award the agents on number of tickets closed, you will get tons of closed tickets, but the quality will suffer. @@benknoodling3683

[@UwUEmmittationUwU]

so true XD

[@RacetasClub]

What a magnificent video and what an incredible journey. This got me excited all over again. You sir took quality to a whole new level, you & the AI should both be very proud :)

[@LavaCreeperPeople]

incredible journey

[@yoshtm]

Thanks Racetas!! I remember you played one of the maps (level 2) last year, I hope you will try again, I just posted all maps on tmx ;)

[@RacetasClub]

oh y I remember, definitely viewed it as beatable but painful to a degree hehe, although now clearly... not so easy. The last one now seems very beatable as mentioned but as for the first two... hats off, I don't see anyone beating it anytime soon@@yoshtm

[@yinyang2385]

​@@yoshtmAre you familiar with the "micromouse" competitions where they train a mini computerised car to zip through a maze. Could the algorithms there be benefit this project?

[@jed3269]

The edit and production is A1!! I loved the whole thing!

[@IAteYourDingoBaby]

Thats so awesome. I remember watching these videos years ago, and being blown away by what you were doing - even with having never played the game. Now... You've taken it to an entirely other level. Its cool to see you're still at it. When someone puts that much passion into a game so old, it only means one thing - and thats the immense embracement of the time they've shared with the game, good and bad. It takes a rare breed to really bunker down and commit to something for so long, looking past all other options, and even the cons you will inevitably find over time, really just focusing on what it has brought you and all the time you have shared with it. Props my friend. Keep at it, and keep enjoying!

[@Ell355ar]

Looking forward to seeing the AI take a shot at Deep Fear one day

[@mifluffy5196]

Its crazy with enough practice really think it has a good shot of completing it. I mean with like a toonn practice

[@inthefade]

@@mifluffy5196 The trick would be to run the simulations faster than real time on a server farm. I know it can be slowed (Riolu!) so I'm sure it can be sped up as well? It would be amazing to see where this could go.

[@MrMeasaftw]

@@inthefade The true challenge would be the pathfinding. An untrained AI would take billions(if not many factors of 10 more than billions) of simulated years to figure out how to drive the map. So you would either have to start the ai with a reinforced path like Muddas run. Or have a godly pathfinding algorithm. Otherwise the AI would just start driving the wrong way.

[@ChuckSploder]

@@MrMeasaftw There's ways of doing it

[@Ivan.Wright]

​@@ChuckSploder My first thought was to structure the layers in a way to take location data. This way the network will respond differently depending on where it is in the map.

[@holyyoghurtman1952]

Shoutout to the one iteration of the AI that managed to fuck up so spectacularly it went over the railings at 2:33

[@ibrahimali3192]

agreed

[@MattlikestrainsOfficial]

Fr

[@1stTitanProductions]

Reminds me of the AI that was learning to play tag, taught its self how to clip through the map and stayed out of bounds so it couldn't be touched

[@kingkongdong7042]

I relate to this AI

[@astupidlylongnamethatstoolong]

@@1stTitanProductions Just like me fr fr as a kid. Designated spot to play tag? Nah, Imma hide out of the area.

[@1by110]

hooked till the end. Amazing work explanation and visuals man

[@cliftonsargent1572]

This was amazing, like watching a child start to walk then slowly just full sprint. I’m subbed from this video alone

[@L4Bomb4]

I hope this goes viral hard!

[@ecks0n]

labonba

[@Blue-vd1pb]

real

[@yoshtm]

Hey thanks Labomba! I've always been very inspired by the Trackmania k-projects of a few years ago, to make these videos. I've watched the one on your channel several times so I'm glad you saw this video ;)

[@L4Bomb4]

​@@yoshtmHope we finish the 100k this century haha

[@yoshtm]

@@L4Bomb4 Can't wait to see that!!

[@kevineina6454]

This project is still (I followed it for a long time) one of the most interesting projects of KZbin. It is indeed fascinating what a smart guy with a laptop can do at home… I’m an engineer and passionate about gaming, I couldn’t even think of how to accomplish such thing. Congrats to the author! 👏🏻

[@dman0odman267]

proof

[@JamaiiQ]

​@@dman0odman267?

[@toxicflicks1928]

code bullet is another great youtuber who has done very similar videos

[@frostyobilivion8365]

Just follow a cursus AI.

[@speed999-uj5kr]

It's easy.

[@HeyItsPM]

very impressive, both the project and the video. thank you for doing thjs and putting it out there

[@Mr.Bagscum]

One of the best videos ive seen on ai learning so far, great video showcases alot about the topic

[@user-sj9sw2yt4p]

this the channels that really need to be awarded for the amount of hard work put in

[@arcanum3882]

It’s called views

[@azultarmizi]

how the hell does this channel below 100k subs with this level of dedication, nice editing, and beautiful data display? This feels like 1M subs kinda level

[@Ethylmorph]

mainly due to its a game that has had a total of about 10million players over the last 20years , and not a major title like CSGO etc. . and about 36% of those have seen this video ! @@azultarmizi

[@tisisonlytemporary]

Three years of struggle

[@aptpupil]

AI edited it

[@halliwedge]

The fact that on the endurance, the AI clipped the back wheels to turn sharper is very cool.

[@anubis520]

Great video. love all the work put into this. Really cool to see how much it progressed.

[@gurur4406]

The editing and the video is very nice man thanks

[@Emile50]

I hope Wirtual does a followup. Would also be fun if the devs release an AI medal on each map

[@goldenwarrior1186]

Wirtual should talk about how huge the implications are in speedrunning. It could be used as a new cheating method that could potentially be harder to detect than TASes if it gets advanced enough (actually now that I think about it its tells are probs similar if not the same as TASes’ tells)

[@Falcodrin]

​@@goldenwarrior1186antagonist AIs can be trained that detect a certain probability of it being AI driven. Kinda similar to all the online apps that tell you if a photo is AI generated. I think those need the training model though to work right.

[@XCanG]

Since you reminded him, now I'm sure that for now AI couldn't beat speedrunners, because they used so many complex tricks, like a bug jump from a nose, etc. To make AI even learn that this is not a mistake, but a feature would be very hard.

[@codexed-i]

@@XCanG About this, if you want the AIs to do anything, you give them some reward, then, save the ones with a higher reward and scrap the others.

[@Falcodrin]

@@XCanG true but not many maps can make use of those "features" I don't think

[@zoltanfridrich1934]

Cant even imagine how painful and time consuming it must have been to create this amazing video. Crazy good stuff!

[@MoeZarella]

And imagine how long it must've taken to create the AI

[@fb1767]

Wow this must be one of the greatest videos I've ever watched. Thanks man!

[@jonifed3676]

This video was shown in the latest linus tech tips video about the LG Wing. Congrats dude

[@brigbrizz]

This was so much fun to watch! Thank you for putting it together, and posting it for us to enjoy!

[@cheezygonzalezthe13th]

100th like, have a great day bro

[@Olodus]

Really interesting approach to have the AI train with a "wrong" reward for a while to overcome a local maxima hard to find ways out of otherwise. That feels like it has some really good parallels to human learning, where a good teacher can help you immensely in how quick you learn something new. Or how athletes sometimes train using special limitations or disadvantages to improve their ability in specific cases in their sport - all to be better when they go into a normal competition. Awesome video.

[@Kyrazlan]

Same principle in life. You can learn from mistakes and failures and take away positives from them. A full technique may not work but partial techniques can be applied in certain situations. It's the foundation of bruce lee's martial arts. Learn as much as you can, use what works best. There's not really any downsides on iteration.

[@xXErr4rXx]

There are many cool approaches to fix local maxima problems, usually inspired by real life processes. For example one method is inspired by how a heated metal cools down and the excitation of the particles, the algorithm has a large chance of picking a random option instead of trained one and that chance decreases logarithmically as time goes on, to not get stuck at first but expecting to settle as time goes on

[@Arkansya]

it's precisely the définition of training

[@XMysticHerox]

Simulated Annealing yes but that is a very basic algorithm. @@xXErr4rXx

[@Jb-db5kj]

This is absolutely incredible I can’t even imagine the work and frustration that went j to this.

[@b.b.4192]

Very amazed by the quality of your work. Really enjoyed it.

[@yoinki_sploinki]

The sheer work and talent this must have taken is insane. Good job, and congratulations!

[@grahamdaniel9319]

Fr

[@hockypockies]

i'm glad you're back man this is genuinely so impressive

[@nerotoxen]

Kudos to your dedication. I hope you go far! ❤

[@Simply_Insane_]

Impressive. I'd be interested in the feeling you had when you finished this video and uploaded it. After so much time, tries and struggle. Congratulations. You did awesome!

[@wesb9546]

Your meticulous dedication pays so many dividends. Keep doing what you're doing, it's insanely entertaining and informative. I guarantee you're persuading millions to become programmers, AI experts, ect

[@Kram1032]

To make it more robust, it could be interesting to add a variety of extra conditions: - you are already spawning in random spots but you can also spawn in random states, i.e. in different orientations and speeds, possibly including upside down so it has to learn to turtle and recover (where that is possible) - in addition to the lateral motion reward, you can try somewhat randomized state rewards (reach weird parts of the state space) - or even exploration rewards (you can basically do a coarse histogram of all possible internal states and then reward it for even coverage of that histogram. Rather than as fast as possible, it should be driving in a way that finds as many states as possible while still finishing the map.) - or action constraints (disable breaking from time to time. Disable *forward* from time to time so it has to learn to deal with backward driving. Maybe occasionally even disable left or right. It's also possible to do "sticky actions" where you just randomly make it commit to an action for a few frames rather than being able to change the action every frame) - or senses (disable some of its inputs either by zeroing them out or by sending random noise through them) - or road conditions (you already mentioned those so I'm guessing your next video is gonna tackle that) - or physics (you also mentioned this as well) People have also experimented with a very weird robustness strategy where you basically add spurious inputs (they just get noise as input) but then *shuffle around which input corresponds to which value* so the AI has to learn to spot patterns in the inputs to figure out what those inputs likely mean before actually acting upon them. All of these together, or even just a solid subset, should make for a really robust and multitalented AI that can theoretically achieve just about anything that can be achieved in the game. Like, in terms of finding the state. Not necessarily yet in terms of beating world records. And then, once you have that, you just finetune that basic broadly capable AI without any of these constraints on any map you like. It'd basically be what you did with the drifting here but training it towards much broader capabilities as a starting point.

[@liamdescoteaux7024]

good idea

[@owendeheer5893]

Can you explain the spurious inputs strategy further please?

[@Kram1032]

​@@owendeheer5893 So I don't remember what particular network style they chose for this. I think it was a recurrent neural network? But anyways, the basic idea is pretty simple: In addition to all the inputs that already are there, you add a bunch more. (Say, three more neurons or whatever) Those extra inputs simply get fed random noise, so they aren't going to be meaningful to the training what so ever. But the twist is, that you then ever so often (after, say, a second) *randomly swap the order* of those inputs so the layer after doesn't know for sure which input is which. It has to figure that out based on the received signals. That way it learns not only to relate input patterns to output actions, but necessarily also what typical input patterns look like. It has to work way harder and "pay way more attention" if you will, to still get a meaningful result. IIRC the idea was, that you can use this to make it possible to extend the network after training. Like, instead of noisy inputs, you can then add additional actually meaningful ones, and there is a chance the network manages to generalize over those additional inputs. To be clear, I don't actually think that particular augmentation would be useful here. Most likely, it *could* be, but it would require a larger network just to allow for the overhead of internally swapping around the data to be routed correctly. IMO the most powerful ones I mentioned are likely to be the road conditions and physics tweaks alongside the histogram over state space (that's somewhat related to Map Elites, although that's an evolutionary algorithm so not quite the same. I think there is a variation of that which is meant to work in this setting though. Differential Map Elites or something?) Simply increasing diversity (i.e. training over multiple maps at the same time) is also likely to give rise to gains, especially paired with a curriculum so easier maps are experienced earlier and more challenging ones later. (Very challenging maps initially will only cause a lot of noise slowing progress even on easier maps, until the AI acquires some basic skills. Definitely don't train it on Kacky maps right away lol)

[@someonespotatohmm9513]

​@@Kram1032 "But the twist is, that you then ever so often (after, say, a second) randomly swap the order of those inputs so the layer after doesn't know for sure which input is which. It has to figure that out based on the received signals.": This sounds like an attempt to force location invariance (it doesn't matter where the signal comes from just the relative strength between the signals as a whole). Which seems only usefull in cases where you want the network to learn statistical things, or you want to force the network to encode information in certain "interesting" ways.

[@Kram1032]

@@someonespotatohmm9513 yeah, as said, I don't think it'd be particularly useful here. I just thought it's a fascinating idea.

[@Pintkonan]

i cannot believe it! you have done it. (: congratulations. the path is the goal, as they say. the fact that the ai was able to grasp the concept of drifting blows my mind ^^

[@jochenpeters7486]

Great Video. And you must have spent so much time to make the video and the visuals, all the cars overlaying each other. I can just imagine. Thanks for sharing the video and best, for next videos to you

[@imjody]

Absolutely incredible story telling, topic and editing, Yosh!!! 🔥

[@IsaiahMutex]

Yoshhhh

[@IsaiahMutex]

Yoshhhhh

[@apocryphgaming9995]

The AI journey feels like a lesson in consistency. It was kicking your ass even without the brakes, and that was purely through being so thoroughly consistent in the corners. The final version with the neo-drifting was even drifting much cleaner than you were, too. It was a joy to watch.

[@UnrivaledPiercer]

This is fascinating... I loved how you taught the AI to drift and then once it learned how it kept doing it without the rewards applied to the action. It shows that it was still focused on the end goal but using skills it was taught when they would help it achieve a faster time. Really amazing to watch.

[@shiftery2k]

Seeing this project come to fruition after three years is incredible to me. I watched from the very beginning and thought it was amazing how you could teach an AI to be the human, and now that we actually get to see it put into play is amazing. Love your work so much, and I guarantee that it will only improve itself from here! I have tried to beat the AI myself and it really has progressed so much. I cant wait to see pro players actually holding a challenge to this AI after such a long dream. ❤

[@Arcalynt]

This editing has addiction potential and Your work might change trackmania forever. Round. Of. Applause!

[@pr0hobo]

I cant wait to see generalization. I hope we can see a full ai playthrough of tmnf at some point if this keeps going.

[@Pandize]

Yes. Beyond million view title.

[@whirled_peas]

Just hope it doesn’t generalise too much or it will push us humans off the planet

[@kazuala]

There's a streamer on twitch that already does this with the Name: PedroAITM

[@timbraska6750]

I don't know how well this approach works in terms of generalization. I have serious doubts on it working on any maps that are designed different from the twisty horizontal characteristics of his current test maps. And if he manages to get them working on different maps it will become much slower on the current ones.

[@BigDBrian]

@@timbraska6750 I think for the AI to drive well on a variety of maps there needs to be some solid foundation to start off with, then some training on individual maps. This is also fair to human play since we play maps over and over to learn them specifically, while having a lot of experience in general to back that up.

[@crazymooseman9087]

One of the best ANN and RL descriptions I have ever heard 👏

[@Giitzerland]

That was a hell of an effort, man! GJ!

[@MazyHadz]

it's so fascinating to see people with such skill and passion do something like that

[@gordonsaintclair641]

Thank you for that beautiful sentence.

[@stevenwolfe591]

Color shading the cars and the training progress bar is a really nice edit! It's not always exciting to see an AI drive badly, but watching the yellow to green is pretty great anticipation.

[@goopi]

This is really impressive! Looking forward to the next one :)

[@T0myTune]

this is awesome, incredible work!!! Hats off to you!

[@KJMTurbo]

Seeing the AI plan it’s most efficient routes was like watching a fluid dynamics simulation, in fact it probably literally is a more accurate representation of such than a lot of CGI engines can perform or “animate”.

[@knownas2017]

9:44 This editing was sick. 10/10

[@grayforsyth3728]

The graph scene is CRAZY. This whole video is a masterpiece, kudos!

[@rasperss_5176]

I remember watching your first AI video. I absolutely love this follow up. Amazing work!

[@primescoreredstonetutorials]

The editing is amazing and I LOVE the animation renders to show the ai's many attempts

[@fffaaa8371]

definitely subscribing. Ive seen a few of your other videos but i figured training the ai was going to be a one off thing but the fact you plan on teaching it more (hopefully a lot more) is super exciting. Ill be counting the days :)

[@pablomolinaponcedeleon8339]

This is one of the most interesting videos on Ai learning I've watched congrats!

[@chadmetcalf2876]

Amazing work man I cant wait to see the next installment

[@SuperNexus14]

At the end of the AI training, it always seems like you just did a few logical changes and then it worked. Coming up with these logical changes is the really hard part. Great work!

[@snared_]

I was hoping there would be a github link so I could try it on my own maps. What would you think about releasing it under a MIT license? I wouldn't mind installing some dependencies and running with a particular OS if necessary, as I can emulate all of them.

[@Sleepyhead080808]

Seconded!

[@cscscscss]

Thirded!

[@dregos]

Fourthed!

[@Lucca-An-Aspiring-Poet]

Fithed!

[@Verosment]

Sixthed!

[@arthrcrown]

Impressive work, I didn't know the project and I'm just in love whith the whole thing. This will be spoken, and referenciated in the future, you my friend is the begining of a new era, that will not only be studied but appreciated. Congrats bro, stunning piece of work.

[@johndoe4314]

Just wow. Your level of tenacity is just wow. Huge congrats. What a unbelieavable achievement this must be for you personally. Stickig so long to a project, never giving up and creating documentary videos with so much passion. Unbelieavable. Hopefully you enjoy your success. This series is a masterpiece.

[@Hacks-rx2jc]

This is the spirit of coding, and CS in general. Trying new things that you aren't certain about. Learning by doing. Thank you for putting in over 3 years to make this masterpiece. BTW can we -steal- *borrow* the training data pls?

[@sahajramachandran348]

You could say he did a bit of reinforcement learning himself

[@egemen_ozturk]

Good catch!@@sahajramachandran348

[@theF_Zone]

i have been waiting for this video for a year now and i must say im more then surprised by how good it is! Incredible editing, voice over and content!!

[@ToloRon]

This is really brilliant and bright way of understanding this driving AIs, congrats on your game!

[@TheBrendanzn]

Awesome video - it does make you think of the game makers themselves who've been ahead of the curve with AI for ages now, but have not been using it for optimal outcomes(i.e. being unbeatable) but rather to fit in with the game objectives and accomodate the player experience.

[@jcorey333]

I like the color coding! Also, I think your idea to give it a specific reward for a certain skill, then taking that away once it mastered the skill, is really genius! Did you get that from a paper? It definitely belongs in one. Kudos to you!

[@MorzakEV]

Yeah, I guess that’s like when humans break learning down in to specific skills for a period of time before putting it all together again later.

[@TheShazMC22]

It’s like positive reinforcement when teaching children or a dog. You reward it for doing the right thing until it does it naturally. Pretty cool

[@Nicodax84]

Really insightful video ! What a comeback from the AI 😮

[@yashsingh6508]

The visuals and editing is super dramatic! Love it

[@EastndFTball]

This was super awesome, man. Thanks for sharing

[@wonkywonky6307]

This is fantastic! The editing, effort, and time put into this are amazing. Congratulations.

[@anesthetized7053]

this is one of my favorite deep cut series on youtube. really impressive work

[@bencalleja471]

this is HUGE dude. excellent work!!

[@MrKyle700]

I loved how you show exactly what a real, practical use of machine learning ai looks like and the pitfalls associated. the journey was very interesting. great video

[@KidNoraa]

16:26 that’s literally addiction

[@samwilde8311]

I'd be very curious to see a research paper comparing human addiction and reward systems to neural networks

[@KidNoraa]

@@samwilde8311 I mean addiction is literally the abuse of the body's reward system.

[@necrochaufagist]

Excellent travail frérot ! C'est une très chouette vidéo et ça a du être beaucoup de travail, je suis content que tu aies fait autant de vue, bravo !

[@Xamarin491]

Awesome work! I'm excited to see the ai try to race on a traditional Trackmania race track with smooth curves, jumps and elevation changes.

[@astrovation3281]

The drifting in particular was very smart, and not something I could've quickly thought of. Please make more, you're a great informative story teller.

[@blackhole927]

As a non-trackmania player who’s been keeping an eye on this series over the past few years, it’s amazing to see the AI drive so well! Amazing work!

[@rhodes_the_kitty]

same yeah trackmania seems cool but I've never played it much. I was super happy to see the ai was able to beat them after all these years tho!

[@nezzysan]

Dude amazing video! This is the kind of videos that belong on the platform!

[@KurtBusiness]

I’d love to see this AI take on real life tracks. I wonder what its time would be at Spa or something. Very nice work!

[@user-qu8iq1xb1k]

Amazing description of how complicated, difficult and challenging is to be an AI engineer. Appreciate that you shared all ups and downs (especially) of such journey. Complicated things take years to be made, but in the end it pays off.

[@faltice1314]

Je ne joue pas à Trackmania, mais le travail que tu as présenté est monstrueux. Félicitations

[@fridaycaliforniaa236]

Idem =)

[@cewla3348]

as a french beginner, i understood this (except for monstreux and félicitations) so im proud now

[@vodcrach8479]

Un jolie accent français en plus de ça mdr

[@racaille_de_Shangai]

félicitation = congratulations et monstrueux is like huge@@cewla3348

[@canard542]

@@cewla3348félicitations=congratulations monstrueux=monstruous but it means incredible

[@anjtapioca1750]

I don't know how I stumbled upon this video but I got so invested and was second hand proud watching this AI grow

[@bredgefox1599]

Félicitations pour tout ce travail accompli. Je suis persuadé que si tu continue en cette voie tu finira par obtenir une IA performante sur n’importe quel type de map. Qui peut-être même que ça finirais par attirer l’attention de Nadeo ;)

[@Ciboullete]

Un Français sur une vidéos américaine ? Étonnant !

[@Ciboullete]

Qui est Nadeo ?

[@lzjzj3jke90]

c'est parce que yosh est français

[@Ciboullete]

Ah je savais pas@@lzjzj3jke90

[@adbroussillon]

@@Ciboullete vu l'accent c'est un gars de chez nous

[@ghostinng274]

Seeing every single attempt made by the AI is so satisfying. And its all color coded too. I love it. Its also really cool watching it slowly understand exactly what its doing.

[@observe_and_purport]

A pleasure to watch, thank you! EDIT: 17:30+ was just *chef's kiss* driving

[@ronan_ptr]

Bravo ! Pour le travail que tu as réalisé et aussi pour la manière dont tu l'as présenté

[@deanvangreunen6457]

Amazing dude, much respect, well done bro

[@sametragichero]

Congrats on the video and this project - amazing work!