AI plays the new Trackmania game (2020)

[@dmarsub]

You could try driving a map with only having the input the a.i has (only seeing the lines) and see if you can beat it :D.

[@jf500game6]

Trop bien tout ça. Continue surtout ne t'arrête pas c'est trop fun à regarder et à essayer de comprendre comment ça marche

[@jeromelageyre5287]

Toujours aussi génial ! Les explications sont claires, le son bien équilibré, on voit qu'il y a beaucoup de travail derrière :) Hâte de voir le renforcement learning et les shortcuts !!

[@Kreypossukr]

Vraiment hâte de voir ce que va donner l'IA dans trackmania dans le futur, continues comme ça ! En plus on sent que ton accent s'améliore !

[@nicvermeeren3011]

Great job!! I am curious as to what the AI will improve on and be able to do next time. Keep it up!

[@beware2962]

Great work man. Been waiting on an update. Keep at it!

[@BFHThePunisher3000]

Cool project! For me it would be interesting to see the AI perform on very simple tracks, e.g. one turn. I wonder if it then beats most humans, or even maxes the track.

[@yoshtm]

Thanks ! I want to try that with reinforcement learning :)

[@joshbearden4956]

Your accent is nothing to apologize for! You speak English very well. You are sharing incredible knowledge with the world! Thank you!

[@joshbearden4956]

BTW I landed here because this video is now something my AI class is discussing.

[@gugus8081]

Stylé ! Hâte de voir la suite, continues comme ça !

[@nevo1878]

Nickel j'attendais la vidéo et je suis pas déçu ! Continues ton super boulot :D

[@neyth1942]

Vraiment propre pour une petite chaîne, j'espère que tu va continuer sur la lancée !

[@japawa09]

I love this content. Very well done, thank you!

[@whynot48151]

Passionnant ! Merci de nous partager ce travail ! Et c'est une super idée d'avoir fait une IA simple avec ''que'' 16 inputs

[@dimimen]

This is more of a Breitberg machine than an AI really. It's essentially an output directly controlled by the distance it is from a wall.

[@bklydesign]

Cool! Keep up doing these experiments on track or even other games

[@martelyggdrasil3655]

It would be funny to see the AI on some of the easier tracks that have jumps or multiple surfaces. Ice would completely ruin it but dirt and grass behave fairly close to concrete at the relatively low speeds the AI goes at.

[@klitlikaswtfzone3220]

keen on the follow up video :) with AI Drifting

[@flaviensoirot28]

C'est du bon boulot, j'y connais rien et c'est super intéressant. J'ai hâte de voir la suite et comment tu vas l'améliorer

[@Shukaname2]

are you french? your accent let me think you are x) SI tu l'es GG a toi content de voir un génie français comme toi^^ ça à l'air si complexe x)

[@flageur6106]

Ouep c'est un français

[@paulbaspeyre8119]

Je me disais bien il avait un petit accent

[@yannisrenoux]

Il est francais l'accent plus les sous-titres en anglais et français....il est francais

[@urano1988]

Nice job! It looks like a quite significant amount of inaccuracy comes just the frame processing algorithm, like the crash at 1:21. Also I wouldn't start all the rays from the center of the bottom of the frame: the rays should start from the center of the car and you need to take into account the FOV of the camera, in simple words: spread their origins a bit along the bottom of the frame.

[@Speedself]

Je trouve ça tellement cool comme projet ! Bravo 👌

[@eliumlabessirou3063]

Ultra intéressant! 2 idées passent dans ma tête: 1.Avec une méthode de deep learning je suis trop curieux de voir ce que ça donnerait! Et si c'est envisageable.. 2.En considérant les performances de la voiture et la forme du circuit, serait-il possible de calculer "la trajectoire "parfaite" puis de la faire suivre par l'IA. (Et pourquoi pas mélanger 1 et 2 tant qu'on y est vu que la perfection ne sera pas forcément simple atteindre) En revanche avant que l'IA trouve le cut d'AUREL y a du boulot aha TMCUP2020💪

[@Atomguy112]

I look forward to seeing advanced AI stuff

[@cKaOsTiK]

J'adorerai voir un IA comme AlphaZero pour les échecs jouer la map de mieux en mieux pour finir par atteindre un niveau surhumain à la fin ^^

[@zarcointheduck1853]

Yop ! Je parle très bien anglais, je suis un dev et je connais un peu TrackMania, donc si t'as besoin d'un doubleur n'hésite pas :D Sinon continue tes IA c'est insane ce que tu fais

[@zarcointheduck1853]

D'ailleurs ce serait ouf que tu puisses récupérer des runs de professionnel pour les données de l'IA, ça la changerait tellement ! Je pense qu'il y a moyen de demander à des joueurs, y'en a surement qui accepteront

[@yoshtm]

Merci :) le probleme c'est que si j'entraine mon ia avec les donnees de runs de "pros", le model va overfit les trajectoires "parfaites", et des que l'ia va devier un peu de ces trajectoires, elle va se retrouver face a un type de donnee qu'elle n'a jamais rencontré pendant l'entrainement. J'ai meme fais expres de conduire un peu moins bien quand je collectais des donnees d'entrainement, pour avoir une plus grande diversité de donnees, et ca marchait mieux que si j'essayais de toujours suivre les memes traj optimales

[@yoshtm]

Tu as un discord pour te contacter ? Ou mail ?

[@zarcointheduck1853]

@@yoshtm Salut, désolé de répondre si tard, je suis actuellement en vacances. Du coup voilà mon discord: ZarcoinTheDuck#2152 Je comprends mieux le problème en effet, après la récolte des données des pros serait dans l'objectif d'une IA parfaite, mais comme tu l'as dis dans tes vidéos pour cela il faudrait sans doute plus de connaissance que seulement les murs proches, et en plus faudrait faire ensuite de la sélection et des mutations ! Mais c'est possible à faire !

[@GerbenMeijer]

Okay this is very cool. I've kind of had the same project on my mind for a while, but have not really looked into it yet. I was wondering how you acquire the gamestate in real time (ie. what the agent sees). My problem with using real games and reinforcement learning is always the fact that I don't really have enough control over the game, leading to buggy code and bad performance. Could you possibly share your methodology?

[@beastzz7824]

To get real time data you can use dll injection. But it is a very time consuming method. If you want you can contact me on Discord and I will explain it to you in more detail. Discord: BeastzZ#5343

[@Danaby2]

Very nice video

[@leonnilein]

Very interesting project. I have tried something similar, although I fed the last to frames to a CNN and used supervised learning with some of my races as training data. Didn't turn out well unfortunately (but I also didn't put a lot of time in it and had very little training data). I wonder if some form of a two-stage NN could help here. The first stage would be a CNN with a kind of lane-detection, which would detect not only the walls in certain spots, like your point algorithm does, but instead continous lines that follow the track boundaries. It could also detect the type of surface (road, dirt, grass, etc.), which should enable the AI to drive on different surfaces. All this information (and maybe even more like the current speed etc.) could then be fed into a second stage NN (maybe even some form of RNN to enable memory?). This second stage actually predicts, which keys should be pressed. This could also make reinforcement learning easier. The first stage probably has to be trained using supervised learning with labeled images. But if that stage is working well, the second stage could be trained using reinforcement learning, which should make the model converge a lot faster. Just some ideas and a lot of gut feeling though :D but maybe you could use some of the ideas. Still, your project is already very impressive!

[@rafafafayo]

Hello, Super boulot, fan d'AI et de TM aussi :p Je pense que tes inputs sont bien pour une premiere approche, tu pourrais faire du CNN etc. mais bon un peu plus complique a setup j'magine. As tu imagine utiliser une autre cam que la cam3 ? La 1 vue de dessus te donnera peut etre plus de hauteur pour apprehender un peu plus en avance les virages a venir. A prendre ou a laisser :P Je me demandais egalement, tu n'as aucun moyen de savoir a quelle distance "reelle est la bordure n'est ce pas (car ce n'est pas vu de dessus), as tu une idee pour contrer ce soucis ? super cool en tout cas, super plaisant, GG

[@Bapttheo]

Very nice ! I wanted to do reinforcement learning but what's very frustrating is that beacause we can't simulate the runs in tm (you have to do the training in real time), In order to have a decent amout of data to see interesting behaviors like drifting and cuts you have let the ai learn for a very very long time. I don't think with like days or even months of training it will be able to do that kind of things...

[@musumia2232]

Isn't it possible to run multiple cars at once? Like each generation drives 10 cars simultaneously. It could provide more data in less time.

[@Bapttheo]

@@musumia2232 Maybe running multiples games at once but with like 10 windows, to compute with enought fps it will be hard i think. And I think 10 times faster is still very slow.

[@musumia2232]

@@Bapttheo true

[@yoshtm]

Thanks ! Yeah I agree with you, I theory it could work very well, but in practice it takes a lot of time

[@AMIOS]

I think that a good way to do this is to run the game on a lot of computer at the same time and finding a way to centralise data

[@nassimouyed7658]

Salut ton projet est magnifique je suis pressé de voir comment l'IA sera capable de jouer bonne continuation

[@benjaminguillerm42]

Hey ! Très cool comme projet ! J'imagine que si tu implémentes à ton programme du TD learning et que tes rewards prennent en compte le temps mis pour effectuer la course tu pourrais avoir de meilleur résultats ? En tous cas c'est du super taffe, je serais curieux de voir ton programme :)

[@thibault7135]

Hello ! Very interesting project, i love it ! Having develloped simpler IA it is always great seeing when people pushes it a lot farther ! The car seems like it has issues when there is a slope, what about adding a detection to see if the car is not on flat grounds ? Also, using 3rd person view limits the view of the car. I know it will be harder, but what about using a different camera, starting the lines from the front of the car to have the distance ? You could also have the angle of the car and it would help with knowing if it is flat or not. Keep up the great work !

[@MasterScrat]

Did you actually try learning from the downsampled black and white frames? using convolutional neural network you may actually get better results! or, you could use both the wall detection output AND the downsampled images at the same time to train the network.

[@macarrony00]

Hello, love the video! I am trying to do something similar, did you use python? How did you connect to OpenPlanet API? I cannot found any documentation about any API, just the code to make the scripts for the game.

[@officialcorevision]

That would be sick: AI finds every shortcut on map ^^

[@stephane_branly]

Vraiment intéressante cette vidéo d'IA avec trackmania ! Je vois que à un moment, tu as accès, via API à certaines données comme la vitesse , nombre de roues touchant le sol et qui sont en drifts, les utilises-tu en Input ?

[@AMIOS]

Hello ! Amazing project ! Is it possible to have more information on how do you get distances with walls ?

[@AMIOS]

Ps : si jamais tu acceptes de me répondre, je suis français si c’est plus simple qu’en anglais !

[@yoshtm]

@@AMIOS Salut, merci :) Je fait ca sur python. Mon script prend une capture d'écran, puis il compte le nombre de pixels entre le bas de l'image (au centre) et le premier pixel plus ou moins noir le long d'une direction donnée. Je fais ca pour 15 directions différentes. Aussi, je déforme un peu l'image en amont, pour "mieux voir au loin"

[@AMIOS]

yoshtm ok merci ! Vu la taille d’une capture d’écran ça me parait super lourd de chercher pixel par pixel sur 15 directions mais bon apparement ça marche ! As tu une grosse config d’ordi ? Et pour ce qui est de la vitesse

[@yoshtm]

@@AMIOS Je diminue un peu la taille de l'image, mais en effet c'est assez long ! Je fais ca sur un pc portable avec une assez bonne config. Je reccupere la vitesse via Openplanet, qui me permet de communiquer directement avec l'API du jeu

[@AMIOS]

yoshtm ok merci beaucoup !

[@Sayahebi]

Did you use openCV for your program to capture distances?

[@flageur6106]

Continue comme ça

[@johnnychabin6982]

I’m two years late, but would the AI benefit from also being given the difference in each distance from the previous frame? (a delta-distance value), making it easier to tell when approaching a wall? Could work to help resolve no memory by giving it better time-based data.

[@buxxypooh8381]

I wonder how supervised learning would perform on trackmania But getting enough data would be hard except if you can feed the AI other people's replays And input would also need modification, distances from the walls aren't enough, more like direct feed of game display

[@saveyourlifes7867]

yea but... it IS supervised learning, the algorithms tries to copy @yoshtm's driving

[@buxxypooh8381]

​@@saveyourlifes7867 Wouldn't it be able to learn from top X records on a track? Then from that you do some mutation and maybe the AI can find improvements

[@JonathanFontaine]

@@buxxypooh8381 It probably could.. you could train an ai with a lot of different humans runs.. for one particular map the ai would probably beat humans quickly.. on different maps probably less efficient.. then again you coudl train it with massive amount of data for lots and lots of maps..just giving indeed full screenshot of the game + human inputs. An Ai trained like this could probably give pretty good results.. BUT all of this is wouls basically glorified statistic.. and you wouldnt know what the software is actually doing where here.. He is trying to actualy write algorithm himself.. think about how to interpret information and react .. way more interesting and challenging !

[@oschell914]

Nice job ! Is there a repo of your code so I could play around with it ?

[@Cyr1lbibi]

Good job. I had an idea, would it be possible to improve the AI by giving some informations? Like for example, easiest case: straight road: the fastest way would be to go straight ok, so it's basically the shortest distance between the beginning and the end, you want to avoid big turn which decreases a lot of the speed. I don't know if you follow me.

[@someonespotatohmm9513]

have you tried using other activation functions. I don't know which you use but elu has always performed the best for my limited amount of projects. edit: I don't know if this is possible but because trackmania works with standard track pieces you could use the car state and the momentarily interesting track pieces as input. That should provide all the relevant info as each track piece and their order contains all the info of the track. The only problem is that now your network will need to "decode" from track pieces to useful info and I doubt it will properly do that.

[@mafiawerbung]

pretty cool

[@Anwesh77]

Great work! Is it possible for you to share the github link?

[@marekpetrovic1181]

Great work! I had this in my mind for a while, but never really tried it... I wonder how would recursive neural network play (because of the short term memory)

[@yoshtm]

Thanks ! I tried to use a recursive neural network, but it doesn't change much

[@jairjuliocc]

Fabolous! , maybe a genetic algorithm, not all your cars seem bad in the curve road, maybe only working with the best performing could improve the model

[@yoshtm]

Thanks ! All cars are controlled by the same model, so there is no interest to select the "best ones"

[@JonathanFontaine]

I think the main problem for you IA is indeed the memory.. not having more information from the frame.. and what would be the most imortant would be to know at least the sharpness (angle) of the current turn aswell as the direction and sharpness of the next one.. Try to think about how you as a human pay not so much attention about the details on screen.. and to get good score on TM have to replay and memorize the turn sequence to optimize his path.

[@JonathanFontaine]

to make this memory you could just save all the outputs the ai does on his first run.. wich would be a simple blueprint of the map acutal successive turns.. the next runs the could look into this array and look how long he did steer giving information on how tight the turn is ..and then see if it turn in the other direction after it or just stay straight.. if it did turn it could then try to anticipate this change of direction earlier than what it did the first time. another good part would be to attach a "time stamp" to each identified turn.. and store it into another array called "best time" .. so the ai could compare if he is faster and choose the best solution the next time.. Anyway cool project, looking forward for your next iterations ;)

[@atomkillerssklave7950]

Nice!

[@xenyatox9814]

Tu utilises quoi pour implémenter ton code avec trackmania

[@MrIkariam2]

Bon, Nadéo, c'est quand que vous engagez ce gars là ?

[@DWThommie]

Looks promising, could you maybe explain some more on how you're accessing the Openplanet data?

[@yoshtm]

There are some tutorials on the Openplanet website : openplanet.nl/

[@remink3092]

Very interesting video ! Do you think the AI could drive faster than humans with some improvements ?

[@yoshtm]

Thanks :) I don't think so

[@kArTerhns]

Jolie travail, s'il y a besoin d'aide pour les replays ou la programmation, je veux bien m'y essayer !

[@AkashiKNB]

bientot carlJR dans le rétro

[@saveyourlifes7867]

sympa comme vidéo ! je venais juste de me dire que ce serait intéressant de faire une IA sur trackmania du RL plus précisément ), apparemment l'algo youtube a lu dans mes pensées... Du coup, je me demandais comment comment tu faisait le lien entre le jeu et le code, il existe un pipeline pour ca ?

[@yoshtm]

Merci ! J'utilise Python pour la capture d'écran, la détection des murs, le controle du jeu, etc. Et j'utilise aussi Openplanet pour communiquer avec l'API du jeu (notamment pour récupérer la vitesse)

[@ldlework]

Where's the code?!

[@tructruc00]

Comment tu as crée cet ia ?

[@ivanzivkovic7572]

Are you the author of MapEdit(TGYoshi)? If so, is the source code for it still available somewhere? The github link in MP forums is broken

[@yoshtm]

It's not me

[@MILMYtm]

sick

[@cyrillaville2367]

essaye peut etre un cnn - lstm ca rajouterait de la memoire et c'est plutot ideal pour de la vision (je lui fournirait l'image couleur part contre en noir et blanc l'IA risque d'etre confuse)

[@yoshtm]

J'ai essayé d'ajouter des LSTM avec les variables de distances, mais ca n'améliore pas le résultat. Mais c'est la première fois que j'utilisais ça donc je suis peut-être passé à coté de quelque chose.

[@cyrillaville2367]

@@yoshtm je pense qu'il faut eliminer les variables de distances et envoyer les frames en couleur (avec une resolution un peu reduite quand meme) dans un cnn qui sera suivit d'un lstm apres je ne sait pas comment ca marche quand les IA essayent de copier un humain perso je les laisserait se demerder en renforcement XD

[@julienr1672]

Ajoute pas d'inputs stp, ton IA devient meilleure que moi

[@pixlgamerfr2004]

Where can I buy this game ?

[@yoshtm]

www.trackmania.com/

[@pixlgamerfr2004]

@@yoshtm thanks Is this downloadable for free ?

[@yoshtm]

@@pixlgamerfr2004 There is a free version if you want

[@pixlgamerfr2004]

@@yoshtm yes thanks again

[@Logan_Abad]

You french ?

[@yanndegrange978]

Il y a des sous-titre je te rapel

[@yanndegrange978]

Et tes vidéos sont cool. ;)

[@GeneralKenobi69420]

Oui oui croissant

[@KRISONTHETABLET]

Next time,70 fps and 1440×720 pixels =1,036,800 pixels or ~1000000 pixels

[@rickythegreaetist]

Make the best ai possible!

[@yanndegrange978]

Et cool y joue depuis quand '' papa''?

[@trochic]

How se prononce Haw, pas Ow, Ow c'est moche

[@zozai7432]

ton accent trop sexy

[@dubiousinsights4008]

Yosh, this is a great video. I'm working on some AI playing games on my channel as well, I wanted to contact you to see if you might be open to a collaborative project in the future. I don't have any particular project in mind at the moment. I'd send you an email, but I don't see it on your channel. If you want, please go to the About section of my channel to find my email. Thanks! D.B.

[@UberBug1]

Your father also needs some training 🙂