Teach AI To Play Snake - Reinforcement Learning Tutorial With PyTorch And Pygame (Part 1)

[@shadabanwar2101]

This is the first time I am seeing something practical to learn Reinforcement learning. Thanks a lot for your effort! Love from Bangladesh.

[@patloeber]

glad you like it! yes not many tutorials about this out there

[@shivamroy1775]

Amazing, quality content ! Can’t appreciate this tutorial enough.

[@wacky_inferno1023]

This series is so awesome... I'm really new to ML ...As I was coursing through the internet for info on ML and DL ... I found this awesome channel... I really appreciate your efforts... It's very informative and I'm learning a lot... Please keep posting more vids... I'm totally a fan of your teaching style now...🤩

[@patloeber]

Happy to hear that :)

[@szymonkrawiec320]

There are many Deep Reinforcement Learning courses out there, but noone of them are as good as your, and many of them are not sufficient enough to learn anything. Simple, basic, very well explained from scratch to finish, everything works perfect... Thank you very much for this!!!!!!! it really helped me a lot!!!!!

[@patloeber]

So glad to hear this, thank you!

[@kahowong6619]

I’m just 13 and i want to understand the ai for programming, this video is very very clear, thank you!

[@embeyond]

absolutely awesome tutorial, finally something practical. Thanks!

[@leonlysak4927]

Thanks for the tutorial, I've been looking everywhere for a Deep Q Learning tutorial that doesn't use Open AI's Gym- I'm following along in a completely different language and you made this very simple to follow

[@patloeber]

Great to hear!

[@VR_Wizard]

11:39 He ment direction right set to one not dangour right, meaning the snake moves to the right,

[@philippmuller2086]

holy shit, this thing is alive!

[@yash-w5k5j]

Absolutely amazing!! Please do more videos like this

[@patloeber]

Thanks :) Yes more to come...

[@НиколайТодоров-и9т]

Just finished doing a simple vanilla JS snake for training 🙂 Will be glad to study your demo

[@casper0165ful]

Your channel looks nice cant wait to watch all your videos!

[@patloeber]

Hi, glad you like it!

[@saurrav3801]

🤙🤙🤙 nothing to say ....bro...you are always something special....nice video keep going... full support 🔥

[@patloeber]

thanks a lot!

[@ansarbashashaik4345]

amazing tutorial, you simplified everything

[@mortendmortensen]

With the states you provide there is no way for the agent to tell if it is going into a dead end. Which you can see it do a lot of times. Would it not be better to give it a danger value that tells it how far the danger is in every direction? it would make for better scores, I'm sure.

[@ciscoserrano]

Another great job. Keep up the great work.

[@patloeber]

thanks!

[@tunaroll3957]

Awesome mini series, thank you so much!

[@patloeber]

glad you like it!

[@bedeamadi9317]

Nice!I am looking forward to the follow up videos on this👍

[@patloeber]

Hope you enjoy all 4 parts

[@samsikder4901]

I am Lucky found your tutorial, I have also watched your tutorials on pytorch. Hope you may suggest me a serial on your playlists to learn things like this. Great tutorial.

[@vito6137]

Hey , wie ich sehe bist du deutsch. Gutes Video, kurze frage am Rande. werden die Trainingsdaten irgendwo zwischengespeichert oder muss die ki immer wieder von neu lerne?

[@rupertbauernfeind]

have you ever tried to use the full grid of the map as input? It would be interesting if the model could learn to play perfect then.

[@matthiasho3216]

Great job Patrick for this video. I have a question: Can the snake still be trained, if the food is also moving? How different things would be, if the food is able to move up down left right at a point in time. (Not withstanding the fact that the food should be within the boundaries and can blocked by the body of the snake). Thank you!

[@sushantkumar4917]

I love u for this❤️🔥

[@patloeber]

haha thanks :)

[@FilipeRebollo]

Would it be possible to set the state as the whole game matriz? Containing the snake food and so on?

[@guilhemsaint-gaudin5795]

Thanks man, I'll do an oral presentation on that subject next week so you are really helping me out! Just to make sure, the Q value is the prediction of how good a move will be?

[@connor-shorten]

Super cool!

[@patloeber]

thanks!

[@codebreakthrough]

Ugh how am I supposed to compete with this 😩 🐍 (Edit) nice vid btw

[@patloeber]

Haha you'd better watch out ;)

[@dlc7596]

I love PyTorch.

[@patloeber]

me too :)

[@craigrichards5472]

What a cool fun idea. Was hoping I would come across something like this. Can you do this for Doom next. 🎉

[@RisenYah]

Love it man thank you.

[@patloeber]

Glad to help

[@kaiye4954]

Hi Patrick, thanks for this awesome tutorial. I have watched your Tensorflow series. That was great. I haven't watched your PyTorch series yet. I guess I need to watch that series first before taking AI snake series, right? Also, I guessTtensorflow is used for machine learning. PyTorch is used for deep learning. Is that right?

[@patloeber]

It's not needed but the beginner series helps a lot to understand some of the concepts I apply. Both TF and PyTorch are mostly used for deep learning

[@vladimirgetselevich4704]

Great work! But one question in introducing State this way do not you present some human level heuristics about understanding of this game? Wouldn't it be cleaner the state just contain pixel of the game? Or in this way, you wouldn't be able to train it so fast?

[@reinerheiner1148]

Why did you choose 11 state values instead of providing the ai with the complete playfield? A human player would also be able to see the whole area. Great tutorial btw.

[@venugopalbv2069]

Any reference book you could suggest for this topic?

[@RedPillDS]

How about you train multiple snakes, let's say n = 100, and then use NSGA ? (Non-Dominated Sorting Genetic Algorithm)

[@greatsaid5271]

thank you please make more pytorch tutorials 🙏

[@patloeber]

I will :)

[@danazoumi6346]

nice, keep teaching this please

[@patloeber]

Thank you, I will

[@ArunKumar-sg6jf]

how to use reonforcement for diffrent application like advertisment, heathcare

[@iritesh]

Can you please do the same video with multi agent?

[@Peanut-jf7mr]

Hi great tutorial, I just did not understand the model part. What is the hidden layer and I didn't really get the example quite well. Are those 5.0, 2.7 and 0.1 random numbers and why is the max 1, 0, 0. Sorry I'm new to all this. Hope u will help me understand. Also sorry for bad English.

[@brucelewis6366]

So is so learning or is it trying to see how much is allowing you to learn

[@abhinavraghunandan5343]

Is this a model based, value based or policy based model ?

[@sidyniang_coding4101]

We can also use vscode and pygame ?!

[@debarchanbasu1962]

HI! Didn't understand the part where you explained why we are taking 3 states instead of 4. 180 deg. turn would just reverse your snake, right? It shouldn't be a problem. Could you please explain that part? 😅

[@jacobolson5582]

a 180 degree turn would turn the snake into itself, killing it

[@debarchanbasu1962]

@@jacobolson5582 it won't. E.g. if the snake is going up, a 180 deg. turn would make the snake go down. So yeah, it won't kill the snake.

[@patloeber]

Going up and then hitting down key would kill the snake. There is no “180 turn” that rotates the whole snake

[@debarchanbasu1962]

@@patloeber ah, I see. So by 180 deg. turn you mean 'up-down' and not 'up-right-down' or 'up-left-down'. Got it, thanks!

[@mikesmith853]

brilliant!!

[@patloeber]

glad you like it!

[@navneetsinghtaneja3901]

Good work

[@patloeber]

thank you!

[@rajeev1731]

Finally I found it

[@patloeber]

yeah :)

[@saksham384]

Hey,I followed the whole tutorial and successfully started to train the AI but I wanted to know is there some way to perform the training on GPU instead of the CPU?

[@vishaljain2641]

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu"), if you have an Nvidia gpu, you can use that while building your neural net

[@ahamedalvi1218]

Hello brother first of all this tutorial was awesome. I am new to pygame. Is there a way to take the frame and convert them to image in pygame so that I can feed the image into a CNN?

[@ahamedalvi1218]

Bro one thing how can I avoid loop?

[@destru1914]

Can't you take a screenshoot of it every frame, save it and then read it with OpenCV or something like that? (with pyautogui or PIL to take screenshoots)

[@coopercharles8164]

Do you need a powerful computer for this? Could a raspberry pi 4 work?

[@patloeber]

no this was done on CPU only. Raspberry should definitely work for deployment, training might be a little slow...

[@harperbye]

Can we do this with tensorflow also?

[@patloeber]

sure you can :) it is available with Tensorflow already somewhere on GitHub

[@jackjones1345]

HI, how to set terminal to like yours?

[@patloeber]

I use iTerm2 and oh my zsh

[@alexi_space]

What is gamma?

[@rog0079]

Can this be the start of what's called Artificial General Intelligence?

[@patloeber]

a start, but still a looong way to go ;) it's not really general, because it gets a fixed set of rules just for the game. other than that it does not know anything

[@rog0079]

@@patloeber I see, do you plan to do any project related to general intelligence? Like GPT-3/GPT-J Language model which can come into the category of "general intelligence". What do you think>? I'm so inspired by the movie "Free Guy". You should definitely watch it :D

[@shivamdas9895]

Can a AI agent learn from player after deployment of the game ?

[@patloeber]

no I did not implement this, but you can try :)

[@deepudeepak1390]

This is really awesome ... But I tried the same .. it's took 215 game but still there was no improvement in game in my lap ! Any reason ?

[@patloeber]

Hm this should not happen. Did you compare with my code on github?

[@deepudeepak1390]

@@patloeber yes its working ! i made a mistake while making model.py .. thank you so much for this .. can you please make some videos on GAN's please

[@yan3817]

OMG🔥🔥🔥

[@ankitchetri2968]

I want to learn AI ML stuff with python from scratch can u suggest me the playlist for it please

[@ciscoserrano]

A great place to start is the Machine Learning with Sci-Kit Learn, Keras, and Tensorflow book from Oriley publishing. It’s a great place to start. By the end of the book you’ll have a very solid understanding of ML principles.

[@patloeber]

You can also have a look at my ML from scratch playlist: kzbin.info/aero/PLqnslRFeH2Upcrywf-u2etjdxxkL8nl7E

[@ankitchetri2968]

@@patloeber thanks sir

[@ankitchetri2968]

@@ciscoserrano thank you sir

[@axolotlJonah]

Can we generate a dataset for this project? Awesome tutorial by the way, looking forward to watching the next parts.

[@markadyash]

thanks

[@quentje0484]

Is it possible on phone

[@ianthethird420]

What?

[@cryptojeff3993]

learning this and hopefully can apply into crypto trading!

[@neuron8186]

Why the fuck in pytorch

[@axel5129]

t bo

[@saadjadoon6397]

First

[@modz9711]

Super cool!!

[@patloeber]

thanks!

[@CodeWithTomi]

First