Build an Mario AI Model with Python | Gaming Reinforcement Learning

[@amanlohia6399]

Your videos are freaking amazing. I m currently a senior undergrad and will start my career in applied ML next year. Love watching your videos and learning.

[@PlacidoYT]

Goodluck man!

[@NicholasRenotte]

Thanks so much Aman! Hoping you smash it in the new job!!

[@unixtreme]

I hope it’s going well!

[@Seriosso]

Was waiting for such a video for so long! Your videos shouldb be taught in CS classes! Thank you for the great effort!

[@doremekarma3873]

for people who are getting following error at 15:14: not enough values to unpack (expected 5, got 4) you can fix it by changing the line at 10:44: env = gym_super_mario_bros.make('SuperMarioBros-v0') to env = gym_super_mario_bros.make('SuperMarioBros-v0',apply_api_compatibility=True,render_mode="human" )

[@preston748159263]

Now it says "too many values to unpack (expected 4)"

[@ManishGupta-mo4gj]

Excellent, was struggling for last 2 days. Thanks a ton

[@_obegi_]

or you can do this state, reward, done, info, _ = env.step(env.action_space.sample())

[@Sirobasa]

thanks a lot. what has changed by using the bottom code?

[@eruditeboen3860]

@@_obegi_ You would have to do both

[@offstyler]

Maybe you already know this by now, but by wrapping your env in the Monitor Wrapper by SB3 you can get additional info in your Tensorboard log like how much reward your model is collecting and how long the episodes run.

[@haguda4096]

Can you give me a code example?

[@darkwave_2000]

Great video again!! I learned already a lot from your videos. Big thanks for all the effort !! For future vidoes I would like to see how to advance to a next level with RL. I read something about concept networks where an RL agent can be trained to do a bunch of dedicated tasks. Here in the mario game that would be i.e. jump on the enemies, jump over hole, jump on pipe, collect coins,.... It would be interessting to see how to teach an RL agent some kind of skills instead of just letting the algorithm do all at once. Something like a 2 layer approach where first the needed skills get trained and then secondly by using the skills the complete tasks gets trained.

[@vakurbarut8550]

Newer version nes_py and gym cause some bugs as 1/1/2024 here is how I managed to run the code: 1. JoypadSpace.reset = lambda self, **kwargs: self.env.reset(**kwargs) env = gym_super_mario_bros.make('SuperMarioBros-v0', apply_api_compatibility=True, render_mode='human') env = JoypadSpace(env, SIMPLE_MOVEMENT) 2. done = True for step in range(100000): if done: env.reset() obs, reward, terminated, truncated, info = env.step(env.action_space.sample()) done = terminated or truncated env.render() env.close() Hope it helps!

[@singhamasingh22]

can you share the notebook with me because I'm getting error like SuperMarioBrosEnv.__init__() got an unexpected keyword argument 'apply_api_compatibility'

[@ibandr2749]

what a hero

[@avosoftwareinc.5079]

Bro this is probably the best help comment

[@geriain7448]

I really hope that you will never stop making such great videos! Cheers up man and happy Christmas!

[@NicholasRenotte]

Thank you so much!!! I’m hoping I can keep the run going!

[@geriain7448]

@@NicholasRenotte Due to projects at work I would love to see more about audio recognition or voice enhancement. If you need some inspiration for future videos 😅 Wish you all the best!

[@mrCetus]

amazing tutorial, tried it and almost beat the first level, gotta keep training the model! will check if there are more games to try this in the future!

[@NicholasRenotte]

HELL YEAHHHH!!

[@johanneszwilling]

🤯 Superbly done!! Really appreciate the commenting and explanations. You know your stuff!

[@windrago]

much of this tutorial no longer works (due to updates on ALE and gym) it was well done but for the purposing of learning it would be great to have a refreshed version - great work nonetheless

[@CodingEntrepreneurs]

Incredible work once again Nicholas! Looking forward to diving in. Do you have a discord server?

[@NicholasRenotte]

Yup! Here you go! discord.gg/ZuU5Z5na

[@carveR01js]

Really great tutorial! I just finished a RL cource at school where we build our own DDQN for the gym lunar lander and stable baselines is my next step into RL.

[@NicholasRenotte]

Sweet! What did you build the DDQN with?

[@carveR01js]

@@NicholasRenotte I builded it with Pytorch, it worked really wel and we trained a model that could land the lunarlander quite good most of the time 😁

[@anonymousking2053]

Thank you so much! Demanding for it from 2 months! Again thanks a lot!

[@gplgomes]

Good work. Perhaps you should make a simpler game, showing the reinforcement algorithm being applied: a hole in a wall at a random height moves towards a horse, which it must jump through. The movement is when to jump and the strength of the jump. Once it's in the air, only gravity is at work. In this case you will have to draw the game and make the animations too.

[@NicholasRenotte]

Got something coming in this space with PyGame, stay tuned!

[@study_with_thor]

Never ever missing any of your videos !

[@NicholasRenotte]

Thanks so much for checking them out man!

[@lukedirsuwei664]

great explanation! Used this to solve a few environments. one of the best rl explanations i have seen

[@henkhbit5748]

Super Nicolas, after many youtube training you got your 50K reward 👍🍻

[@NicholasRenotte]

🙏 hahaha it's getting there right @Henk!

[@KodandocomFaria]

this example is so cool. You could create a custom environment to play games like sentdex did with GTA to drive ... Maybe you could do a different game like clash royale where you must to analyse life and choose the best card against the enemy. Or maybe street fighter V, to fighting games you may need to convert image with life and power data to numpy and it can be used as a observation to gym... the interesting pointing here is how to collect those data to create the enviroment . Because from them on you could solve multiple problems beyond games.

[@NicholasRenotte]

On it man! Going to do a ton more games. This was a first crack to see if people were interesed in it. Got Doom coming up

[@Scizyr]

From the papers I've been reading it seems most of the success of RL models is dependent on a deep understanding of the subject environment in order to define proper rewards. I went through your video twice but I can't seem to find the section where you defined your rewards. You mentioned it briefly during the "Using the AI Model" section but I can't find the point it references. Since it's the most important aspect of RL I'm surprised there wasn't an entire section dedicated to it. Thanks for the video I really like the way you set up your workspace.

[@NicholasRenotte]

Thanks a mil @scizyr, you are a hundred percent right. I glossed over it in this video but very much brought it to the fore in the doom video. This is a more detailed description of the reward function (search for reward on this page): pypi.org/project/gym-super-mario-bros/

[@Scizyr]

@@NicholasRenotte Thank you for the reply, I haven't gotten to the doom video yet but I will move to that after I feel I have a good handle on gym retro with some SNES games I'm more familiar with. :)

[@ciaranmalone1700]

Just to note you will need to set !pip install stable-baselines3[extra]==1.3.0 or you will get a render_mode error

[@unknownbro9857]

Did you get error at the 1st command? When installing mario nes py? If not how did you avoided the error

[@tortor2717]

you saved me so much !!

[@darkshot3700]

It’s not letting me install baseline any idea why?

[@ndiritu_nyakiago]

Where did you get the resources to teach this tutorial from Nick? As of august 2023 most of the parts in the video does not work anymore because of version differences and lots of updates but the content is great.I will very glad when you help me

[@vineetsharma189]

Thanks for the amazing videos. Your videos are super informative and entertaining ! What GPU did you use to train this RL Model ?

[@Ugunark]

Great video? What version of python are you using? I'm having trouble installing nes-py

[@NicholasRenotte]

Using 3.7.3 for this my guy!

[@anuragmukherjee1878]

where you able to solve it?

[@adityapunetha6606]

We can also simplify the input by maybe removing the background image, flattening the pixels into just black or white instead of the 8 bit range and maybe replacing the blocks and enemies with just white squares and black squares(this would require a new gym environment), this simpler input might be easier for the agent to understand and might learn faster

[@High_Priest_Jonko]

How would you do all that though

[@Joulespersecond]

Even if you did all that, I'm not convinced it would help. If coins are encoded in the same way as boxes then the AI is not going to learn to use them properly (trying to jump on coins, or avoiding them entirely).

[@alejo0789]

WOW how great stuff is better than any course. Thank you so much I wish you the best!!!! it could be great if we can learn about uav drone or airsim which is on open ai it definitely will be a stunning stuff for all of us. Thanks again.

[@prod.bynach]

Crazy how in just two years everything breaks

[@adamsnell2088]

Seriously, I'm struggling to get most of this to work

[@themoneymaker03]

What broke?

[@canpolarbearfly1833]

I suggest putting the exact python version + all the library versions you used for tutorial videos like this, which would make your video stay relevant for a very long time, as a new learner doesn't necessarily care about using the latest library but just want to learn how things work. I noticed you did it for most of the library but leave out on the nes-py package's version, which caused the source code you posted to no longer work without some modifications.

[@IcyClench]

I believe this was done with gym 0.21.0 - that's the version that was out when the video was released + it worked for me

[@User-ud5sz]

Can you please let me know on the modifications you made since i encountered a lot of issues as well?

[@canpolarbearfly1833]

You can try this: python 3.7 gym_super_mario_bros==7.4.0 nes_py==8.2.1 stable-baselines3[extra]==1.6.2

[@User-ud5sz]

Hello, after your reccomendations i unfortunately still get the error which is a value error and more specifically it says, not enough values to unpack (expexted 5, got 4).

[@canpolarbearfly1833]

That error message means you are trying to assign 5 values, but the function only returns 4, for example, below will return the error: def foo(): return 1,2,3,4 a, b, c, d, e = foo() If I remember correctly, env.step in older gym version return 4-tuple, while in newer version it returns 5-tuple. I guess you were probably doing something like: observation, reward, terminated, truncated, info = env.step(action) instead of observation, reward, done, info = env.step(action)

[@adityapunetha6606]

Love your videos, btw can you point to some resources where i can learn about different RL algorithms like PPO and when to use which one

[@NicholasRenotte]

Yup, check this out, probably my fav sources: spinningup.openai.com/en/latest/

[@jnu4355]

I watched so many AI tutorial videos and you are the best! Can you also do the Health related AI videos such as genome sequencing or symptom diagnosis? Thank you always!

[@NicholasRenotte]

Ooooh, a little out of my skillset, have you checked out some of the stuff from data professor? He's a gun in bioinformatics.

[@_obegi_]

For those that are getting an error saying expected 5 got 4. Change the line where you are taking the step to this. state, reward, done, info, _ = env.step(env.action_space.sample()) Adding the underscore makes it so all other variables are ignored, making the expected more error go away

[@Sirobasa]

still not working. but thx (ValueError: not enough values to unpack (expected 5, got 4))

[@Joyboy_1044_]

​@@Sirobasa change these two lines you'll be good 1) # Setup game env = gym_super_mario_bros.make('SuperMarioBros-v0',apply_api_compatibility=True,render_mode="human" ) 2) state, reward, terminated, truncated, info = env.step(env.action_space.sample()) done = terminated or truncated # add this line

[@Yupppi]

@@Joyboy_1044_ Thanks, this actually made it run. Just got endlessly expanding ConnectionResetError though.

[@manKeGames]

@@Joyboy_1044_ ty !

[@shikhargupta6626]

Awesome work, just discovered your channel, I wish I get more videos from you!!!!!

[@midimusicforever]

Video suggestion - Build a model for a game where nobody has done the integration with Python already.

[@NicholasRenotte]

Yeah, this is pretty hardcore. Will save that for a monster video!

[@midimusicforever]

@@NicholasRenotte *Awaiting eagerly*

[@xolefray6724]

This is freaking awesome! BTW what gpu are you using to train your model that fast?

[@henriquefantato9804]

Hey Nick!!! Another awesome video, I just love this integration of AI and games! One thing that is bothering me is that random output for the same input, running the agent in the same level, the model gets different results for each run.... Do you know why this happens and how I can change it?

[@carsonstevens7280]

Set your random seed when predicting could fix this problem. They are statistical models and will produce slightly different outputs based on the random seed used. For consistent output, use the same random seed. If the game has any randomness to its sprites and locations, this will not fix your problem (if the random seed has been set to be different/changing elsewhere). The input will be different each time your run the level/give the frames to your model and will produce a different action

[@unixtreme]

@@carsonstevens7280huh funny coincidence just one day ago, this is the way.

[@ege621]

Those client talks were simply AMAZING. This could be your step up from other youtubers. While developing a model (or anything really), you basically gave an example of how to talk to your client in a non-technical way so she/he could understand regardless of profession. I liked, disliked and liked again just to double like it. Great job.

[@NicholasRenotte]

YESSS! Thanks so much, stoked you liked them. Was debating whether they were actually good or not.

[@CAGonRiv]

Pretty awesome isn't he bruv ;)

[@rakeshchaudhary1918]

@@CAGonRiv how can i fixed my graphics problem ?? There is not showing clear screen

[@rakeshchaudhary1918]

@@NicholasRenotte how can i fixed my graphics problem ? i am having some glitch in output

[@CAGonRiv]

@@rakeshchaudhary1918 Im so sorry for not replying sooner! Idk why I didn't see your comment. Have you been able to resolve your issue?

[@lamtungphung-cq5bn]

Can someone help me with my bug? Why does my state return a tuple? It shows exactly what it suppose to return in the video but inside the (). This prevents me from checking the shape since we cannot check the shape of a tuple. Thanks for helping me!

[@bboyflamer]

The order of preprocessing matters. Which is in your opinion the most efficient way to process the frames? SkipFrame, then GrayScale then ResizeObservation ? def add_wrapper_functionality(): # 1. CREATE THE MARIO ENV mario_env = gym_super_mario_bros.make("SuperMarioBros-1-1-v0") # 2. SIMPLIFY THE CONTROLS mario_env = JoypadSpace(mario_env, SIMPLE_MOVEMENT) # 3. SKIP FRAMES AND TAKE ACTION EVERY N FRAMES mario_env = SkipFrame(mario_env, skip=4) # 4. TRANSFORM OBSERVATIONS INTO GRAYSCALE mario_env = GrayScaleObservation(mario_env) # 5. RESIZE OBSERVATIONS TO REDUCE DIMENSIONALITY mario_env = ResizeObservation(mario_env, shape=84) # 6. NORMALIZE OBSERVATIONS mario_env = TransformObservation(mario_env, f=lambda x: x / 255.) # 7. STACK N FRAMES TO INTRODUCE TEMPORAL ASPECT mario_env = FrameStack(mario_env, num_stack=4) return mario_env mario_env = add_wrapper_functionality()

[@AdvancedNoob1908]

Couple of things that were not working for me, hopefully it will help others, It's 2023 so some of the API libraries etc are behaving a bit differently, but it's a nice tutorial nonetheless. My changes are as follows: 1. kzbin.info/www/bejne/aJbIiqSAZdp1gac - had to use this instead 'env = gym_super_mario_bros.make('SuperMarioBros-v0',apply_api_compatibility=True, render_mode="human")' 2. kzbin.info/www/bejne/aJbIiqSAZdp1gac - state.shape was not working, but state[0].shape did work

[@singhamasingh22]

can you share the notebook with me because I'm getting error like SuperMarioBrosEnv.__init__() got an unexpected keyword argument 'apply_api_compatibility'

[@cyborgx1156]

Amazing. Can we use reinforcement learning for an agent which can play strategy game like clash royale

[@NicholasRenotte]

Oooh yeah, would need an alternate reward function but highly likely!

[@leosong8091]

Amazing video! BTW, (240, 256, 3) should be height x width x channel

[@robkjohnson]

It would be a bit different of a project for you, but programming a Rocket League Bot would be cool to see. RLbot makes it easy to connect with the game and gather info, some community members have built environments in gym for ML training, can use multiple languages (including Python.. obviously), and overall a cool project

[@NicholasRenotte]

On it! Already found an integration we can take a look at!

[@robkjohnson]

@@NicholasRenotte Sick, I look forward to it! I’m pretty new to reinforcement learning and decided to take on an rlbot which is going… interestingly lol. If you throw in advice on making better reward functions I wouldn’t mind;)

[@Dream4514]

Amazing as usual!

[@FireCulex]

Awesome video. I learned so much. Even cropped to 176x144 and scaled down 2x and added Monitor and TimeLimit Writing livesplits made it easy for to use the integration to add flag and big rewards. also score and dying. Couldn't converge after 8M tho, tried lr le-8, le-6, le-5. Forgot the n_step. Running hyperparam tests for a while...want to see if ent_coefs will help.Setting deterministic just encouraged Mario to run at first goomba.

[@codewithkarthik7136]

hey nich loved the video! Could you make a tutorial on gym retro because there are very few videos which explain them properly. Since Gym retro has over 1000 environments it would be pretty nice to have an introduction video. Cheers!

[@NicholasRenotte]

Coming this week for street fighter!

[@codewithkarthik7136]

@@NicholasRenotte I can't wait for the video nich. I'm so excited!!!!

[@flashkachannel2756]

Cool, let's go Rockstar ⭐

[@hsuan.kung.]

Amazing tutorial! It is really straightforward to follow. If you could add the intrinsic reward to the agent, then it could learn even much better!

[@NicholasRenotte]

Hmmm, good suggestion. I've been learning about reward shaping this week for an upcoming project, agreed that would've helped a ton.

[@Powercube7]

YOOO LET'S FREAKING GOOOOOOOO

[@NicholasRenotte]

Ayyyyyyy! See! Told ya I’d do it.

[@Powercube7]

@@NicholasRenotte Thanks a lot dude

[@benjaminrivera3577]

Good news to those with an M1, pytorch is now optimized, but it is in beta so it doesn't take full use but it runs this faster than before.

[@meetvardoriya2550]

Amazing as always 😍

[@sylvainmazet52]

Amazing presentation! I'll go and plug PPO in smash bros ultimate and see who wins! (hmmmm, just joking, this seems pretty hard to do) Do you know if some game AI actually do reinforcement learning, or any machine learning, "while playing"? I am thinking of AI bots in 1v1 fighting games.

[@NicholasRenotte]

I think fundamentally yeah, what Deepmind did with Dota comes to mind!

[@funduino8322]

thank you so much for the video, I really learned a lot.

[@Yupppi]

My favourite part of this was "NameError" from gym_super_mario_bros and SIMPLE_MOVEMENT and env even though I installed both gymsmb and nespy before using jupyter-lab but still trying to include the pip install like in the tutorial. A classic situation of followign a coding tutorial and copying every move, but getting errors all over. Until I learned I have to click play on every cell from the beginning in order. Just never managed to get that screen of python playing mario. (Until I read comments and tried a couple of fixes). I bet this would be a pretty nice tutorial if every cell wasn't full of red and the left side brackets weren't full of * and me having no idea if it does anything at all.

[@emanuelrouse7076]

Thank you, I couldn't figure out how to get the simple movement to work lol

[@ChristFan868]

Awesome stuff!

[@veil6666]

The fact that anybody can just do RL on their machines now is fucking incredible. How is AGI not here already xD

[@NicholasRenotte]

IKR, literally anybody can pick it up. I mean it takes a ton of time to train but still cool!

[@nostradamus9132]

You should implement a simple baseline that spams jumping, to see if that is everything the model learned.

[@NicholasRenotte]

😂 ngl I tried that to see if trainng for five days was actually worth it. Looking back there's more stuff I would do to improve performance, been getting back into Deep RL this month. A bunch of things we could use to improve outside of just training for longer.

[@nostradamus9132]

@@NicholasRenotte what was the result of the baseline test?

[@nostradamus9132]

@@NicholasRenotte I did a bit of rl myself, it can be quite hard. You should focus on presenting the rl agent with very well preprocessed data. Everything you can simply code should be implemented so that the model does not have to learn that much. I also would recommend trying deep Q learning with a dense NN to predict the Q value. If you are feeling fancy you could go for a Actor Critic Model. Also you should punish your agent for not necessary jumps so that it is forced to use jumps more smartly. You should also modify your Reward function further and it is important to start in a easy environment so that the agent learns fundamental reactions first.

[@chamberselias1903]

Love your videos, was wondering if you could make a tutorial making a object counting api using tensor flow and tensor flow lite

[@michaelalthoff1172]

Great video! Thank you for this excellent work!

[@Louisljz]

Awesome Video

[@eshwaranugandula6907]

Hey NIcholas Renotte, Iam getting erros right from first line of code ModuleNotFoundError: No module named 'gym_super_mario_bros' please help me by solving this error.

[@DK-1907]

after all preprocessing steps when i try to run env.reset() it pops an error -> TypeError: reset() got an unexpected keyword argument 'seed' can anyone help me to rectify it as i cant proceed without rectifying it

[@JohnsonBonson]

did you end up figuring it out?

[@horrorstargames7592]

doesn't work anymore@@JohnsonBonson

[@JohnsonBonson]

@@horrorstargames7592 took a while but ending up getting it working

[@nilakshkashyap8936]

@@JohnsonBonson hey did you figure it out. Could you please help. Stuck with the same problem.

[@shreyasingh3410]

same

[@unknownbro9857]

Is someone getting an error from the starting command?! error: subprocess-exited-with-error python setup.py bdist_wheel did not run successfully. Please HELP

[@amarsarvan6422]

I solved the problem by installing Visual Studio from Microsoft. I guess it was an issue with the C++ compiler, if I understood the forums correctly

[@Ohnomcrose]

I got the game up and running with PYthon with env = gym_super_mario_bros.make('SuperMarioBros-v0',apply_api_compatibility=True,render_mode="human") COMBINED with state, reward, done, info, _ = env.step(env.action_space.sample()) otherwise it would not work. Thought I got through the hard part but then I could not run env.reset() after stacking the frames in the preprocessing part.

[@salmanelouali7041]

did you get env.reset() to work?

[@Moti1197]

Did you get it to work I am also stuck in that part @@salmanelouali7041

[@crem_rock]

bump

[@Moti1197]

@@crem_rock I just want to know if the code still works :(

[@unknownbro9857]

Same prob, any solution?

[@KhangNguyen-jg5ok]

Love your video, may I ask about the sources you researched to figure out the adjustment of hyper parameters ? I am trying to improve the model since the model 700000 is not better than 500000 or random one.

[@NicholasRenotte]

I built a hyper parameter optimisation flow @Khang, I’m going to be teaching it this week!

[@rozzerus]

So, It's really cool. It's banged my mind. But cooler will be to see creating some strategy game RL (AoE2/3/4 or SC2 or WC3 or etc). How to create an environment for big game how to learn AI so difficult movement. Is it possible? Anyway, Thank you very much for your doing.

[@Icekeytrust]

hey, I really like this video :) is it possible to continue it in a certain step? so can I say continue from run 100.000 or so :)

[@NicholasRenotte]

Yep, I don't think it preloads training schedules but you can reload weights using PPO.load then train from there!

[@eonryan8491]

36:13 - where the trouble begins 46:06 - train and logging callback (optional)

[@aakruti8232]

The project won't have significant differences for other games too right? For example contra

[@NicholasRenotte]

Hmmm, it likely will. You'll need a different reward function and gym environment for each.

[@FuZZbaLLbee]

Are you using anaconda, or does the gym now work on windows as well? I remember the Atari gym didn’t work on windows

[@NicholasRenotte]

Works on windows now! I'm using Windows 10 on this machine.

[@dl7140]

Thanks for this video. wow.! It so Amazing PPO.

[@ingluissantana]

First of all, thanks a lot for this greattttt videoooo!!! 🍻🍻🍻 I have one question to you/all: Do you know if there is a web environment that I can use to run this code? I don’t have a laptop when I’m traveling and I would like to try this code… I tried already with google colab but it is not working.. Thanks a lot in advance!! Cheers!!! 🙌🏼🙌🏼🙌🏼🙌🏼

[@avivbenari7061]

on 37:10 when trying to implement env #4 i get: AttributeError: 'SuperMarioBrosEnv' object has no attribute 'render_mode' . can anybody think of a solution?

[@Moti1197]

Be sure to have the same versions that the tutorial has, that would be: gym = 1.17.2 mario=7.3.0 stable-baselines3=1.3.0

[@MrCaradechancho]

Hey, great video. What parameters we should change so the mario learn faster and that we can avoid doing 1 million steps ?, cause my pc is slow. Thanks !

[@jacobstech1777]

the performance is really not good, need to address several issues

[@nourinahmedeka9518]

In the last client interview, you mentioned that there are a bunch of different algorithms. Can you please tell me what are those algorithms?

[@NicholasRenotte]

Here's a solid table of them! stable-baselines3.readthedocs.io/en/master/guide/algos.html

[@nourinahmedeka9518]

@@NicholasRenotte Thank you so much! Can we implement those algorithm to see how they perform for the Mario game? Or is there already published work out there?

[@ProbeSC2]

No offense, but you are absolutely amazing :D Keep up the awesome work.

[@mrWhite81]

Jump and run fast buttons are in or off buttons, Kinda like analogs. Not sure how openAI gym handles that. But great content. 👍

[@SameerKhan-nd5qb]

I have a MacBook M1 Pro. It takes a lot of time to train what can I do to speed it up?

[@NicholasRenotte]

Heya @Sameer, try using an environment with a GPU. Colab or Watson Studio are my recs.

[@donaldduck2852]

which software do you use for typing in the code

[@donaldduck2852]

?

[@hunta2097]

Amazing video

[@sanjaratamuradov5982]

Awesome!

[@januszmewa5229]

Hi Nick!!!! Is there any option to contact you? Because I have stucked in the two lines of this code :( and I am really close to solving it.

[@nocstarix]

hi i got a question, i can't seem to use cuda even though i've copied the tutorial exactly. i'm running a 3070 and i hve the toolkit installed just in case and the model is still running on a cpu instead of gpu. any help would be greatly appreciated

[@christophermoore7676]

Why am i getting " ERROR: Command errored out with exit status 1:" on step 1?

[@ashleysami1640]

👏🏽👏🏽

[@somewhere8]

very good !!!

[@charleslc1853]

that's cool

[@mazenelsayed8306]

u are the best bro u saved me from the fuking college

[@NicholasRenotte]

Ayoooo, hell yeah my guy!

[@yevgeniybondar7509]

Ai thanks for the name idea

[@evairmona]

ValueError: not enough values to unpack (expected 5, got 4)

[@w00tix]

I get that too, which I assume is some versioning issue but can't figure out what's outdated.

[@dan3370]

👏🏻👏🏻

[@aakruti8232]

it says "Building wheel for nes-py (setup.py): finished with status 'error'" what shall I do?

[@uacrazy4481]

Same issue here, cannot even get started as this "!pip install gym_super_mario_bros nes_py" already comes with error's (sad face)

[@NicholasRenotte]

Weird, can you try installing Node.Js? stackoverflow.com/questions/62620393/failed-building-wheel-for-nes-py

[@anuragmukherjee1878]

@@uacrazy4481 did you guys get any solution?

[@uacrazy4481]

@@anuragmukherjee1878 I believe I tried installing Node.Js and still didn't get anywhere, so I do not know what i am doing sorry.

[@firas_pi]

How to train the model after it has already been trained, say that you trained your model for 1 million and you what to train it to 5 million do you need to start over from the beginning?

[@romaricovargas2657]

Hi Nicholas, Your video tutorial is awesome! I would just like to know how I can run this in colab, it seems that the env.render() is not working in colab. Thanks!

[@ProjectSaraswati]

Same Problem here!!! Have you got solution?

[@tomzhao6113]

Same Problem here!!!

[@MrSurya-hn7pu]

It will not work in colab. Run it in your local jupyter.

[@mijkolsmith]

So, do you need to install CUDA before this program will work? I don't see a link in the description on how to install it, pytorch seems to be installing fine

[@mijkolsmith]

Also could you make/recommend a video where we write an algorithm like ppo to train the AI?

[@marceloferreira950]

Hi! where is the link that helps to install cuda? I installed it but when I run the PPO model it always says that it will run with the cpu. however with other projects I have in which I use tensorflow it detects the gpu and runs the models with the gpu

[@davidpulcifer3792]

It worked when I initially ran it, but I came back to the notebook, reran all of the cells except the model.learn cell and am getting an attribute error when I try to load the model code: model = PPO.load('./train/best_model_1000000') error: AttributeError: 'Box' object has no attribute 'shape' Any ideas on what I need to fix?

[@NicholasRenotte]

Double check your observation is still an image @David, Box is the type of space (an openai gym term) that is returned by the environment. This error sounds like the observation space is not being captured correctly. Also how big is your best_model_1000000 file, should be around ~250mb?

[@davidpulcifer3792]

@@NicholasRenotte That file size is correct. 282mb. I restarted the kernel and then reran only the bare minimum cells needed to load the model and run the environment and that seemed to fix it. When I went back to rerun any of the cells involved with the first section where we sent completely random commands, then the Load function would not work. Not sure why, but it is at least working now. Thanks for this awesome video!

[@mauriciojimenez1123]

Hi, I have the same problem. I restarted the kernel and run only the necessary cells to load the model but I still have the error AttributeError: 'Box' object has no attribute 'shape'.

[@Amrhossam96]

Is there any way i could start in ML for an absolute beginner ? Also whenever i try to get through a course i always get stuck in setting up the environment and i feel trapped inside jupyter, spiders. All that kind of stuff. Like i thought it can be as simple as importing whatever i want in vs code and that’s it

[@NicholasRenotte]

Skip jupyter local and just try using Colab (way easier and pre configured). I don't have my course out yet but I recommend looking at PyCaret for some beginner tutorials @Amr!

[@Nemrof0]

Amazing tutorial! Thank you very much for this! I have two questions: 1. How do you speed up this emulation? Tried to search the web for some information but got nothing. 2. How did you manage to run open-gym AI on Windows? I had some troubles running it on my PC, tried installing some Visual Studio Toolkit dependencies, but that didn't help. Thanks again for this tutorial, it helped me a lot with a collage project!

[@ilyaslyusarchuk3664]

the environment already speeds up the emulation to that speed, and about gym/openai on Windows, first install python3, then jupyter notebook, create a notebook environment, and inside of that notebook execute 'import sys' and '!{sys.executable} -m pip install gym_super_mario_bros'. That should install nes_py and gym as well

[@mauzeking6661]

I wish I could find where you setup your enviorment to this point so that the setup steps would be easier

[@NicholasRenotte]

These are the commands I normally run python -m venv mariorl # Creates a new environment .\mariorl\Scripts\activate # Activates it python -m pip install pip ipykernel --upgrade # Upgrades pip and installs ipykernel so we can add the env to jupyter python -m ipykernel install --name mariorl # Adds mariorl env to jupyter jupyter notebook # Starts jupyter, can also run jupyter lab

[@cyborgx1156]

can we change the reward system?

[@funduino8322]

did you find out?

[@NicholasRenotte]

Haven't tried myself but you probably could just by wrapping the existing environment inside a new Gym wrapper. I'm going to show how to do this in an upcoming vid.

[@cyborgx1156]

@@NicholasRenotte ok thanks for replying : )