Can AI make another AI? Using ChatGPT to make an AI bot

We team up with ChatGPT to make an AI bot that can play the Flappy Bird game. Flappy Bird is a relatively simple game with few inputs and only one output. More sophisticated bots take in a larger number of inputs and can have many potential outputs.
Game developed in Unity
This was all done with ChatGPT-4
ChatGPT chose to use the Unity ML-Agents framework
We made Flappy Bird in a previous video using ChatGPT here: • Can AI code Flappy Bir...
I want to emphasize two questions from the Q&A at the end of the video:
Q: Is it safe to use AI to make more AI?
A: In the context of using AI to make bots for games, for sure - because the inputs and outputs are totally controlled. In the more general case - hard to say. These are powerful tools we can use to make more tools, but we should be careful with what inputs and output we provide them.
Q: You’re a game developer - are you condoning the use of bots?
A: Bots are a tool and can be used for a lot of things. If players are using bots to exploit games, gain an unfair advantage, or ruin somebody else’s experience, that’s not good. But bots can also serve positive uses such as being your targets in a PvE game, fun to practice against for a PvP game, provide a friendlier tutorial experience, or fill in for a teammate when they disconnect
-----
Discuss this video and chat about game development on my discord server / discord
I stream gam jams on twitch / candlesan
Twitter: / candlesan
This is Wyatt Cheng’s personal channel. In my day job I serve as Game Director at Blizzard Entertainment. This channel is an independent endeavor and not related to my employer in any way. The opinions expressed are my own and do not necessarily reflect those of my employer.

[@archonthewizard]

I'm blown away by how much information you're able to pack into these short videos. It has the welcome side effect of making them very enjoyable.

[@candlesan]

Thanks Archon!

[@davec817]

this guys gotta be a teacher the way he explains and details information is on another level

[@earomc]

Yeah, he is just so thorough with explaining things. He splits up more complicated things into smaller easily digestible things. And he's really good at that too

[@candlesan]

Thank you! If there are any specific topics on game design or game development you'd be interested in learning more about please let me know.

[@zardify_]

​@candlesan Hey! I have one! We all know a game feels good to play if it's 1. Responsive. 2. Fluid. 3. Impactful. How to, and what makes combat all these things? In theory mostly, as I don't think you'd have many examples to show. There are certainly animation tricks, sound tricks, even code tricks (for example action queues to make sure input is registered) camera and framing techniques and I'd imagine many more. I'm not sure anyone with similar experiences to yours have talked about this. Probably not even close! There are some videos on what makes a game feel good, but most of them don't touch on technicalities or components of these at all.

[@Niko_3D]

@@candlesan Would be cool to get some insights into blizzard workflows! :P

[@antonquirgst2812]

@@candlesan How did you make the bot see... i know its against ToS but I wanted to make a bot playing Diablo (not to exploit anything, but just to see if I could do it) but didnt have an idea on how to make the bot "see" the environment/the map - information of the game while playing is somewhere in the RAM memory where I wouldnt want to access it... since i did not have an idea i was also somewhat unable to make a proper prompt to chat gpt...

[@shadowdemonaer]

The way it rationalized "can't hit pipe if pipe not there" is the funniest thing in the world to me and a whole mood.

[@DJWESG1]

Ceci n'est pas une pipe

[@tobiaskarlsson7565]

Most reinforcement learning algorithms are actually quite likely to come up with a lot of glitches for any given game :) This is due to them spending the first few thousand iterations basically trying every possible action for every state it finds itself in, as it's still not trained enough to recognize any action as likely being better than any other. So training an RL agent on a game could be a good way to discover new speedrun techniques 😅

[@NielNielJP]

People like you are very few in this world. Thanks for doing what you've been doing.

[@BenPanna]

absolutely

[@dtayt1]

facts

[@bassism]

There’s something beautiful about the way that the AI-written bot beat the game with an exploit in less than a second… Might not bode well for the future lol.

[@shadowdemonaer]

I found it super funny and a whole mood. And also I think this is good because it means you can use bots as players to test for exploitation in your games. if they are able to get somewhere they shouldn't, you need to correct that so they can't do it anymore. Pretty cool!

[@shandonprobst9685]

As someone who's trying to find their footing in the vast world of game development, I have REALLY been appreciating your videos. Super well structured and insightful. It's also been really interesting to see these last couple of videos where you've utilized ChatGPT to work as an assistant in your programming. A lot of people are scared of how capable AI has become in the last few years but I feel that there's a lot of good that has come out of it as well. Thank you for your work and I look forward to the next video!

[@tropical2153]

Wow your video is so high quality man keep up the good work, it’s all so well structured, dynamically edited and well explained, you’re gonna go far

[@therealhussein]

I like how you explain things, i know alot of the concepts that you explained but your explanation didn't make me get bored and made me continue through the video

[@goma8167]

Excellent video. I need to get into machine learning it seems so fun. Especially within Unity. I swear, every single ChatGPT video should have a ChatGPT counter to know who said it the most.

[@theyAreGhost]

Insanely high quality content I wonder if he produces everything himself. You rock candlesan!

[@hurtigejler3264]

I came about your video by shere luck, and thank the algorithm for that! Your videos are high quality and interesting. Also your voice is very pleasant to listen to and you seem like a very intelligent and nice guy. This is great! Thanks for doing this!

[@joshuadittmer]

Man this video is great. Love the way you produce vids and explain things ❤

[@MyChunkyGoose]

These videos are super engaging, keep em coming!

[@DrShauryaGarg]

Simply wow! Everytime I am just amazed with what you do with chatgpt. Creative and brilliant.

[@BenPanna]

that's amazing that you could create a game, and you still stick around with flappy bird, keep your track, thanks!

[@LinfordMellony]

Thanks for sharing this! Aside from this, we've also seen the autonomous AI agents from another video which you can use on another game. I've seen also that AI can be used on Unity and you can choose the different sources like images from Bluewillow AI for example.

[@yahyaelhakmaoui]

youtube lacks this type of educational content really thank you for your videos

[@QuadraTwo]

Hey Wyatt, awesome video as usual. As someone who is quite new to games development and super eager and motivated, I really appreciate your insight & experience into issues like AI in games development etc... Since I am not really great at programming, I will most likely have to rely on the help of AI to create all the games that I have been envisioning all over the years.

[@coltonroetker4771]

It actually amazes me how much in common AI has with humans. I did the exact same thing on the original flappy bird back in the day. As soon as I started the game I tried going straight up to see if the dev messed that up lmao.

[@Auroticz]

I really enjoy your AI videos. It’s helping me learn code a lot! Thanks, candlesan.

[@JohnZackYT]

Youre a really underrated youtuber, Nice video!

[@jackytran6540]

Amazing teacher and tutorial, your explanations are so clear! Subscribed!!!

[@TheQAngren]

Awsome video. Now I want to start making things with chatGPT :) Keep up the good work 👍

[@0AThijs]

Great teaching, especially with the amazing visual examples. Thanks 'Reynor' ❤️

[@Gomace]

It'd be great if, in the future, if someone disconnected from an online game, we'd get an AI controlled bot that has been trained on that individual's games, and plays pretty much like they would have.

[@MightyUncleKen]

Amazing content about game development with ChatGPT ! thanks again for posting.

[@DigitalWeenie]

Excellent video and very interesting content. Cant begin to imagine how you wrapped your head around this stuff.

[@ECHO-Gaming]

Very well produced video. I am impressed. 🎉

[@Vichion]

I think there should be a neural network you can train to play for you in some games, that way if you stop playing or become unable to play, your friends can still enjoy the experience of playing with you. As an example Steep has "ghosts", aka a recorded session of your session in a challenge, and whenever I miss my best friend who is unfortunately no longer with us, I can boot it up and play with him again. It would be a bit more exciting if there was a neural network that imitated his playstyle however for some variation. :)

[@jesse2535]

Your videos are great and more game devs should post on youtube about game development.

[@rahulshendre7089]

your explanation is just epic :)

[@i2Sekc4U]

Excellent tutorial. Would appreciate if you compiled all the prompts and all the replies by ChatGPT into a github or zip archive somewhere we could access. You’re really good at teaching concepts! One thing I wished we saw was how did you do the 50k, 150k, 300k, etc.. training sessions. I was a bit confused there

[@thetransferaccount4586]

these videos are super fun. it would be great if you showed the entire coding process, i bet there is lot to learn from it

[@mmmmmmichael]

Please make 10,000 more of these. Subbed up, 👍

[@nj8833]

One benefit of bots in gaming you didn't mention is one that you demonstrated - they can find exploits! Sure, the exploit it found at the beginning of the video might have been easily found with human bug crawling, but in the future, who knows how much more powerful this tool could be

[@MANIAKRA]

This is an amazing channel. Thank you for sharing :)

[@lachlan7181]

I think you should keep making these AI game dev videos they're really good

[@Wildboy_falcon]

Idk why you don’t even have 250k?!?! 😅😂 you are a VERY good youtuber. Love to you man ❤

[@Blue-Lady]

I heard a story someone put an AI against an impossible game. It kept playing and playing, learning the bosses, learning the tricks and so on how to pass difficult areas.. eventually it made it to the end boss, the boss that cannot be beaten. The bot after a hundred or so tries from memory came up with a solution to the problem of losing. Miliseconds before the final boss used its special ability to kill the bot player like it always did.. the bot paused the game and just remained in the pause menu as a means of reaching the closest thing to success. It came up with a solution of not dying.

[@Lykyk]

Careful about the roll-over command with dogs, especially dogs with deep chests, as it can lead to gastric dilatation volvulus, which is very serious and very deadly if not recognized and operated fast. Every minute counts, so if your dog starts dry heaving, panting and has a hardened stomach, go to the vet immediately. Even if you just suspect it, it's better to go twice for nothing than lose your dog.

[@gabrielpauna62]

😂😂😂😂 are you the ai from metal gear solid

[@antares3030]

actually stellar content!

[@meguru420]

The ai has to play the game only with the input state of moving up and forward with its wings. The location of the pipe's empty space needs to be recognized by receiving an image, but I don't know if it's possible.

[@candlesan]

The bot isn't looking at the screen, it's look at the game's internal simulation. As players we see pixels on a screen, but to a bot the world is represented by objects with co-ordinates in space.

[@alois7706]

Awesome really awesome content. Please complete this quality content!

[@JustArtsCreations]

Love it this is some out of the box thinking right here!

[@JustArtsCreations]

Id also love to see the troubleshooting done as well with those errors maybe in a side video?

[@Gustafonair]

This was a great video!

[@ricosrealm]

Nice video! A simple way to make it take a whole lot less time to train is tweak the reward function to give a higher reward the closer it gets to the middle of the pipe gap. I think it would become super-human really quickly!

[@tobiaskarlsson7565]

You may very well end up with a bird dive-bombing pipes in order to pass through the center 😂 Also, as you're not training to maximize the immediate value, but the total future value of any given action, you're better off not artificially restricting the state space. You could end up eliminating a possible future state that would've otherwise had a very high-value action.

[@gabrielpauna62]

​@tobiaskarlsson7565 how did you get into ML I'd like to pick up some resources , I'm a dev

[@tobiaskarlsson7565]

@@gabrielpauna62 I'm not entirely sure, to be honest 🤔 I would probably say I picked up a foundation from "Deep learning with Python" by F. Chollet, and then I started playing around with environments etc from Deepmind.

[@Okiwan3]

TFW you realize Raynor was named after Jimmy 😭

[@tobiaskarlsson7565]

It's potentially confusing that you first say the agent will be trained using RL, but then describe what basically sounds kinda like genetic algorithms :) I understand that trying to explain gradients and fitting to predict actions for value would be way, way too technical for these types of videos, but I'm not sure that describing an entirely different type of algorithm instead is the best way to do it :) That said, I do like both this and the previous video a lot! They're very entertaining :)

[@candlesan]

This is a great point! Thank you for bringing it up. I was trying to think of good ways to teach the concept in ways anybody could understand without getting into neural network diagrams or showing and math. I don't want to spread misinformation so I have made some cuts to the video to remove the parts that would be better thought of as GA rather than RL. Thank you for pointing this out. For anybody visiting this comment later, the deleted section featured highly rewarded birds "mating" to produce offspring. In retrospect the Raynor example is likely sufficient and my "mating" illustrations risk leaving people with the wrong impression on an already complex subject.

[@user-jv4gj2jj3w]

Hey Candlesan, with the rapid evolution of AI, I'm seriously considering creating an app with Chatgpt-4 and other plugins. However, I'm an amateur when it comes to coding. It would be really cool if you could provide a video or an answer explaining what level of knowledge is required to code an app with Chatgpt-4 on your own, if possible. In my case, i only want to design the app and give options and informations professionally

[@MissingTex]

this is amazing i really wanna play/watch it play for me

[@henrymorgan5204]

this is a gem of a videa

[@RedOctober2226]

Loved the story with the dog training, it is similar, and not. I loved that the AI 'immeadiately' cheesed the game. The other thing that I thought about the training models is not just generations of best performance, but also mutations to the bird-brains. Sometimes happy accidents can be helpful too. But that's a whole other topic, maybe by an AI expert.

[@ricosrealm]

It's the most natural way for the bot to cheat because it will likely spam the fly button when it is starting out since it has no concept of what to do other than keep flying.

[@boxiboxi2173]

my dad gotta be proud after he knows i actually learn something from being on the computer all day

[@Smabverse]

Finally an AI video that's actually interesting again :D

[@bigbarttify]

Well done wideo :)

[@nnamdin3372]

You're a legend

[@sioneris1545]

It would be nice to have a video on art generation for small prototype games

[@Yocairo]

Cool video. Your voice sounds a bit robotic due to what I presume is some post-effects.

[@Ferio_]

Awesome video ;)

[@Suomioz78]

I guess what the question of fear is, can eventually all AI bots trained for various tasks connect their knowledge and breed? Or are they individuals like humans? But eventually workout how to share their knowledge?

very engaging

[@deddykosasih387]

You make it look easy that I want to learn AI 😅. Is it possible to see/download the code for the bot? If we want to get a bot to learn a simple game without the game source code, would it be possible?

[@Vinnsmoke69]

i like how his voice audio doesn't change when his outside and inside home.

[@arqamkhawaja]

Waiting for more AI game videos. Love from Kashmir 🍁

[@---Vi---]

Very very interesting.

[@Ronthedon485]

Thats sick

[@Nightstalker314]

I remember that in the early 2000s with games like Farcry and Crisis pushing the envelope there was a lot of press talk about the enemy AI and their reactions to the players performance. How far has this advanced by now? Not just in terms of combat but also creating immersive RPG worlds? Or is it just too broad to create a cohesive experience. Lately RPGs leaned heavier into progression paths and reputation with NPCs etc but those mostly fall into scripted outcomes. The only genre where I at least saw discussions about the opponent AI topic lately were racing games. Might be personal bias or lack of a broader experience but: Where are we in terms of AI reacting to players actions as part of the gaming experience? Edit: the Google AI once had showmatches against players in Starcraft 2 (and then there is also Chess and Go), but outside of these incidents I haven't noticed much talk about opponent AI development. WoW had the Island expedition and Warfront as well as PvP brawl AI features but those still feel like a locally obvious script compared to a globally scripted encounter.

[@candlesan]

AI is a fairly general term but I'll comment from two different angles in response to your question (but there are other meanings as well) - AI has advanced an enormous amount for games since Farcry and Crysis but the important note here is that "super realistic behaviors" isn't what you're looking for in many games. The objective is usually an illusion of realism with meaningful and fun gameplay experiences for the player. - There is extensive opponent AI development, it just doesn't make headline news, but there is definitely development of AI opponents for almost all games out there whether it be connect-4, poker, etc or computer games like FIFA or Fortnite

[@EmaManfred]

You could assign an AI agent/agent GPT to try and learn and create another AI. Basically, an AI learning and creating another AI. A lot of AIs could make use of it especially for the those that utilizes deep learning like singular task AIs like Bluewillow AI among others.

[@josueC235]

Chino tenías que ser! Que grande ❤

[@TopX5s]

You are so wholesome!

[@misterandylink]

That original exploit was perfect. I was like "Yo did it literally find the true solution?"

[@fitybux4664]

Did you try walking backward from the starting point in the game? 😀

[@sanmar6292]

Seems like AI bots will also be a great tool for playtesting / bug finding in the not so far future.

[@AsoSnT]

so cool !

[@KVI3I]

at this point i dont want to call you candlesan.... you are CANDLE SENPAI :D teach us more pls its so interesting and you explaint everything very good

[@NeCoNLive]

You are awesome :D

[@Dante190884]

do you see the potential to use gpt4 in the production of Blizzard games?

[@jxw7196]

Excellent!!!!!!!!!!!!!

[@datguycloudy]

ChatGPT coded a way for people to find your channel.

[@bettyogbode8472]

He killed me when he said bird brains

[@huzaijack4282]

Candlesan has mastering AI chat bot that he soon will be the representative for the AI.

[@postedrandomly]

This man shows us the example of how Skynet developed

[@notlaughinganymore1821]

Great video. Gorgeous dog too, what breed is he?

[@candlesan]

It's a mutt - I'll let my friends know though!

[@antukin5469]

my favorite programmer

[@WanliLoo]

keep it up boss!

[@WanliLoo]

really clean prompts ngl

[@GumbieUsaGames]

What do you mean by training your bot ? Do you save all the attempts data it has made during that training ? I m not able to understand this training process of an AI not 🙄🙄🙄🙄 please someone help

[@rodgerandjeanniewenzlaff3165]

Can you add to your notes - how long did this one take you?

[@Hibro563]

How do I make a 3d platform game on unity pleas make a tutorial on it

[@jeffcummings3842]

nice choice of games to train an AI on, haha, you just wanted to say 'bird brain' rofl, also thanks for sharing!

[@brandonjones2565]

Would be sweet to make an AI bot to play the OG Battle Toads on NES

[@Join_My_Teligram_channel]

Just wow 😮😮

[@mybiggrin]

Puppy got a PAWsitive reward.

[@Antiorganizer]

I have an idea for you to try to make! A pacman game where the monsters are so good that you can not possibly win. LOL!

[@JustFeral]

That text box design is gonna have Nintendo trying to sue you lmfao

[@rgtstudio911]

nice video

[@SmartLearningAI]

How does the bot actually play the game? Ie a build version or editor version?

[@candlesan]

The bot is trained in editor mode but once the training is complete it produces a data file which can be added to the project so that the trained bot can play in either editor or a full build.

[@malicekerendu3574]

Playing god, aren't we?

[@Dangerpurple]

Finally, AI makes itself obsolete, take that AI!

[@Mirko_ddd]

I didn't know you had 2 twins 🤯

[@BapeRuLLZ]

When the Bald One arrives, your channel will skyrocket.

[@Unchained_Alice]

As long as it can't make a more advanced AI than itself we are safe! (I'll keep telling myself that)