Neural Network Learns to Play Snake

[@MrPman1999]

You guys think that the snake died because of the lack of left turns, but in reality the snake evolved to the point where it got consciousness and understood that life dedicated to running in circles is not worth living.

[@nsfrozen]

No One ur joke is so dark, it darken my life

[@user-mi8ew2to8e]

WoW

[@2glizzyy]

almost cut myself on all that edge

[@Alva1326]

No One agreed

[@benbabu9404]

That's deep

[@morphman86]

For anyone who wonders why it seems to prefer right turns, I believe that is because it started at the top-left, going towards the right. There was no way for it to turn left. So with 2000 snakes per generation, a LOT of those learned that left is death. Since right worked every time, it simply had no reason to learn that turning left after leaving the wall would be safe. I believe that is also why you got those wiggly motions. That's it trying to turn left, but then immediately turning right again, so its profile won't go any further to the right than the starting position.

[@linuxatheist5361]

It might be interesting to see what happens if the starting position is randomised

[@trex70]

What about the middle ?

[@Tracer527]

@@trex70 middle and just go down, snake will choose Left or right way by red points

[@badkingjohn5235]

About what I thought, but does this mean it is unable to get significantly better scores, because it will suicide inevitably by coiling up instead of folding itself?

[@morphman86]

@@badkingjohn5235 The most likely scenario is that it will discover that it can fold itself in that direction, which makes it survive for some time longer. It will therefore take days, possibly even weeks, to simulate it to the point where it learns to fold in both directions, or fold and twist around.

[@funny3511]

ME: "Hello World">>20 errors found.

[@NoctumusTV]

Funny: Funny! Funny ... ?

[@tophatv2902]

@@NoctumusTV what?

[@marls3518]

@@NoctumusTV Can you pls explain again. Thanks in advance.

[@tophatv2902]

@@NoctumusTV oh ok

[@NoctumusTV]

@@marls3518 Explain what again?

[@cap1819]

My favorite part is every time you think the AI finally has it down, then runs into a wall for no reason Every time

[@user84074]

that's what God said, when watching humanity

[@AtticusDenzil]

there is a reason, the human element is a fuck up disgrace in this case

[@bluespottedcube100]

Maybe time for revolution

[@puppergump4117]

@@user84074 Then God killed the program

[@mostlife7]

Same as humans)

[@Taikore_]

2:26, that’s literally just a dog

[@justronjay9226]

literally

[@KazimierzRat]

literally

[@ubern00bkye]

Literally

[@tinweirdytthingy9571]

literally

[@johanm5018]

yllaretiL

[@XPimKossibleX]

I love that the reason it failed is because that's the one flaw of the technique it's honed from the start

[@benjaminfeddersen7937]

Lock in. You hit on a successful strategy which gets you all the way to the point where it is no longer successful, but by then you can't do anything else. A typical failure mode these kinds of systems, from corporations to civilizations.

[@Bebolife12345]

Benjamin Feddersen While you’re correct that adaptation is a very necessary skill. Michaels point was more about BAD HABITS than an inability to adapt to new circumstance.

[@gingrich1000]

Bebolife A bad habit can inhibit adaption.

[@Alexlink1]

@@benjaminfeddersen7937 Dude this shit is deep as fuck.. It's the epistemological concept of paradigm. Any paradigm in order get surpassed need first to collapse on its own rules, unable to explain or resolve newer problems

[@zybch]

There are no bad habits. Just bad outcomes.

[@riverrist]

This neural network is incredibly inefficient. Right from the beginning, it learned to not turn left by any means. This video is perfect as a demonstration that neural networks can easily get stuck on a very wrong local optimum.

[@superpantman]

bit of an analogous for humanity, don't you think?

[@manzell]

It's not inefficient - it has an energy cost of 0, there's nothing to constrain it's time. If there was an energy cost (negative reward function) for turning, it would optimize its routes.

[@saberseesall]

@@manzell good point!

[@jessicastrat9376]

@@superpantman not really, as humans use a mixture on neural networks and symbol manipulation. That’s why AI (recently overly focused on association and deep learning) are not progressing as fast as hoped.

[@c4kefrosty862]

@@manzell And perhaps adding in more genetic variances from generation to generation to allow novel ideas to die or take hold.

[@prodkilobyte]

Left turns: *Am I a joke to you?*

[@clarkkent6026]

I observed the same thing; is that a design flaw?

[@pocketrocket27]

It's not an ambiturner.

[@vakie3250]

Neural networks is a lazy algorithm and will take the shortest route to achieve its goal. My guess is that the input of the distance from the left wall from the snake plays a significant importance to its decision making. You can use dropout which will force other nodes to train that never gets a chance when the whole network gets trained

[@Super-id7bq]

@@pocketrocket27 God damn you Ivan - you beat me to it. Damn you to hell :D

[@nazmussakib1551]

but it took left turns

[@rutvikrana512]

I think another one reason why this doesn’t get higher because in input it gets - 1. Distance to food 2. Distance to wall 3. Distance to tail Wait but what about its whole body ?? So that’s why snake trap around it’s own body. Just a guess though 🤔

[@PredaFortyTwo]

thought the same, but could the lenght just be another input neuron ?

[@zkenk]

We could probabaly include the previous outputs as an input like an LSTM or GRU

[@suvigyajain9360]

Perfectly correct. If you feed it the location of the whole body in terms of a matrix it will evolve to the point where its better than any human playing the game

[@CostantinoCarta]

Nice observation

[@log234]

Yup, using a CNN would be a good approach for this problem, I think. Use a different weights for the head, tail and the location of food.

[@nanxhu]

Humans: *computers will take over the world and destroy us all* Computer: *hehe line go zoom*

[@mesq999]

not funny didn’t laugh

[@ubern00bkye]

My disappointment is immeasurable and my day is ruined

[@usama2576]

snake go brrr

[@nahimafing]

@@mesq999 And this is why no one likes you at school

[@dawidek4267]

@@nahimafing Just because he has an opinion slightly different than your opinion, it means no one likes him? You are a fucking asshole

[@ItachiUchiha-nx2sw]

I have deeply learned that in the end, nothing is left.

[@TtttTt-ub5xb]

You're right

[@AntoineViallonDevelloper]

Lmao

[@techley4322]

This is hilarious

[@re_claimer_]

But don't massacre your clan in future

[@soupnoodles]

@@re_claimer_ how about you go watch shippuden? clearly you dont know shit

[@uchihatomy]

congratulations for the big work you've done, not only at the algorithmic part, but the visual part which i can see it's a huge effort to present us your job.

[@zackrodriguez6653]

Generation 30: *dies* Me: YOU WERE THE CHOSEN ONE

[@kvadityasrivatsa2445]

"What is my purpose ?" "You pass butter...."

[@markgeorge447]

No one: KZbin when my lil brother uses Wi-Fi 1:22

[@Faisalamin01]

he must be downloading extra ram

[@markgeorge447]

@@Faisalamin01 no he was downloading graphics card 😂😂😁

[@Artsu1993]

It takes a few generations for any significant progress to be made

[@Flourish38]

I think it probably would have learned better if you had started off with a lower number of moves left (maybe like 60?) so that it doesn't have so much security to take its time.

[@GreerViau]

That is definitely possible

[@arthurfacredyn]

@@GreerViau Also, If you want them to evolve how to avoid hitting themselves better try making the map small so that they encounter tat problem sooner

[@brayanhabidcol]

@@arthurfacredyn That's especially true if the improvement yield was already capped, with a lot of room still available around.

[@Arik1989]

You could also add a small negative score for each frame, so that it prefers to die rather than do nothing, but it might get stuck in a local optimum of immediately killing itself.

[@pakokiller89]

@@arthurfacredyn Or making the snake longer right at the beginning so it can develop strategies for avoiding his body

[@abhiramcd]

I can't imagine how happy would be the first guy who developed these algorithms.... ❤️❤️

[@DaDoge927]

Yes

[@mohammadwasifhossain8632]

yes

[@JamesRodriguez10783]

Yes

[@canofpulp]

We

[@twishasahay3178]

Ikr

[@sciencesyfy]

The mind of the Snake in the first few generations, spinning to infinity a pixel away from the food "FOOD FOOD FOOD FOOD FOOD FOOD FOOD FOOD FOOD FOOD FOOD FOOD"

[@gizmo401]

sciencesyfy this actually made me laugh and not just breathe fast out of my nose, gg

[@Yazan_Majdalawi]

I laughed at this

[@sykeassai]

What I think is most fascinating about this project is that the neural network never learned the dimensions of the game board and kept returning to the start

[@blzrL]

It's so fascinating to look at a neural network learn and it be visualized, it's like a mini brain in a computer learning and reacting to their surroundings, telling a machine that only follows orders to figure it out themself

[@RolandTitan]

Theyd get further with more information. You forgot a key piece. Direction of "motion" of its tail. While not immediately obvious in game its something human players take full advantage of when they get stuck on inner loops

[@bencekovacs8726]

becominghuman.ai/designing-ai-solving-snake-with-evolution-f3dd6a9da867

[@dananderson8459]

Do you think it would perform better if the input to the network was the grid array containing all the information about the game state. eg a 50x50 array of numbers 0, for empty, 1 for snake body and 2 for food. Or is it better to explicitly tell it the distance from the food, is it unlikely to work it out itself?

[@romeovalentin5524]

@@dananderson8459 using convolutional layers instead of fully connected layers probably yes, otherwise probably only with a significantly larger network

[@Lucas-jq6kk]

what if there was one value for head position, one for head direction, one for food position and a vector for the entire body I'm a noob but I think it could do very well with this

[@psyneur9182]

if the network also used some recurrent design (such as an LSTM) it could possibly compute motion and have better planning abilities

[@immanuelkant7895]

Could you make a video where you explain your code an how you determined fitness and the mutation and crossing over procedure?

[@nottheengineer4957]

I looked at the code a little and while I don't know the language, most of it is rather simple. The weights are stored using a self-written matrix class, which is a 2-dimensional array with a few methods to do matrix stuff and for mutating and crossover. Mutating adds some random gaussian noise to every weight. You can look that up in the github repo in the file Matrix.pde The crossover method selects a random coordinate inside the matrix. Anything above or left of that coordinate uses the values from partner A, anything below or to the right of that coordinate uses the values from partner B. The fitness is just the length of all the snakes in a generation added up. I learned this kind of stuff in university and this project goes against a lot of what I learned. For a practical application, these functions would be pretty bad and most importantly, very slow. But the whole thing still works very well, so well in fact that without knowledge of the subject, most people wouldn't be able to tell it apart from a more professional approach. It shows that machine learning isn't hard on its own, but the tools that are used nowadays are pretty complex.

[@mauriciomontalvo5885]

@@nottheengineer4957 in which program or app can I do these kind of stuff?

[@ErrorNotFound-ly7zh]

@@mauriciomontalvo5885 Well u can use any programming language i presume, though some are better than others for these kind of things. If u want to hard code it yourself i would use something fast, but you won't likely achieve great performance unless u really know how to optimize the hell out of it. What you can do is use NEAT or tensorflow for example in python. Combined with pygame you could do all kinds of things like this. NEAT is extremely easy to use, to the point that you barely have to understand what is going on.

[@rorhianskall5659]

@@nottheengineer4957 Where to learn about more professional ways and tools they use? Just for curiosity and learning purposes (obviously without getting into uni, too old and too broke now for that).

[@aaaaaahhh9537]

Hi Kant☺️👋

[@SmokeDoinks420]

6:07 love how the snake eating the food is perfectly synced up to the songs snare until around 6:22

[@boo7948]

lmfaoo thats pretty cool

[@chivalrous_chevy1163]

Lol neato

[@Krakyy]

The snake evolved into being able to understand the music

[@ombean6443]

actually also at the start of gen 30 (from around 5:00 onwards) it’s synced up in some ways

[@shanalcordo7174]

😂😂😂

[@13mod72]

A quick suggestion: don't constrain the neural net so much. Give it the entire 38 by 38 grid with three possible values for each location (off, snake, apple) and train using those inputs. It can even be considered a vision problem at that point, and modern ML libraries can solve it with a convolutional neural net pretty effectively.

[@Caffeine_Addict_2020]

Wouldn't that be far, far more computationally intensive? Genuinely asking

[@sukritmanikandan3184]

@@Caffeine_Addict_2020 not really, considering modern hardware can comfortably run CNNs on proper images, 32x32 grid of pixels is nothing

[@CatDevz]

@@Caffeine_Addict_2020 relative to this model? Yeah. But it still wouldn't run slow on modern hardware by any means

[@mconfalonieri]

@@Caffeine_Addict_2020 it would easily run on a consumer-grade GPU.

[@Andrecio64]

2:34: "everybody gansta till snakes start walkin"

[@meisam9592]

This is what happens when you don’t consider the “time-to-solution” in your fitness algorithm!

[@yurimrt]

That's exactly what I was thinking, along with the fact that the player usually is not the snake, so there should be a couple of input neurons more with the position x-y of the food

[@uwu_senpai]

@@yurimrt Yes and the cartesian distance to the food sqrt((Xsnake-Xfruit)²+(Ysnake-Yfruit)²)

[@MrDragonorp]

@@uwu_senpai yeah but that works only for whne the snake itself is not blocking the path, there needs to be a priority set that it just need to find the shortest path to is next objective, like going out of the block by the snake which can be obtained by looking if the snake is on the x way and the y way to the food and if it is look for the shortest path possible for that not to happen.

[@mirabilis]

@@uwu_senpai Euclidian? Sounds like a bad idea as you cannot reach the fruit in less than |xsnake-xfruit| + |ysnake-yfruit| ticks

[@MrDragonorp]

@@mirabilis but you know if there the snake doesn't block that path, it's the fastest way possible, there is no faster way, it's just math.

[@ryannemo1124]

The song works so well with this video. I am feeling so calm right now lol.

[@DaDoge927]

KZbin Algorithm: Dis looks guud, lemme recommend it to everyone

[@marathonour]

At 2:20 starts feeling like I'm watching a movie about a guy who was weak at the beginning but he starts training more and more despite his failures and finally he comes to success

[@zLcss]

Anyone else deeply in love with the first song ? It’s so calm and nostalgic

[@MudakTheMultiplier]

I would like to see this but also with an adversarial neural network placing the next food piece.

[@Rx7man]

or two snakes, each racing for the food

[@MudakTheMultiplier]

@@JohnSmith-xf6nb I feel like allowing it to change size would result in it shrinking the board as small as it can to reduce the number of points available.

[@MudakTheMultiplier]

@@JohnSmith-xf6nb I think that might go to far the other way, because a bigger board would mean less points per apple. Maybe if the board is smaller than whatever the "standard" is, then the points awarded increases in proportion to the number of points lost? If you're trying to add a new thing for the adversarial network to do to try and mess up the main one maybe it could also spawn "bad" apples that either kill the snake or remove points. I think that would be interesting because then the snake couldn't always just navigate directly to the apple, it might need to avoid something it it's way and the adversary could try to place them in choke points and such.

[@RubyPiec]

2:31 me when i play tetris and i know im gonna lose

[@ber2996]

The strategy backfired if it becomes long enough, a rule telling it should calculate its length vs the size of the field before making a move should be applied

[@AD-wg8ik]

I didn’t even know you could move diagonally like that

[@user-tkvlfrnlwjd]

6:27 The snake is so long that it forms an enclosed space, and the new prey is outside. There is a way to use the inner space to get out of the narrow gap.

[@ampotat9018]

People: omg ai is going to dominate the entire world Meanwhile, the AI: gonna go get max scoring in snake

[@RelentlessDebique]

This comment hasn’t aged well :))

[@OneBiasedOpinion]

Question: do you have to run this in real time for every iteration in order for the neural network to properly "train" in the game environment, or can it just run the code, sans graphics, and do it almost instantly with accelerated, simulated time?

[@pootischu]

I only recently watched 3blue1brown's video regarding this, so my understanding is deeply flawed. Please cmiiw. Basically, I think the "learning" part depends on the value added to the function result in the neuron or dots (in this case one of the four rightmost dots) that is the "right" answer. Of course, the "right" answer is a combination of complex sequence of four dots (=sequence of left right up down) and cannot be determined by simple program, so the machine basically determine the next action based on the values added. Sometimes, the "best valued" action may not be the best course of action, and then the machine will have to rethink the whole sequence. Thus, I think for the machine to my understanding, it has to run in a simulation because of the complexness of action needed. It can be speeded up though.

[@OneBiasedOpinion]

@@pootischu gotcha. Thanks for the quick explanation!

[@maxguichard4337]

To clarify, since the snake moving a tile, or an apple generating etc. is bound to a "simulated clock", it doesn't matter how fast or at what intervals it ticks. The results, the data we feed the network, are identical either way. Much like how if you calculate 2 + 2 and wait 5 seconds before saying 4, it's still 4 if you say it immediately because the function calculating 2 + 2 doesn't care about time (the function of snake being the code, with the game state as its input). So the answer is yes, you could run it without graphics (and is in the video since you don't see 2000 snakes at once), and indeed it could be trained as fast as the processor can run the game and the AI.

[@OneBiasedOpinion]

@@maxguichard4337 that's what I was asking. And that's awesome!

[@adomustafa1777]

1st generation : I'm hungry 30th generation : solved the hunger problem 100th generation : discover the network 500th generation : taking over the network 1000th generation : human extension.

[@vibinv8905]

*extinction

[@adomustafa1777]

@@vibinv8905 let it go man 😂😂

[@vibinv8905]

@@adomustafa1777 The OCD just took over :D

[@PsychoBackflip]

@@vibinv8905 Butt in your moment of "OCD" did you notice the choice ? You see even though it might not feel like it (regardless if you have this so called OCD or not) there is always a moment where you have the choice. The thing that told you that you wanted to correct him is simply an impulse and you have complete control over your impulses. It no longer works to say oh blame it on my OCD because YOU chose to listen to it. Whatever reason you have for making the decision, it always comes down to you. A habit is just a choice you keep making.

[@vibinv8905]

@@PsychoBackflip thanks for the pep talk.

[@LanceBryantGrigg]

Given your input layers it makes sense that it started to struggle when the worm got to big. It doesn't have the input layers to detect spatial availability like that.

[@알쓰-k2z]

Can you try starting to train the model with larger length similar to the length at the last death?

[@hoopsgators]

You should nickname your snake Derek Zoolander because it appears to struggle to turn left

[@user-kx5es4kr4x]

it doesnt

[@ubern00bkye]

@@user-kx5es4kr4x it really does

[@alix6xgorg839]

Underrated comment

[@pavlotkachenko6970]

Did your snake consider itself as a wall during the training?

[@ノヨイチ]

0:16 this parameter has its own set of 8 input nodes : "Distance to its tail"

[@pavlotkachenko6970]

@@ノヨイチ Nevermind, I've deleted that answer. I've watched once again to make sure you were right -- the snake considers its tail as an obstacle

[@GoriIIaTactics]

It would be interesting to see colors for the hidden layer nodes as well, colored for their activation level Plus a gradient for the weights instead of just blue/red

[@peeper2070]

This is oddly philosophical. No matter how much we advance, we will keep progressing, all while securing our own downfall.

[@Ozymandias83]

I love how it likes to return to top left before making next manoeuvre, shows the training

[@stewiegriffin6503]

Samir, you are breaking the snake. Samir, you are not listening !

[@pickachu3739]

What the deuce !!

[@stewiegriffin6503]

Who are you ?

[@pickachu3739]

@@stewiegriffin6503 that's what I am supposed to ask. Who are you! And why do we look same

[@farooq8fox]

Shut up, dont tell me who to drive

[@anthonyt4154]

Looks like Stewie has been messing with the time machine again.

[@Swastik15]

2:39 - The snake has evolved into a dog.

[@Muuip]

Nice visualization combination of the neural network firing and its effect.

[@SwetankRaj]

Let's say I also want to create such visualisation, how should I do it?

[@cm-a-jivheshchoudhari9418]

What software is used to visualize the working of neural networks?

[@jeremyr7692]

Let me know if you found the answer please. I am wondering the same thing.

[@Andrecio64]

implementing reinforced learnig to give reward signal as goals are met is a good way to speed up machine learnig process

[@dhairyabhatt3156]

I don't know if you observed or not but snake is doing clockwise rotation most of the time

[@cynthetic4896]

Basically I believe that the snake is using the wall as a map, the neural network doesn't know where it is on the screen, only distance to wall, tail, and food, so it travels around the edge because it's a significant boundary, then when it gets the closest to the food, it travels in a straight line until it meets another wall, with some small differences in between depending on distance to tail.

[@MybeautifulandamazingPrincess]

The snake is Republican

[@deletevil]

@@MybeautifulandamazingPrincess lmao xD

[@nemesis9410]

Optimal strategy often is not the entertaining one.

[@Tethysmeer]

Interesting question. Could be random. A successful generation introduced it randomly. Or it has some deeper sense. More Galaxies are rotating counter clockwise.

[@psteig395]

Heres to where youtube recommendations lead me to today during quarantine :D

[@razkarl]

Woah, great visualization Can you name-drop some of the tools used to create this?

[@seungminshin7652]

ㅇㄷ

[@26dimensions70]

There aren’t any “tools”. You should start by learning about neural networks and deep learning, and try out a few simple networks to learn how to program them. Once you’ve got a grasp of neural network programming you can pretty much adapt them for any problem, and expand the hidden layers and neurona where necessary.

[@razkarl]

@@26dimensions70 Thanks for replying! I am in fact a computer science graduate, and I also have a degree in Industrial Design. My question was about the visualization tools you used to produce the video, I'm fascinated by the animations and would love to learn how to produce similar demos of my own ML research :)

[@razkarl]

You're welcome to take a look at one of my projects (an artificial intelligence constructing objects from a 'LEGO' like building block I designed) where I used Python's mplot3d to create a set of images I converted to animated gifs to visualize the algorithm. www.razkarl.com/projects/kawaz

[@nithinnaikar7373]

@@razkarl Take a look at OpenAI gym. It is a virtual environment used for reinforcement learning

[@stephnes2505]

I always find it so cool how humans have essentially simulated evolution in machines, theres literally nothing we can’t do

[@akruijff]

Sure there is. We just do not know what we can not do because we do not know our own limitations fully.

[@stephnes2505]

@@akruijff Human potential is unlimited. In a few thousand years we might be able to drastically alter our entire genetic evolution in a week, or put out a star. There’s really nothing science can’t achieve.

[@fozcel]

More intense than any latest action movie fight scenes :D Respect!

[@Steelrat1994]

The limiting factor is the input vector IMO. If the snake operates only on relative distances then no matter what - it'll end up encircling itself and getting stuck.

[@skyphab]

Awesome! Now make it two AI-players: Your snake vs AI that places the food with the opposite target: Reward if the snake dies. That would be an interesting experiment :)

[@eliseerickson5994]

this is the coolest thing i have seen in my entire 18 years of existence

[@suzanchhetala3840]

Holy shit! This is amazing

[@ItsSunnyMonster]

Those people who disliked don't know the hard work he put into it. And apparently, 1845 people can relate.

[@_GhostMiner]

*Me:* * _sees the thumbnails_ * *"Wait, That's illegal!"*

[@jiahuiwei3232]

why don't you do this by reinforcement learning? it gonna be better

[@evanbernard577]

he used a genetic algorithm, it's a type of reinforcement learning, the snakes with a higher fitness are more likely to breed, which is reinforcement

[@sherlockwisdom]

5:48 it's starts eating the red dot on the beat 6:17 begins the killer moves to the beat 😂😂

[@gabrieleldose8063]

Nice work bro

[@anthonylomasney7347]

It’s crazy. I feel like watching these videos is like witnessing cellular life form in the oceans billions of years ago. Someday in the future this technology will achieve a higher level of sophistication than we have.

[@walbermr]

Reinforcement learning (RL) can lead to better results in lower time, compared to using genetic algorithms. Google, openai and other research teams, like the one i'm part of (RoboCIn) are using RL to play soccer, dota, starcraft... Great initiative to solve the problem, make a video and share the code! 👏👏👏

[@DarkBrainDevil]

wow cool ,are codes public for that?

[@cmyk8964]

I think it’s good to reduce the number of moves each apple rewards, to discourage a strategy where it wastes moves circling around the board. It would also be interesting to give the snakes a _general_ idea of where the apple is at all times, by giving it a fractional value depending on the angle of the apple relative to its head.

[@tinyplaidninjas8868]

This reduces its ability to reach high scores though. Circling the board is the optimal strategy for surviving when you have a really long body. I think the input neurons, particularly distance to tail, are too simple since they don't take into account the position of the rest of the snake's body. Perhaps an additional neuron could be "distance to nearest piece of body from 3 ticks ago", since 3 ticks is what it would take for the snake to make 3 right turns and eat itself.

[@ralvarezb78]

Terminator: ....SnakeNet begins to learn at a geometric rate. It becomes self-aware at 2:14 AM, Eastern time, August 29th. In a panic, they try to pull the plug.

[@OnemarioOLDCHANNEL]

Nobody: Neutral Network at 2:26: I will now speen

[@romilrh]

Me, who can't get past a score of 20, watching the early generations: *_"A M A T E U R S"_*

[@stk928]

zoolander bot only turns left

[@임재범-i2j]

At last he did it!

[@medexamtoolscom]

You mean right. It only turns right. But on the bright side, it is RIDICULOUSLY good looking.

[@jdegreef]

And eat only from top to bottom.

[@Robiness]

You might have gotten better results had you let it start from the middle or from different places every time ^^ Great video! I glad more people are taking interest in neural networks

[@wallflower5130]

I'd have said that the problem might be that he is selecting only the best out of the 2000 snakes. That leads to a strategy which is only a local maximum. That's also the reason he doesn't get better results by training further. It's hard to get out of that when you don't allow the chance of exploring other strategies which are not locally the best. His population was too small and the mutation rate too low to fix this issue. You'd probably get better results by selecting a small group of snakes with equally distributed fitness.

[@Caffeine_Addict_2020]

I mean, this then adds an "RNG" variable, which you really don't want, no? A snake may have better fitness because it got a lucky placement, and you don't want to breed for luck because that will be completely random

[@ChrisContin]

Wonderful demonstration! To train a condescending, plural-array You’ll always need the “imaginary side-node”. It’s a fictional response that always concurs with the ideal national response. In this case, the network must revolve first left then no other direction, or vice-versa. The side-imagined node will condescend any alternate output, here. You’ll see the snake “win”, but you could’ve done that simply. A “national, variance-norm” sweeping network is not about the output but the internal shape- it’s a complex geometry very easily, sending product information all over! Imagine a space-station inside a server-network on Earth like this! Take care, My Child.

[@ChrisContin]

@Dale Owens A “neural network”, including the human brain, is only lightly about the output, and much, much more about the path. Specific to “winning”, I also said, the ideal “snake game” behavior is simple: turn only left or right, and so any AI playing snake (or anything else either) doesn’t need more than a ruleset to always win. I’m a Researcher, having a PhD in Game Theory, Science of Matt-Brainology, and another, more.

[@wallflower5130]

I'd have said that the problem might be that he is selecting only the best out of the 2000 snakes. That leads to a strategy which is only a local maximum. That's also the reason he doesn't get better results by training further. It's hard to get out of that when you don't allow the chance of exploring other strategies which are not locally the best. His population was too small and the mutation rate too low to fix this issue. You'd probably get better results by selecting a small group of snakes with equally distributed fitness.

[@wallflower5130]

At least he is trying a evolutional algorithmic approach from computational intelligence field. I totally missed that he forgot the bias node. Of course there are several ways to solve that snake game problem. His approach is not useless though.

[@bongbongdalazybum2894]

1:20 the snake was moving like my self-quarantine routine

[@ИванВасильев-ч9щ]

Я всё ждал, что нейронная сеть будет управлять змейкой по оптимальному и короткому пути, в том числе по диагонали! 😃 В конечном итоге я дождался другого, когда нейронная сеть будет проигрывать из-за столкновения змеи об саму себя. 😅

[@ВиталийВи-с6ы]

на самом деле это оптимальный вариант движения по кругу, т.к. змейка может быть ограничена только размером карты, движения по диагонали уменьшает свободную площадь от 10 до 50%.

[@АлеАле-ч2й]

привет от диванных РУвойск __ вот вот_ чето автор логику игры не допилил _ когда змейка заходит во внутринний круг то конец сразу __ хотя может лишние проверки.. а нужна была производительсть .. хотя... хотя...

[@emmettdja]

this isn't really a great way to train a network, but it does get better, just very slowly compared to using backpropageation and natural deduction. these would improve the learning rate as well as extend the scope of its intelligence.

[@The_Mavrik]

а где результат после 1000 и 1.000.000 генераций ? where 1000 and 1000000 gen ? why not? it is very interesting to know what will be the result

[@UberFiLL]

Долго такая прога делается?

[@ВиталийВи-с6ы]

@@UberFiLL на гитхабе лежат исходники в свободном доступе

[@АлеАле-ч2й]

привет от диванных РУвойск

[@spaceowl5957]

At the end it felt like the snake was synced up with the music and dancing along that was pretty groovy

[@dushkin_will_explain]

Is it neuroevolution for reinforcement learning? Great!

[@Adomas_B]

- what is my purpose? - your purpose is to play snake. - oh. Oh my god...

[@Kiri678]

Yeah, welcome to the club pal.

[@donkconklin4356]

What's with the AI's obsession with the top left corner?

[@Glaciace]

Donk Conklin probably that the origin is there so 0 was probably an easier mutation than some other point

[@vladislav6675]

Yes, I noticed it too and it led me to an interesting thought about our human behavior We often do the same in our lifes, choose the only one variant, which is not best and have a lot of alternatives I mean I knew that thing, but this visualization made it more obvious

[@ДмитрийВасильев-я6б]

It is an antient tradition

[@MybeautifulandamazingPrincess]

Because there's no reason for it to go somewhere else. Basically, Don't fix what isn't broken

[@MybeautifulandamazingPrincess]

When an intelligent being makes a decision it needs to consider the "cost/effectiveness" of the result that choice will entail. One of the factors to consider is risk, so if one is achieving success with a certain approach, it makes no sense and is not worth to change that approach to something new that will require a new learning curve and will result in risks, as opposed to something one is used to and is proficient with. It's a matter of logic

[@tjs200]

what software do you use to display the state of the network?

[@rokolczuk]

I think it's all done in Processing but not 100% sure

[@microgeen9357]

This made me genuinely happy, thnx for posting

[@romulus193]

Me at 11 pm: I must go sleep early. Me too at 1 am: neural networks learns how to play snake

[@krakowjr8574]

Still wondering how you code this kind of program. Do you have good ressources to learn it ?

[@Froggo9000]

You have inputs, code that decides how to use those inputs, and outputs. In this case there are 24 inputs, 2 x 18 code stuff, and 4 outputs, one for each direction of movement. Inside of the 2x18 part, there are weights that are put on each input, this will make one output more likely to be chosen. You then make something called a generation. A group of snakes with slightly different weights. The best of the generation is chosen to be the parent of another generation.Over time, with enough generations, a good AI will arise.

[@tantarudragos]

I think using some form of DeepRL coupled with CompVision could yield great results for Snake. Of course, for such a simple game you could skip the CV component, but I feel it'd be more fun that way. Also to avoid biasing, you could perhaps pick a random spot as a starting point.

[@gongjiaji2489]

fantastic result. how about making a tutorial video about it?

[@ueiwqoak]

yes please - or github the code or something that was really interesting

[@SimulationSeries]

Thank you so much for making this educational video! Well done! We are so grateful

[@MarcioSouza1]

At 3:17, in Gen 17, it hits the wall. If it "knows" that hitting the wall is "bad", why would it "decide" to not turn there? It's not like it's tail was too long, such that it was complicated. Any ideas?

[@klimenkor]

Thanks man! Your example is absolutely beautiful. Most AI/ML courses are missing this stuff. It should be taught before moving on with Tensorflow and other high level libraries

[@xiri00]

the snake consistantley modes clockwise.

[@daskraut]

the question is: can it play doom?

[@-poison8075]

Yeah

[@LazyMoka]

i just love that part when the 30th is synced with the music as is turning on walls

[@shubhamjain1328]

2:26 I train my dog everyday and that's the best it can get

[@danielr.]

Why can it only kill coming from the top?

[@JordanMetroidManiac]

Because the neuron that fires off when food is below the snake has the strongest impact on its next direction in which to move.

[@RetepAdam]

The apple will never see it coming.

[@MrDragonorp]

Cus it saw no reason to go another way.

[@sergiomarquina4553]

Wow that's insane, This A.I is seriously groundbreaking could you show me how you did it

[@meeemm]

It is like a growing baby

[@absurdengineering]

In such simple networks, the encoding of inputs can make all the difference. Representing distance in some sort of a grey or logarithmic code may be worth a try to speed things up :)

[@TheWitcher745]

Hello, amazing video, thoroughly enjoyable. I'm very interested in starting to program stuff like this, can you point me what direction I should go to start learning to write programs like these? I already have myself familiar with both neural networks and a few optimization algorithms but this program seems to be a mix of both of them., since it doesn't really have any training data and relies on generations and random behavior to train the neural network.

[@o_real_couto]

4:30 highscore not saveded

[@PsychoBackflip]

@Edvin Tran No need for that. You know what it meant.