Evolving Neural Networks Of Bipedal Creatures

[@amphibious8063]

I love how the first test of tall bipedals learned to jump rather than run

[@underpowerjet]

Yeah, pretty weird adaption.

[@Т1000-м1и]

4 years ago.....

[@underpowerjet]

Ah good times

[@PatrickHoodDaniel]

Is each test getting stuck in local minima? If so, it would be nice to have a mutation that tries to find another local (lower) minimum.

[@underpowerjet]

Most likely yea, local minima. I think I gave instructions to stop training after no progress was being made for ~3 generations in a row after the 13th generation. Looked decent enough haha.

[@PatrickHoodDaniel]

The work is great. Thanks for posting these videos!

[@underpowerjet]

The faster they run, the longer they live.

[@wedding_photography]

Why not let them run till generation 500 or even 5000? You should get much better results.

[@underpowerjet]

From my tests NEAT can get results 10x-30x faster than a conventional static multi-layered perceptron structure. However, you are right I could have just left it for a few hundred generation more and see how they evolve. Should have done it......didn't think about lol.

[@XanderLuciano]

I'm working with the NEAT algorith myself right now, what did you use? I read the research paper and am just messing with sharpNEAT right now seeing how it works and just want to see results. It's a pretty cool topic.

[@underpowerjet]

Wrote my own implementation of it in C# :D. The only source I used was the research paper.

[@XanderLuciano]

The One have you thought about posting it on GitHub with this example program? I'd love to see how you are keeping track of the gene history! I've been struggling to figure the best way to do that for the crossover evolution.

[@underpowerjet]

Xander Luciano Absolutely, here you go. The four main classes that represent the contents of the research paper. NEAConsultor.cs: Keeps track of new genes discovered in the world, and has a master genome of all discovered genes. pastebin.com/8eHp0JCc NEATNet.cs: Takes care of genome storage, genome mutation, crossover reproduction, phenotype activation. etc. pastebin.com/w1smqZ9L NEATGene.cs: The gene....obviously lol. Has in node, out node, weighted connection. pastebin.com/h3AUE9Nj NEATNode.cs: The node defined by node value and it's type. pastebin.com/WABH56dT I made it as compact as possible without having a million classes like with SharpNEAT or UnityNEAT. Some stuff like NEATPacket, or draw connections methods can just be ignored. They are not important to the function of the algorithm. There are still some changes I want to make, that I think will increase the performance even further.

[@Sanemikufan]

this music sounds like something from Naruto

[@underpowerjet]

Yep it's the Naruto training theme. Here ya go! kzbin.info/www/bejne/nqvCpaNujc2Yj7M

[@SchiwiM]

Those Creatures are working hard too ^^

[@nick_of_time3438]

I never realized how good the soundtrack was

[@wiertara1337]

They look cute af

[@underpowerjet]

ikr lol

[@Т1000-м1и]

50th

[@emiljanQ3]

I would love some commentary on the underlying mechanics in future videos. You got yourself a new sub! :)

[@underpowerjet]

Alright I shall :D.

[@mickvanbreukelen7445]

It would be cool if different 'spicies' would be together in 1 shot. So you can see the different styles progress right next to each other

[@underpowerjet]

Same colors are same species. It just doesn't look like it because i had the similarity factor lowered quite a bit lol.

[@adognamedsally]

Do you think there is any merit to running 20~30 different sets of these tests, then pitting the most successful networks which evolved different solutions against each other to find a solution that is the best among all possible solutions?

[@underpowerjet]

Yes and no. The more tests that are run the better. Different tests might come to a solution in different ways. And then pitting the best of the best against each other should lead to to some interesting evolution. Also there is a huge advantage in diverse breeding of networks in normal Multiplayer perceptrons. However for algorithms such as NEAT breeding two different networks from DIFFERENT SPECIES would result in a non-functional brain majority of the time. So im not sure how it would work with NEAT. I have never tried breeding networks from different species, since the NEAT's research-paper says it should be avoided. Thus, the conclusions I am left with is, this sort of competitions between different tests would only be good as long as they are not being crossbred or ONLY crossbred unless they have a similar species score.

[@adognamedsally]

Ahh, I see. I hadn't even considered that their brain structures might be so incompatible that they would just create mush. I also have never researched NEAT, so I wasn't aware of the particulars. Thank you! The video was very interesting, btw.

[@underpowerjet]

Isely Mills No problemo :D

[@jonreeve8183]

Much like real life, what you are hypothesizing would be like breeding a horse with a human. Both able to run, yet incompatible

[@underpowerjet]

Precisely.

[@nightmarechameleon7502]

you need to make more of this!!!

[@ciddim]

Would be awesome to see a video of that neural network programming

[@underpowerjet]

I posted the link to the code in the description of the video if you're interested. I might make a video on explaining the code in the future. But, it's very simple and it's probably easier to understand by reading the research paper.

[@ciddim]

The One I'm really a neophyte when it comes to AI, what paper we talking about ?

[@ciddim]

just saw it in the description so sorry >.

[@underpowerjet]

Dominic Leclerc Haha, its cool :D. The paper actually does an amazing job in explaining it. The first time I read it, I had very little machine learning/genetic algorithm experience. Had to read it a few time to finally understand it :P

[@DamianHallbauer]

i love to see how the good ones surge ahead. Would be cool to see any attempts to climb up and down hills or to surmount obstacles. .. The game we make, Kontrol, is played in the same context of 2d walking and running. The 2d walker works in a semi-automatic way, so the users can feel the effect his timing choices and his knowledge of which platforms are stable. did this over rough terrain

[@mdeb85]

Very cool. I'm just starting to try to understand these types of networks. I'm guessing the four outputs would be "flexing" of the joint motors. but I'm curious what all the input nodes would be

[@underpowerjet]

Unity has a component called WheelJoint2D. There are 10 input nodes: 1 body radian, 2 thigh joints radian with respect to the body, 2 leg joints radian with respect to the thighs, 4 touch detection sensors (-1 or 1) for legs and thighs, and 1 bias. There are 4 output nodes: New motor speeds for each joint.

[@mdeb85]

Ahhhh very cool, didn't think about having their position fed back in. So they are blind, but they can feel their limbs and have the inner ear balance and feel the ground.. thanks for the response

[@underpowerjet]

mdeb85 Yeah, otherwise it's like asking them to learn without being able to get their limbs information. In the human world that's called "paralysis" hahaha (not funny).

[@revimfadli4666]

The One what is the "bias" input for? and what makes it different from the bias present in each neuron?

[@underpowerjet]

Good question :D. Think of the formula of a line. Y = M*X + B. Where M is the slope, and B is the constant value. If you never use B (effectively making B = 0), then the line would ALWAYS pass through 0,0 coordinate, because there is no way to shift the the line up or down. It's stuck in one spot. But, suppose you where able to control that B value! Now you can draw a line anywhere you want. The bias in a neural network does the exact same thing! It gives the ability to shift the prediction left or right. It can also serve as the default value. If all the inputs are 0, then the bias helps give a non-zero output. So, to summarize, the job of a normal neuron is to learn some way of parsing in coming data and converting it into some useful output which can then be used by the agent OR another neuron. The job of a bias neuron is to provide a non-zero output OR just to help the normal neurons in shifting their predictions left or right. The role of a bias diminishes as the size of the neural network increases, because a larger neural network can make use of a larger polynomial function to learn data.

[@stenlan8932]

You have some great videos! You just earned my sub

[@underpowerjet]

Thank you! :)

[@eVITORIOe]

I think you should add a hidden layer. Once a teacher told me that 3 layers are enough to calculate a non-linear function.

[@underpowerjet]

The red neurons are the hidden neurons part of the hidden layer. This project was using an algorithm called NEAT. Where essentially the hidden neurons and neural connections are learnt through evolution over time. The non-linear decoding ability for a neural network comes from its use of various non-linear activation function such as Hyperbolic tangent or Sigmoid. And yes, having more layers and neurons, the easier it is to teach. However, in this case I wanted the agent to learn to do the task on it's own, so I let it evolve hidden connections on it's own :D.

[@davidm.johnston8994]

I want to applaud the choice of music man, I think I wouldn't have stayed if it wasn't for that. You should do some voice commenting if you want more views ;-)

[@underpowerjet]

ty ty. i do that in a few vids

[@alexandre3989]

At 4:35 it looks like me playing QWOP.

[@user-gp8fr1nd3w]

Did you implement NEAT from scratch? If not, how did you incorporate/connect it to Unity?

[@kim15742]

What are the few neurons at the bottom?

[@underpowerjet]

So a neural network has 3 different types of neurons Input, Hidden, and Output. The input and output are already predefined by me and cannot change. However, the hidden neurons can be learnt through evolution. So the neural network is contently modifying its neural connections by adding new hidden neurons and adding new connection to get the creature to move in a specific way to maximize distance traveled. Hidden neuron are where the real "brain" activity happens. That is where neural network gets the ability to make stronger decisions.

[@kim15742]

The One Oh, so it is not just adjusting weights?

[@underpowerjet]

Mein Kanal! Yep, it's adjusting weights, adding new connections, adding new nodes, AND disabling some connections. So suppose there was a connection from A-B. New neuron C was added. The connections between A-B get's disabled. And 2 new connections are added. A-C-B. The connection A-B has a chance of being reactivated in the future based on mutation parameters.

[@marcibaer7998]

Am I the only one who saw the creature that was actually running at 4:13? I was so disapointed that he didn't run the test for more generations.

[@underpowerjet]

As I remember it, that species was there for 10+ generation and didn't really get better. When i first saw it, I was like woah.....skills. But, if you watch closely it's very apparent that its brain structure is very unstable. Out of all the species that must have spawned, it was the only one that would make it far. Thus, we can conclude any tiny change to the brain structure of that creature would cause it to fall down. And this can be seen if you look closely, species of its kind just fall during the start of each generation. Coupled with the fact that it had a unstable brain structure which did not allow for mutation, and the fact that the spawning algorithm did not give its species many slots to breed. It was doomed to fail. :( D: :( D: :(

[@SuperNuclearBoss]

Like a take on the Evolution Simulator made by carykh

[@underpowerjet]

Actually based on this video kzbin.info/www/bejne/rpSsc6KGqrN8n9k My LSES project is based on Carykh's Evolv.io

[@BrrrIce]

This is amazingly cool. I'm only a freshman, so a lot of this goes over my head. Where would you recommend I start if I wanted to learn this? Also, what (simple-ish) program would you recommend for implementing it?

[@underpowerjet]

I would recommend just making fun games :). Unity3D engine is completely free and there are plenty of tutorials online teaching how to make simple games with it. Also look up some videos on Object Oriented programming, and learn C# (You can write code in C# for Unity). Start small, make some some simple games. Once you think you have a decent grasp on various programming concepts you can move onto more advanced programs. ----IF YOU are already a good programmer (Strong OO Skills, Know various languages etc)---- Then you can move straight on to learning neural network. How neural network feed-forward works: stevenmiller888.github.io/mind-how-to-build-a-neural-network/ Amazing video by Welch Labs on Feed-forward and Back-propagation kzbin.info/www/bejne/mKnIY4dji52LiNU kzbin.info/www/bejne/i3vafGmgdrino8k kzbin.info/www/bejne/a6aTm5R3aNZ0fc0 kzbin.info/www/bejne/fZ3Gn6uLodenm80 More mathematical ufldl.stanford.edu/tutorial/supervised/MultiLayerNeuralNetworks/ Two simple high-level videos on neural network by MIT kzbin.info/www/bejne/q4nXaaR8Z7-tnNE kzbin.info/www/bejne/jKOweXRprr2Sh6s Finally....Back-propagation. Which is probably the hardest concept to understand. If you don't understand back-propagation by the end of watching all the videos and studying, then it's fine. You can still build neural networks using libraries. But, it's good to some understanding of how Back-propagation works and why it's so important! Here are two videos that I watched when trying to learn it. kzbin.info/www/bejne/l4eslWuBgtiXi7c kzbin.info/www/bejne/sKHcnJl5abuhpq8 Good luck :)

[@zachgeorge5427]

Let me start of by saying this is amazing. The source code for the main network files are very intriguing so thanks for allowing us to study it. One thing however is that it is missing a NEATPacket class and I was wondering what exactly was included in that file and if maybe you could explain it or give a link to a pastebin of that code. I noticed that part of the NEATConsultor class as well as a bit of the NEATNet class contain references to a NEATPacket and it uses node_inputs, node_outputs, consultor_genome, genome, and has a Split method.

[@underpowerjet]

NEATPacket is just for database. Since i was storing NEATNet information in a database. NEATPacket parsed and converted JSON into information that could be used to create NEATNet.

[@zachgeorge5427]

The One oh makes sense. So the split method was just for parsing then. What values were in the data packet other than input nodes and output nodes?

[@underpowerjet]

Zach George First i used a JSON library to parse JSON. I used split string for parsing the genome.

[@nightmarechameleon7502]

can you release this to the public?

[@underpowerjet]

To public, or not to public. That is the question.

[@nightmarechameleon7502]

public, please!

[@DownTheBarrelOfaGun]

Awesome work! It would be awesome if you still have the unity project and you could share it :)

[@tomaselmejor123]

Good video dude, did you do these codes?

[@underpowerjet]

The program was written it in C#. Check the video description for the code. Let me know if you have any question :D

[@tomaselmejor123]

I know it was written in C++, but you did that?

[@underpowerjet]

Lol no, not C++, I said C#. C# is different from C++. C# is more object oriented, where as C++ is more pointer based. Unity3D does not allow C++ as a scripting language. And yes, I wrote the whole algorithm, which was based on a research paper called NEAT (Neuro-evolution of augmenting Topology).

[@tomaselmejor123]

Thanks about C++ and C# difference xD, do you have skype or something?

[@underpowerjet]

RyzoX Yes, I have skype. But I would prefer to use only to talk to close friends and for work. Sorry ;( ;( ;(. If you have ANY questions, I don't mind answering them here :D.

[@mmtf]

We should just stop walking and do the thing that they're doing. They're better at it :D

[@underpowerjet]

ikr, might as well learn some amazing method of running haha

[@CanMetan]

You guys must have joints of steel

[@underpowerjet]

You makin fun of my legs and running capabilities? How dare you!!

[@KukiolGAME]

6:14 the machine learned how to dance

[@underpowerjet]

Yeah haha

[@sebo641]

Congratulations! You must be a proud parent

[@underpowerjet]

Certainly.

[@wingit2296]

I want to test this code on my computer. What software does this code run on?

[@wingit2296]

Are you using Unity? I tried, but it said it was missing NEATPacket.

[@underpowerjet]

First, I just updated the links in the descriptions. Yes, this is using Unity and is written in C#. NEATPacket is for saving information into the database. That can be ignored! Just remove all methods or constructors that have NEATPacket in it. Also treat the those 4 classes as an API call. I highly recommenced reading the NEAT Research Paper before you try to accompany this classes into your code.

[@wingit2296]

Ahh... Thank you very much! I will do more than skim over the research paper this time.

[@Benjackalope]

I want to try building my own neural network sometime, but what if each species had a goal of fighting one another. In this case a species would be any neuron network having run an x amount of time. Imagine a neural network simulating & learning by fighting a separate case of itself for 10-30 sec, once network can get more intelligent about their technique, then even longer, unless that misses up a few things. I wonder how the unique fighting styles would interact with other neuron networks.

[@underpowerjet]

It's called a Neural network not Neuron Network :D. I'm sure it's possible for neural networks to learn to "fight". First, I would suggest learning the basics of neural network. Building from training a basic XOR neural network to a complex fighting neural network :).

[@Benjackalope]

XD saw my mistake, thx for pointing it out, fixed it. Thanks for the tip, I have been looking into the general logistics of neural networks, & kind of understand them. I will try what you suggest, & build something simplistic before adding the more complexities of combat. Do you happen to know any good unity tutorials for this kind of thing?

[@underpowerjet]

Yep, I got a list of tutorials and videos I watched when I was learning it :D. How neural network feed-forward works: stevenmiller888.github.io/mind-how-to-build-a-neural-network/ Amazing video by Welch Labs on Feed-forward and Back-propagation kzbin.info/www/bejne/mKnIY4dji52LiNU kzbin.info/www/bejne/i3vafGmgdrino8k kzbin.info/www/bejne/a6aTm5R3aNZ0fc0 kzbin.info/www/bejne/fZ3Gn6uLodenm80 More mathematical ufldl.stanford.edu/tutorial/supervised/MultiLayerNeuralNetworks/ Two simple high-level videos on neural network by MIT kzbin.info/www/bejne/q4nXaaR8Z7-tnNE kzbin.info/www/bejne/jKOweXRprr2Sh6s Finally....Back-propagation. Which is probably the hardest concept to understand. If you don't understand back-propagation by the end of watching all the videos and studying, then it's fine. You can still build neural networks using libraries. But, it's good to some understanding of how Back-propagation works and why it's so important! Here are two videos that I watched when trying to learn it. kzbin.info/www/bejne/l4eslWuBgtiXi7c kzbin.info/www/bejne/sKHcnJl5abuhpq8 Good luck :)

[@Benjackalope]

The One thank you very much, I will be sure to look into these & see what I can come up with! Who knows, I might even start a KZbin channel about my works, but we'll see!

[@underpowerjet]

Making videos keeps you motivated to complete your projects :D. Otherwise I stop half way and move on :P

[@AihornMac]

I wonder what is the background music

[@underpowerjet]

*wonders*

[@jzjzjzj]

What software do u use to do this?

[@underpowerjet]

Unity3D using C#

[@98alla]

how long does one generation last and how do you count it?

[@underpowerjet]

Max time I think I set it for was ~12 seconds. 12 seconds is the best case scenario. The agents take damage if their limbs are not moving. This force agents to develop brains over many generations to run forward with smooth movement of all limbs. You can see in the video during Generation 1, the agents die in like 2 seconds.

[@98alla]

ohhh now it makes sense but is it always 12 seconds like in general or it is until something dies or while it alive that it counts as one generation?? Thanks btw I'm a programmer in a way and have a long way to go but i am trying to understand this and at some point evolve it into AI, it my goal right now hahah. I know it does not mean anything to you but anyway hahaha just wanted to point it out for some reason hahaha :)

[@underpowerjet]

The generation ends when the last agent dies or 12 seconds is over. There is no point in continuing the generation if all agents are dead lol. 12 seconds is just an arbitrary number and can be changed at any time. Just keep working on it and eventually you'll be able to make things better than this in no time :D. I was in the same position as you a year ago. It's just a matter of time and effort.

[@98alla]

Thank you very much sir!!! :)))

[@AidenOcelot]

Ever thought of making an evolution program where you can make a creature. And then it figures out how to move?

[@underpowerjet]

Yeah, I have thought about it.

[@AidenOcelot]

Do you think it would be extremely hard to make?

[@underpowerjet]

Joints in Unity are annoying to deal with as it is. It can be done. it's just going to take a weekend worth of brain power. I might give it a shot this weekend since this problem has peaked me interested.

[@underpowerjet]

Here you go :D imgur.com/a/rFKr2 A little something I started working on since you asked me this question. I'll make a video on it in a week or so when I can release it. Thanks for the idea :D! I'll make sure to thank you in the video.

[@AidenOcelot]

:O that looks beautiful! I'll be eagerly waiting for your video!

[@PauloConstantino167]

I am mind blown!

[@underpowerjet]

Working on a real world version of that with servos right now :D. And thanks

[@PauloConstantino167]

How did you build this program? Is it Java? or did you use a type of software that lets you build these things an explore? It's amazing. Soon we will have almost exact copies of ourselves inside the computers being simulated, and this makes me wonder if we are already :D

[@underpowerjet]

***** Servos and a HC-06 bluethooth sheild are controlled with Arduino micro-controller, which uses C coding to set everything up. And yes, Java connects desktop's bluetooth to Arduino's bluethooth. And this way I can write programs to control servos any way I want. :)

[@mlink3918]

Life Finds A Way

[@underpowerjet]

That's true :D

[@gorimikk]

Where I can download this?

[@underpowerjet]

It was personal project of mine. I linked the code i wrote for NEAT algorithm in the description.

[@juiceex136]

3:35 The Bipedal *-Jumper-* *Hopper.*

[@underpowerjet]

How dare you attempt to rename my creation. Treason!

[@HerrLehmann0xcc]

3:44 What do you mean? That's what I do.

[@underpowerjet]

I see no other logical alternative. It must be true.

[@AlexanderBollbach]

whats the music?

[@underpowerjet]

Naruto Training Theme, enjoy :) kzbin.info/www/bejne/fXvOhWOFm7SrnK8

[@Sanemikufan]

I KNEW IT

[@underpowerjet]

Good guess!!!!

[@SinanAkkoyun]

Wow, please show me the code!

[@underpowerjet]

Description :). The neural network code.

[@acciaiomorti]

I was hoping to see them crossing legs to run

[@underpowerjet]

yes me too,

[@sidroksasalways]

Wow, brilliant :) I myself want to learn about Neural Networks, and hope I could get as good as you :) Can you take a moment, and help an absolute beginner in this field sir/ma'am?

[@underpowerjet]

Check out my recent video on Neural Network tutorial. If you have basic programming experience it should be easy for you to follow along and learn :). Let me know if you have any questions :D! And...bro........I am not a ma'am....

[@sidroksasalways]

Thanks a lot for replying sir :) Apart from videos, what all do you recommend ? How did you learn ML/Neural Networks? I usually do programming in python, watching your video :3

[@underpowerjet]

No jokes (serious). One day I woke up, decided I wanted to learn Neural Network. Typed into google "Neural Network Tutorial" and read every link on the first page and I had a decent idea of how they worked. It really IS NOT that hard. It's just a little concept that you have to get your mind across. Here are some links. Watch all this guy's videos on Neural Networks. He builds it up from the bare-bones: kzbin.info/www/bejne/mKnIY4dji52LiNU Nice visualization of feed-forward and back-propagation: stevenmiller888.github.io/mind-how-to-build-a-neural-network/ A bit mathematical explanation of NN's from Stanford when you're comfortable with neural network structure: ufldl.stanford.edu/tutorial/supervised/MultiLayerNeuralNetworks/

[@sidroksasalways]

Thanks a lot again :)

[@underpowerjet]

No problem! Let me know if you have questions.

[@asalindahl6320]

You are the one to do This?

[@underpowerjet]

Yes. Full sourcecode is in the description of my "Evolving Neural Networks NEAT With 3D Cars + Tutorial" video.

[@levistoermer4820]

Okay yeah this is cool and all. But why the naruto music tho?

[@underpowerjet]

Song represents persistence, training and dedication. And that has nothing to do with this video. Very good question, I ask that myself too sometimes.

[@Steel404]

I like the naruto training music xD

[@underpowerjet]

Had recently watched Naruto rasenshuriken training episode hahaha

[@tjreynolds685]

Third test, we got kangaroos

[@underpowerjet]

M8, i hate to break it to you. That ain't no kangaroo. We got a kangaroo? M8 what do you mean "we"? Where you there when "I" trained them. I quite remember pressing the test button and you where quite missing at that point M8. Jk, they do act like kangaroos a little :D. Joking they look nothing like kangaroos.

[@Disgustedorite]

Unleash the weird dogs

[@converserich]

watch at 1.5 speed

[@underpowerjet]

After watching it at that speed.......i feel like I should made the video at that speed haha

[@user-tm4uo9un2z]

they run like the big dog robot

[@underpowerjet]

Boston dynamics would be proud. ^_^

[@user-tm4uo9un2z]

i think so, do you think they ran a similar simulation to yours prior to engineering the big dog?

[@underpowerjet]

They probably ran a vastly more complicated simulation than this just to make a proof of concept lol.

[@user-tm4uo9un2z]

o_o

[@blazingwind4408]

liked because of naruto

[@underpowerjet]

:D

[@repeaterlanes8024]

TEACH ME

[@underpowerjet]

You have yet to be chosen young one.

[@repeaterlanes8024]

The One :

[@Michelle-wr6kp]

zombis legs

[@maniacalo2901]

Naruto!!!!!! Eeeyyyeeeaaah!!!