Training A Neural Network From SCRATCH! (Part 3)

[@TheZazatv]

We've been waiting for this! Thank you sir, gotta try it :)

[@bigmike7112]

Oh yeah, was waiting for this. Thanks!

[@Nabuuug]

my dear lord he put out the part 3 video thank god I'm so happy

[@michael_pio]

Great and clear explanation ❤

[@jonda_mc]

Will you do tutorial on reinforcement learning (PPO)?

[@JohnnyCodes]

Yeah I am going to start working on getting more videos out soon and the current order of what I am thinking is back propagation, actor-critic. And then possibly PPO ( there may be something in between actor critic and PPO because I want to make sure I cover everything needed to understand PPO first )

[@trollchicken]

Will there be a new tutorial video? I need more detailed version of this video.

[@kiyasuihito]

Great tutorial. Really makes it simple to understand. One thing I don't understand is how the raycast/hit distances work as inputs to influence the creatures to favor directions where the closest food object is. 1) How is direction calculated and mapped to distance, 2) how does the current NN favor shorter distances over longer distances using its current algorithms?

[@theobakshi3705]

1) Not sure what you mean, multiple rays are being casted at different angles from the creature based on the parameters of CreateRaycasts() and when they hit food the distance array is updated with their hit distance 2) The neural network isnt programmed to "favor" anything and doesn't have any algorithms, its basically a glorified equation that takes in certain values and outputs certain values. At first the weights and biases are randomly chosen but the creatures lucky enough to have weights and biases that cause the network to "favor shorter distances" have a higher chance to reproduce and over time the population will be better and better at moving towards the closest food.

[@Doxestic]

Hello there nice video i really wanted something like this i have a question though first is the output is based on the finall number in the output layer right? for example if the output 1 is + it will go right and if output 2 is - it goes backwards right? the second thing is for training them how much should i change the weight ? like 10% with 20% base chance or what? and also if i use the same Brain with same weights will it be able to for example drive car but in different road and with more obstacles?

[@batzmatasanos3526]

Hi, first of thanks for the tutorial, really great one, I just have two questions, I experimented with the sim and created 2 other types of creatures, i made a list that tracked the best five of each type, but how can i save there brains in to a file so i can stop for a day? And the second question is how do load a brain?

[@JohnnyCodes]

Awesome! Excited to hear how the multiple creatures turn out. For the saving, this is actually from the self driving car code, in that project I added a plugin and two functions to the neural network script. The plugin in free on the unity asset store and is called quick save (assetstore.unity.com/packages/tools/integration/quick-save-107676) At the top of the script you need to add this: using CI.QuickSave; Then I made these 2 functions: public void Save() { QuickSaveWriter.Create("jumping-v1-Round-" + roundNum) .Write("Car-"+ carId, layers) .Commit(); } public void LoadRound(int roundToLoad) { QuickSaveReader.Create("jumping-v1-Round-" + roundToLoad ) .Read("Car-"+ carId, (r) => { layers = r; }); } Then you can call the save function whenever you want and it will save it to a file, since the evolution sim isn't round based maybe you can call them days instead of rounds. Just make sure to make the string the exact same when you want to save and load a specific session. You call the LoadRound on the creature instances neural network script and as you can see it is currently set up to save and load all the networks layers onto the instance of the script it gets called on. I believe you can save and load any object using quick save so if you also wanted to save the entire creature I think you could do that too with some modifications, but I haven't tried that yet. Hope this helps! Let me know if you have any other questions!

[@batzmatasanos3526]

@@JohnnyCodes thanks for your amazing quick response, hope i may ask you another question, that's a simple one (i think) how can i change the length of the rays that is casting?

[@dingus42]

@@batzmatasanos3526 mate idk if it's too late and you already solved it - but in case anyone has the same question it's basically just the viewDistance variable in the ray casting code

[@DigitalCanineGames_]

HI sir, why are we turning off off the movement script at 3:55, wouldn't it destroy that instance's movement script anyways?

[@JohnnyCodes]

that was to make them stop moving so I can make them flip over when they die. Otherwise they would flip over but keep moving. The destroy call has the 3 as an argument which means it will be destroyed in 3 seconds. So for 3 seconds the creature will still exist but it will flip over and stop moving during those 3 seconds

[@DigitalCanineGames_]

@@JohnnyCodes Oh ok, thankyou sir:)

[@natebrett9782]

This is a stupid question... In the energy management section you have this " elapsed = elapsed % 1f; " could you explain what it does? why do you not use " elapsed = 0f ; " I understand what the function overall does but I don't know what the % sign does here?

[@dingus42]

% is modulo division, basically it takes the remainder so e.g. 2.25 % 1 = 0.25 or 9 % 2 = 1. in this case this allows us to take the actual elapsed time and continue on from there. For example if the timer counted 1.04 seconds, this "resets" it using % to 0.04 seconds, instead of resetting it to zero, where we would lose the 0.04 that we counted

[@akotoy]

is it possible that your sim speed controller some how vbrakes unity's time thing because i added it it and got weird issues evan when i take them out. im not 100% sure if the simsppeed controller is the issue

[@JohnnyCodes]

The sim speed controller will probably cause issues if you aren’t using fixedUpdate for pretty much everything. Because when you speed it up the frame rate changes a lot and normal update only updates the physics based on the frame rate while fixed updates it based on in game time. It makes for a slower simulation because it doesn’t skip frames like update but it is more accurate too. There may be a better way to change the game speed but that is the way I found to do it.

[@lanha3385]

I want to add predator, how do I do this

[@akotoy]

I'm getting an error on this line, nn.MutateNetwork(mutationAmount, mutationChance); it says object reference not set to instance of object.

[@JohnnyCodes]

Are you doing that in the update or start. I think I had to do the mutate in the update function because the start would run it too soon and give a similar error

[@akotoy]

@@JohnnyCodes I have fixed the error but have a new question. I am doing a car driving AI but following this tutorial and everything works except I want to copy the neural network from the best preforming agent and mutate it and I don't know how to copy the neural network from the best agent to the instance's neural networks. I am just using this: GameObject child = Instantiate(carAgent, new Vector3((float)x, 49.0f, (float)z), agentRotation); child.GetComponent().layers = GetComponent().copyLayers(); and don't really know what it does.(it's from your code)

[@JohnnyCodes]

In that case you will want to switch child with the other car you want to copy the network to. Since you probably want to copy the network to more than one car you will need to loop over all the other cars somehow and then copy the layers of the best car to the other cars. This might be better to do in a different location than on the car script itself. Probably wherever you are figuring out which car did the best. That line inside the loop would look something like this: otherCar[i].GetComponent().layers = bestCar.GetComponent().copyLayers(); Then you will need to mutate all of the otherCars too so right after the copy line you could call the mutate network function inside the same loop.

[@akotoy]

@@JohnnyCodes thanks so much, I instantiate the child in a loop already, so all I need to do is set the child to the neural network of the best car. This was the line of could I was trying to figure out. Thanks and great video!

[@Coolae450]

At line 61,13 it says the name layer does not exist in the current context :/ and when I downloaded the script from git hub, it showed the same error can someone help please. thx

[@JohnnyCodes]

Sorry about that it should say layers in that script(just updated it on GitHub). But I would recommend looking at the new link in the description of this video, it has all the scripts instead of just the NN script.

[@Coolae450]

@@JohnnyCodes ok thx

[@cultistsincorporated]

I'm having some index out of range errors within the forward pass, I made sure that my input set up was correct and I can't find any other errors. I'm doing this in unity 2D though, maybe I need to reformat my raycasts?

[@JohnnyCodes]

Hmm strange we’re you able fix the issue yet? That can happen if the shape of your network isn’t set right. Its possible that maybe you don’t have the right number of inputs and outputs to the network.

[@cultistsincorporated]

@@JohnnyCodes It looks to be set up correctly, 9 inputs for the 9 raycasts and 2 outputs to change index values on an array that handles movement. The issue may be from how I added the function to create raycasts. Just adding the function was returning an error that distances did not exist in the current context. The way I fixed that initial error was to define distances at the start of the script class with public float[] distances = new float[9]; .Should I move that function to separate script on the object and call the function each time data is being input to the network? Or maybe call the create raycasts function at the start of the function that adds inputs to the network? I understand these may be novice questions, I've only starting learning neural networks recently

[@cultistsincorporated]

I've been able to change the functions to my second proposed solution, but the index out of bounds error is still occurring on the line that finds the sum of weights * inputs where nodeArray[i] += weightsArray[i,j] * inputsArray[j]; Maybe I made a typo that I can't find, I'll leave a new comment if I encounter new errors or find a solution

[@cultistsincorporated]

So I did end up finding a typo, in the second for loop of the forward pass there was somehow an i in place of the j < n_inputs that i didn't notice from my typface making them so similar at a glance. Everything is running correctly, except the outputs are all 0 and my creatures. Which for the movement array to be [0,0] means they're stationary and not turning. is there a way for the neural network to default an output of [1,0] or [1,1]? Those arrays would be moving straight and making a left turn while moving respectively. Both indexes can be 0, 1 or 2 and I'm not sure how to implement that to the network so that my creatures don't starve before they reproduce. I'm going to run through some more options and leave what I find here in case anyone else is similarly stuck

[@JohnnyCodes]

@@cultistsincorporated Did you mutate the first batch of creatures? I put an if statement to mutate every creature at least once no matter how it was created (either being spawned in or being a child)

[@cricketeer606]

Severity Code Description Project File Line Suppression State Error CS0246 The type or namespace name 'ObjectTracker' could not be found (are you missing a using directive or an assembly reference?) Assembly-CSharp C:\Users icor\Unity Games\Create with Code\Neural Networking\Assets\Scripts\Simulation\Movement.cs 21 Active

[@AgentCryo]

problem is my ai seams to be all the same when I spawn multiple at once to speed the training proccess up

[@JohnnyCodes]

I had this issue too and I had to make sure I was making a deep copy of the weights and biases and not a shallow copy. If you do just the normal copy they will be shallow which means the values are all linked together so if you mutate one network it mutates all of them.

[@AgentCryo]

@@JohnnyCodes Yeah I had the deep copy, but it still seamed to have the same weights and biases. But then I has to make a new layers each time. Do you have a discord to talk overthis?