Self-driving car - No libraries - JavaScript course [Lecture 8]

⭐️LECTURE 8: OPTIMIZING NEURAL NETWORKS⭐️
We learn a way to optimize a neural network. First we will do trial and error in parallel. Then, I will teach how to use a genetic algorithm that does mutation. We will use local storage to save the best networks so that they are not lost when refreshing the page.
⭐️COURSE INTRODUCTION⭐️
Hi and welcome to the self-driving car (no libraries) JavaScript course.
You will learn to make a self-driving car simulation by implementing every component one by one. I will teach you how to implement the car driving mechanics, how to define the environment, how to simulate some sensors, how to detect collisions and how to make the car control itself using a neural network.
I will explain how artificial neural networks work, by comparing them with the real neural networks in our brain. I will teach you how to implement a neural network and how to visualize it so we can see it in action. You will also learn some optimization techniques to make our car smarter.
The entire system is quite complex, however, each component is relatively easy in and of itself. I made this course keeping in mind my high-school days back in Romania. So, the knowledge you get from school could be enough to follow along (depending on the curriculum).
We use JavaScript to implement this system and I will teach modern JavaScript techniques during this course, so, this course is perfect no matter if you want to become a software engineer or a machine learning specialist (like me - I have over 10 years research experience with machine learning).
⭐️MORE LINKS⭐️
NEAT: nn.cs.utexas.edu/downloads/pap...
JS NN Libraries: blog.logrocket.com/best-javas...
⭐️CODE⭐️
Github: github.com/gniziemazity/Self-... (8. Optimizing neural networks)
Use '7. Visualizing neural networks' if you plan to follow along!
My website: radufromfinland.com
⭐️ TABLE OF CONTENT ️⭐️
0:00 Intro
1:13 Parallelization
6:08 Fitness function
9:13 Saving networks in local storage
12:22 Other fitness functions
16:58 Mutation
22:42 A hard scenario
27:23 Other genetic algorithms

[@Radu]

Learned something today? :-) What? Full course playlist: kzbin.info/aero/PLB0Tybl0UNfYoJE7ZwsBQoDIG4YN9ptyY

[@JanderSilva01]

This is a unique content, thanks for that Radu.

[@Radu]

Glad you like it. Thanks for watching :-)

[@FarQuZeDesigns]

Bro you came to my rescue. I was so lost in all the mathematical fundamentals of training an AI and had no clue where to start. Thank you so much for making such valuable information available for free!!! Thank you so much!!!

[@Radu]

Glad to hear it helped!

[@aliph-null]

Nice, very educative and cool. Your Intro is so nostalgic, I like it.

[@Radu]

Hi! Thanks :-) I'm happy if you learned something and that you like the intro!

[@JunaidQadir-ls6pb]

Awesome! i managed to do exactly like this. Love it! Thanks

[@Radu]

Glad you liked it :-)

[@Infinium]

This is a really well produced and good quality video! Thank you :)

[@Radu]

Thank you! Check out the full course if haven't already :-)

[@ssikarim]

Original content. Thank you very much

[@Radu]

Thanks and thank you for watching :-)

[@marcyanus1430]

Been waiting whole week for this! Thank you Radu!

[@Radu]

Wow! Thanks for being patient :-)

[@aitorplaza2560]

As always, incredible lesson.

[@Radu]

Thank you, Aitor!

[@andy_lamax]

Been following through for a while now. Thanks for your brilliant explanations

[@Radu]

Thank you for watching :-)

[@georgelewis7334]

Awesome

[@Radu]

Thanks!

[@experimentnameuntitled22]

Very cool course. I followed this course and implemented this in python with pygame. It worked really well. I had some cars being more thin and the car learned to pass through the tiny gap between the lanes which is very cool.

[@Radu]

Awesome :-) I would be curious to see your python implementation. I have a couple: one with tkinter and one with pygame and they both are lagging slightly compared to the web implementation (not sure why).

[@experimentnameuntitled22]

@@Radu yes I have noticed as well, when running with 100 cars, it only runs at around half of the fps. I guess it's because python or those libraries are slower?

[@Radu]

@@experimentnameuntitled22 Ok. I thought the same... maybe JavaScript is more optimized when visualizing things.

[@experimentnameuntitled22]

@@Radu speaking of rendering, could you make something that explores with using WebGL or the new WebGPU? Maybe a simple 3D game or just visuals. I think it would be an interesting topic.

[@Radu]

@@experimentnameuntitled22 I was thinking to expand this to 3D, but not necessarily using webGL... just doing it from scratch :-) but let's see. Currently working on phase 2 which will be a 2D virtual world. Phase 3 will be more advanced neural networks and Phase 4 could be 3D.

[@elescritoriodejasso]

Se ve genial tu proyecto!!! saludos

[@Radu]

Thanks :-)

[@volodyslove]

The more clones the better 😁 i hope my laptop has enough amount of chakra😆

[@Radu]

:-))) Good Luck!

[@vedprakash-zz6hb]

Liked and shared it with all my whatsapp group 😊

[@Radu]

Wow! Really? Thanks! :-)

[@cianmcnally4694]

I’m one minute in and I love this channel already

[@Radu]

Glad to hear :-) Thanks for watching!

[@user-qe4nu5sz7n]

Watched, just wonderful! I would add ref to repl web view where I implemented the project so far but unfortunately posts with references are automatically banned. I wonder if adding more levels or making longer arrays on the levels can really improve existing neural network? I feel like not. A crossover algorithm implies that we identify somehow the best weights from both cars or just pick them randomly?

[@Radu]

Would be cool to see what you did. Maybe send it on my Discord? I will make more content related to the self-driving car soon :-) But neural networks... I will only teach more of that in January, I think.

[@user-qe4nu5sz7n]

Thanks for the answer, Radu ) I have done nothing outstanding, yet. Just contemplating the work of this simple neuron network and trying to get used to it. Though I would like to make games allowing player to train characters for various gameplays and using them in multiplayer contest so that models compete by themselves) I see that you have a new big neuron course, hope that it will gives me new insights ) Thank you, Radu for all your great work for the popularization of science @@Radu

[@Radu]

@@user-qe4nu5sz7n Thanks for watching! :-)

[@ScriptRaccoon]

This moment where your creatures (in this project: cars) start to evolve, as soon as we put in some mutation, is pure magic :). I really like the visualization. I have a few code suggestions. First of all, we have some for-loops where we didn't need the index at all, so a forEach loop would be easier to write. But more importantly, it is a good idea to refactor function definitions with many arguments in such a way that the meaning of the arguments become clear. For example, the line new Car(road.getLaneCenter(1),-100,30,50,"DUMMY",2) could be replaced by new Car({x : road.getLaneCenter(1), y : -100, width: 30, height : 50, controlType: "Dummy", maxSpeed: 2}). or even by new Car({position: {x : road.getLaneCenter(1), y : -100}, size: {x : 30, y : 50}, controlType: TYPES.DUMMY, maxSpeed: 2}). In the original line, the numbers appear to random numbers whose meaning we have to look up in the definition of the constructor function. In the refactored line, the meaning of every argument is absolutely clear. It is a bit longer to write, and the function definition needs to be updated (once), but in my experience it is really worth it. So basically we have just argument in the function, called options, and get everything from this object with suitable keys (say, via destructuring). Also, TYPES could be an enum in which DUMMY is an available key - this has the advantage that we get auto completion when typing TYPES. and will not misspell the string so easily. The car constructor is just one example in this project where this type of refactoring may be applied. Another example in this part was the mutation function whose function call could be refactored to mutate({network: car[i].brain, amount: 0.2}). By the way I learned all this (and more) from Fireship's great video "Code this, not that".

[@Radu]

Thanks again for all your tips. I remember seeing Fireship's video a while ago, but knowing about those tips and making them... 'part of my system' is a different story. I really like the idea to use one parameter and destructuring inside the function definition itself. Things are maybe clearer in that way even with just 2 arguments. But somehow I consider 2-3 as the threshold where it's ok to do either way... And then if the function 'grows' I would need to refactor quite a bit, which can be distracting (can happen in many places / files) so, I would need to plan ahead somehow. But again, everything you said is useful, and I'll try to use it in future projects (that I haven't recorded yet :-P) and I'm considering to make a live stream of this self-driving car project sometime in the autumn with all these code improvements.

[@Markard0]

I found a faster way to train the cars was to get the best brain with one car in traffic, then 3, then 5, and then finally 7. This allowed always for at least 1 mutated car to get by in 1-3 tries!

[@Radu]

Cool! :-)

[@diegocassinera]

Great video, but i would like to point out the the function to find the best car even do i looks very small code its very much inefficient and it would just be better implemented in a single loop rather than 3 loops over all the values. First we start with the spread operator which loops over all the cars, then pass that to the min function which also loops over all the values, and finally the find which depending on the order of the array may need to look at all the cars again. I would be faster to just loop over all the cars and keep track of the lowest value.

[@Radu]

Thanks! You are right :-) You can also do even better if you separate the cars in 2 arrays (damaged vs not damaged) and only search in the non-damaged one. I'm not sure why I chose that way of doing it... Maybe because it's common in some environments today (like React) to work like that, for various reasons... Or maybe because I didn't want to implement another function to use just here, actually, it could help when implementing the sensors too (the nearest touch is considered). But current code is not that inefficient, actually. Sure, if we have 1000 cars, it will do roughly 3000 calculations instead of 1000. But if you think about other parts of the app, like the functioning of the sensors, the number is much larger. 5 rays need to be compared against 7 traffic cars, each of them rectangles with 4 sides (not bothering with counting the road border checks). This happens for each of the 1000 cars, so 140,000 segment intersection calculations, which are even more demanding than going through the elements of an array like in those 3000 steps. A similar calculation can be done for collision assessment 4 sides of each of the 1000 cars compared compared against 4 sides of each traffic car, giving 2 orders of magnitude more steps than in deciding the best car. So, major speedup would come if optimizing these other things, maybe I will include a section on how to do this in Phase 2.

[@diegocassinera]

@@Radu Nothin wrong with the code, it works for the context it is in. I just wanted to take a note over the description of code and functional programing. When writing code, when it comes down to make a decision about a line of code should be, functional programing seems to be a simple and good choice. However, when you step back a bit with wider context, some of these solutions hinder efficiency by repeating operations. There is nothing wrong of doing a for loop if you get everything done in one sweep. Most times, more descriptive code achieves faster results, in exchange of more code to read. By the way, love your work and the way you explain things. A fan.

[@Radu]

Well, you were right that it could have been better :-) I'm also not a big fan of quick coding solutions like that, but I guess I got used to them from teaching react at work... At least what I did was still linear time complexity :-D I get many students doing it by first sorting the array (linearithmic time complexity) and then taking the first element (O(1)). I actually wrote a script to make a funny sketch about it, but didn't have the energy to record it this year. Maybe later in the autumn. On holiday now. Thanks for watching :-)

[@ahmedshalaby948]

Great As Always! How can I add some extra chaos to my surroundings to truly capture the essence of a third-world country? No one respects lanes here; it's like a free-for-all with overlapping traffic.

[@Radu]

There's a lot of chaos coming in Phase 2 of the course (already on the channel). You learn to build a virtual world with intersections, roundabouts, road markings, etc. The cars will go crazy there, at first :-)

[@mrJohnLocke2342]

Perfect, but how to generate random cars with random positions so i can test it better. i've tried but couldn't use a for loop or function inside the traffic array.

[@Radu]

Hi, not really easy to explain in a comment, but if you join the Discord linked on my channel banner, there is someone (devperate) who posted a solution for that in the general channel. Hope it helps.

[@mrJohnLocke2342]

@@Radu yeah thanks i kinda figured it out it's working well just gotta improve it.

[@berserks03]

24:08 Increase N to 2000, for me better results:)

[@Radu]

Cool :D

[@rmsv]

Ok, I thought you were going to discuss backpropagation / error correcting code.

[@Radu]

Not here... I may mention it in phase 3 of this course (currently still working on phase 2). But I don't understand it well enough to make an explainer that is different from others out there.

[@rmsv]

@@Radu OK, good luck with your project. I think it was also interesting to use a darwinist approach to neural networks evolution. It's also more funny to watch and experiment "growing" neural networks. But originally, I thought you were going to train your neural network using the KEYS car, trying to make your output layer copy your own moves.

[@Radu]

@@rmsv You describe imitation learning. I may do something like that in phase 3... But not sure... In phase 2 I plan to teach how to create a more complex virtual world (so the car can drive in a more 'real' setting). Then the plan is to use the more complex scenarios to explain more sophisticated AI concepts. Let's see...

[@rmsv]

@@Radu in Javascript?

[@Radu]

Yes. I also teach using Python at work, but on this channel I focus on JavaScript.

[@juheehong7445]

I'm still a little confused by what mutate() does. I see that it's changing weights and biases, but I still don't quite get what exactly the lerp here does/how exactly these weights and biases are being mutated. Also, what exactly is the "amount" used here-?

[@Radu]

Good question. I didn't describe it that well, I think. Mutate will change the value of a weight and bias from it's current value, towards a random value, depending on the 'amount', so... even if the random value is very different, if the amount is 10%, it will be still 10 times closer to the original value than to the random value. It's just a way to control the amount of change that happens in a given network. Having said that... it's not the only way you could implement such a function. For example, another version of mutate could be to take a random amount of weights and biases (say, 10%) and change these randomly, without touching the others at all. So, here, there wouldn't be a need for the lerp function. I haven't tested if this version is better (if it finds a good brain faster) but both of the strategies will work if repeated enough.

[@tatwadarshihota3467]

I am trying to get best brain by refreshing again and again but always it gets behind a dummy car plays like back and forth. Is there a better algorithm to mutate. Because I don't know how my mutation not working properly though I didn't skip a single part of the video and wrote the complete code by watching the video

[@Radu]

It's possible your mutation is too low. I would increase the value a bit and repeat multiple times. You can learn many things about how neural networks work from phase 3 of this course (the understanding AI playlist). You can also try implementing a system that automatically tries different mutation values, and lets the cars drive for, say... 10 seconds and then saves the best result. Just leave it running for a while and it should progress by itself. Hope this helps :-)

[@tatwadarshihota3467]

@@Radu Thanks for reply and your suggestion. I would try

[@SmilingRob]

Wow the payoff of following along this far has been great!

[@Radu]

Thanks, Robert! Really happy to hear!

[@3nt3_]

at around 7:00 why did you write cars.find(c => c.y == Math.min(...cars.map(c => c.y))) instead of just cars.sort((a, b) => b.y - a.y)[0]?

[@Radu]

Both would work... yours is quicker to write but time complexity is O(NlogN) mine is O(N).

[@3nt3_]

@@Radu math.min still has to map over all elements?

[@Radu]

@@3nt3_ yes, there are many linear time operations. The mapping, the min search and the find are each O(N). But O(3N) = O(N) as constants are avoided when talking about time complexity. It could be that sorting here is actually faster, because N is so small, but, in general, as N grows towards infinity O(N) is better than O(NlogN) and it's a bad habit to use sorting when all you need is to find the maximum. If you are really looking for the best implementation, write a traditional for loop and min search...

[@3nt3_]

@@Radu or use reduce

[@Radu]

@@3nt3_ yes, I think you can use it in some clever way too.

[@ivanovchinnikov1230]

Very disappointed with this video. I was waiting for it because I was curious how you would train a neural network. When I opened the video and saw the word "Mutations" in the timecodes, I thought, "Wow, genetic algorithms! Cool! Let's watch!" But no... just a RANDOM SEARCH! In this case, even mutations are not needed - just generate more completely random cars and sooner or later you will be lucky. In such a simple task, this method works, but it cannot be applied anywhere else!

[@Radu]

Hi, sorry to disappoint. With this course, I wanted to give an overview of how an entire system like this works. But to do that and keep it manageable, I kept each component simple as possible. I am planning a part 2 for this course, where I focus on the neural network, but maybe also on genetic algorithms, let's see. Few more comments: 1. Not 'just' a random search. It's random in the first part and guided in the second. 2. Sooner or later you will be lucky even with a network with a billion nodes and hidden layers (most likely later). 3. Small networks are used in many classification tasks where the number of features is small. They outnumber deep neural networks in real world applications, but the later get more attention because they are seen as a kind of 'true AI', or close to that anyway :-)

[@ivanovchinnikov1230]

@@Radu Hi and thanks for the answer. In any case, it was very interesting course. I found your channel a few days ago and found a lot of useful and interesting videos here. Thank you for your work! In my opinion, recording the second part of this course is a really great idea!

[@juheehong7445]

@@Radu wow I'd be excited for a part 2 of this course! do you know when it might come out?

[@Radu]

@@ivanovchinnikov1230 Glad you found it interesting, even if not exactly what you were looking for :-) I'll see about the second part when I return to work.

[@Radu]

@@juheehong7445 I'll get back to work in mid August. Will start planning it then, maybe it will be out this year.