Teaching our neural network to think - Let's code a neural network in plain JavaScript Part 2

[@Kalmanheyn]

This is my new Monday morning "talk-show" to watch while I drink my coffee! Love the entertaining exposure to JS.

[@funfunfunction]

Thank you!

[@jacksonlenhartmusic]

I love your show! Always challenging us with something new. Your old videos remind me of how much I've learned since I started watching, your new videos consistently remind me I have much more to learn. Thank you!

[@funfunfunction]

Wow, what a nice comment, thank you!

[@ruimmvilela]

The sum number is not a blob, the idea behind the neuron is to take inputs and transform into one single output which in our case can only assume 2 outputs (-1, 1). So depending if the blob is positive or negative we'll know the output. The reason why we multiply might be just because it's easier to generate a number, that can immediately be classified into two possible groups. In our case we just need to check if it's positive or negative. If we would do another operation like adding the weight then, we would need to work with a scale which seems less intuitive. So technically you could use any other method as long as you could find a way too classify them into two possible outputs.

[@carlos.arenas]

I was kind of confused with your explanation at first, but it was pretty clear after the tests! Thank you a lot!!

[@DerIstDerBeste]

I think usually you would constrain the input and output values from -1 to 1. The input would be scaled (point.x/maximumWidth) and the output would be sig(sum).

[@AndrewSmithDev]

Whats the reason for doing this over what MPJ did?

[@DrempDK]

Andrew Smith Then the values are independent of the size, and you could reuse the AI in other scenarios, I think.

[@funfunfunction]

To be honest I'm not sure what Sogolumbo means, we are transformning the value to -1/1. As for reusability, we're working ourselves there. One thing that annoyed the crap out of me with most other neural network videos is that they magically started with the abstract/generalised version without explaining how they got there. It's the same reason why I use the plain objects and doing manual multiplication - we're going to get another case in there after this and then make the neural network more general.

[@cboisvert2]

2:08 "it's just some bloody number" - this hits the nail on the head of the power, and limits, of neural networks. A neural net takes inputs and make outputs based on simple calculations. We can train the network - that's the great thing - but the parameters and the computation "mean" nothing - that's the terrible thing. The result is, neural nets give results (a trained system that does what we want) but the results don't have a reason for being this way, other than "we trained it and the parameters are this now". Working out what the parameters "mean" once the network is trained, is one of the major problems of neural network study.

[@rohscx]

I’ve been waiting all week for this!

[@nazarm6215]

How are you able to assign variables inside the object trainedWeights at 13:51? What is that called? It's not a function. So its like an IIFE(Immediately Invoked Function)?

[@funfunfunction]

It's the way obserable blocks are declared in observable. You can think of it sort of like an IIFE that is re-executed every time another observable it depends on is changed. I have a video on observable here: kzbin.info/www/bejne/g561nquwp9dnos0 The pattern was actually extremely common in the generation of frameworks that preceded React, so it is very familiar to me. However, it's clear from the comments here that I need to make a detour into the computed observable pattern and how it's made.

[@dillonharless7190]

Did anyone else get tripped up on the syntax trainedWeights = { const example1 = ... ... } ??? Can someone explain how that works to me? Can't seem to get anything from Google. I'm building this in React, not Observable, if that helps.

[@chrisdel1483]

I don't get this either. I made it a function instead of an object but the result doesn't work. I'm just watching this now and doesn't look like many ppl reply this far out.

[@victorb9503]

some time ago while I was in college we had a mini course on something called genetic algorithms where you would want to calculate say for example the maximum of a mathematical function and you would start by running random values through it and based on the result and a fitness function you would compute new values from the old ones and rerun the cycle many times. as sessions went by the new values tended to be better than the old ones (you could say that they converge to at least one of the maximum points of the function). it's somewhat familiar with what happens here.

[@funfunfunction]

Yes! That is indeed very similar.

[@tekv2]

Kinda late to the party here, but to consolidate my understanding of the idea and summarize the steps: 1) We get an expected value from the randomly weighted points (the magical sum). 2) Instead of determining what team it falls into with a formula, we generate a training function that understands the error deviation with the formula. In other words, we aren't using the formula to determine the numbers, we are using it to determine the error. 3) By providing an error to the weight training (buffed BRUH!!!!) function, we see how far from weights.x weights.y we actually are... (the error ranging from 0:2) for example, if error = 0, then x = weights.x + (point.x * (0)), so x = weights.x if error = -2, then x = weights.x - point.x * 2 4) trainedWeights does this process with a few points, showing the error (training) of the last trainedWeight. This is where technically*, over a certain amount of iterations (points), the "correction" of the error should tend to 0. *The technically part is the one that remains unclear. Could you maybe place the actual "correction" into words, please?

[@mengyangchen8563]

"Gradient descend" may scare those who didn't have any experience on AI or Machine learning, your way of mentoring is good and more acceptable for many people afraid of Math and Data, cool

[@SHUBHAMSHARMA-zc1rw]

another step towards skynet,.......great job!!

[@armanb9778]

Hey MPJ! I've been a long time fan of funfunfunction as you bring the fun and function to coding. I watched your promises video for the billionth time recently and it's now coming together, but it got me wondering what else is out there for asynchronous code & functional programming. Currently I am a grad student and am considering learning some out there tools like Erlang or Clojure for example. I initially hesitated, but finally I have come to a realization: since I get to choose the tools I want to learn for research projects, this could be one of the best times for me to try some funky stuff, but there's always that worry I'll get out of touch with industry. Now that you're doing KZbin full time, do you feel the same way?

[@funfunfunction]

oh absolutely. Not sure what to do about it yet, honestly. :)

[@luanlmd]

Was the video long? Felt like it was too short...

[@xploit6589]

Still a tad bit confused lol... Gonna take me some time to get this down but good hope to see how i can implement this into some type of game hack..

[@germanaguirre9584]

this video din't have subtitles?

[@domski196]

So, the train function, the calculation which returns new trained weights (weight + point * error) is that something quite generic (where does it come from, what mathematical process perhaps), or it's something very specific to this problem? Love the video, thanks! :)

[@funfunfunction]

It's really just multiplication. I find this to be the tricky part about neural networks - they are deceptively simple and your brain tends to look for a more complicated explanation. It's not too important exactly how to to it. The purpose is to have some math/code that adjusts the weight "away" from the error, and we do that with some normal multiplication. It's not a process or anything specific, it's quite literally just multiplication.

[@sirfurness]

Well, the reason this helps to train the data is that it makes the weights more likely to create the correct result. The guess function multiplies the x and y coordinates by the corresponding weights and if the sum of that is greater than 0 it gives team 1 otherwise it gives team -1. The next thing to notice is that if the actual team was -1 but the guess was 1 then the error will be negative and so when we multiply that by the coordinate and add it to the weight, it will make the weight smaller, increasing the chance that the sum of the weights will output a number less than 0 and thus classify this point correctly. This is the same thing that happens if the actual team was 1 but the guess was -1 except instead of making the weights smaller, it makes them larger. You can see how this will gradually improve the weights.

[@domski196]

Right, so the point is to reduce error and it's kind of arbitrary and problem specific how exactly you do that right? Thanks

[@andresgutgon]

Sorry for the dumb question. But could you explain de mathematical logic behind weight + error formula? Why is this improving the AI?

[@funfunfunction]

I can try to illustrate it more in the following videos, but I'm unsure how I can make it clearer than I did in this video. I say it several times in the video: The math is just adjusting some magic numbers (it's brain) up and down depending on how wrong the brain guessed for a given value and in what direction it guessed wrong. I don't think I can say it more simply, but perhaps I can run the examples interactively to show how the math is done to give an intuitive sense of it.

[@patrickstival6179]

Please talk more about that in the next videos

[@ilmammourtada4538]

So at first, I thought you would use some library that does the AI, but it turned out even better by creating our own AI (so to speak). My question though, is are you going to introduce us to some libraries that help us in different scenarios (such as Image recognition, voice recognition, etc..)? For like you mentioned, it's a basic example that will help us understand the concept, but not actually practical.

[@funfunfunction]

Maybe, we'll see. I'll explore it and see if it's interesting. The libraries sometimes have the problem that it's basically no programming at all. It would be like using the the squarespace designer for a web development channel.

[@ilmammourtada4538]

Totally agree. But I'm just eager to see how we can apply more complex scenarios. If you, however, are planning to dive deeper into this topic in a similar way, that would be even much better. Many thanks any way.

[@funfunfunction]

I'm kind of thinking that we could perhaps find a finished tool that uses a neural network under the hood and play around with it, to give us a sense of where we are going. We'll see.

[@ilmammourtada4538]

That'd be lovely. Thanks!

[@funfunfunction]

Then again, there is also people asking about an explanation of how the math actually works, so I might also go the other way and do the math on paper. :)

[@ryanisvibing]

First. Btw, liking these videos, Mpj. Keep up the good work!

[@AbhishekKumar-mq1tt]

Thank u for this awesome video, I am waiting for next video

[@djsamke384]

Wow MPJ This video is cool. It actually simplifies the hidden layer of the neural network. I think the margin of error is acceptable because your want it to think like a human and not like GOD. Therefore never make your AI results precise, keep them experienced.

[@geraldfoushee]

Have you never made any vanilla JS classes.... For beginners?

[@funfunfunction]

FFF is targeted towards professional, junior developers (I imagine a person one year into their first job as the persona) so beginner JavaScript does not makes sense (which is what I assume you actually mean, because pretty much all JS I use is vanilla + I've done many pure JS videos). On DevTips, that we just recently taken over, we've considered it, but there is SO much of beginner JS videos on KZbin already and we're honestly not sure what we can add to those.

[@geraldfoushee]

Fun Fun Function thanks, Yes there are several beginner level tutorials on vanilla JS. Sometimes they show you more of how to do something than what to use it for. I have just been learning about objects and constructor functions... But seeing what to use them for and examples of what they can do. I am saying sometimes we don't learn why it's used or better than something else. I first thought variables was all the rage... Then came arrays... And now objects... I know how to make them but the reason and examples and explanations is what's lacking.. I will subscribe to you anyway.... You explain well

[@sinyolinuxozy]

indeed, there's tons of JS beginner videos spread over the KZbin already. But Sir, your way of teaching somehow enlightened me even though i'm pretty sure that i've already in intermediate level in the terms of programming. Again, i do believe if DevTips launching vanilla JS for beginner that'll be booming as you gained popularity over the JS forums. So please take this as a consideration. Regards, :)

[@ownagejuice1394]

thank God this is not a beginner JS show coz KZbin is fucking full of Hello World useless tutorials. leave that to other channels. We want ADVANCED!!

[@rolerino21]

Where's part 3???

[@funfunfunction]

I always release one video per week. :)

[@rolerino21]

Fun Fun Function I've been following in observable, it's a really cool tool! I work doing web apps with visualizations with d3 so it'll probably be very useful. I'll be waiting for the third video. keep up withth the good work :)

[@Treedboxcom]

Finally! :P

[@SantanaFPV]

Missing part 3 :(

[@funfunfunction]

Sorry, you'll have to stick it out til' next week. :) Thanks your your eagerness!

[@calebprenger3928]

I feel like using the platform you did to code this made the episode extremely confusing. please don't use it again.

[@funfunfunction]

What would you suggest?

[@calebprenger3928]

A normal editor would have sufficed. Visual Studio Code or Atom?

[@grafgrantula6100]

I like observables as a teaching platform very much! I get that you are unhappy about having to learn how a new tool behaves before you can get to actual content, however. Still, Observables are easy to grasp, just take 10 minutes and play around with them it's fun and useful :)

[@calebprenger3928]

@@grafgrantula6100 it's kind of the problem. At home and at work I have so many projects + things I Wana do , learning an obscure platform is not something I have time for. It's line learning webpack. Overly complicated and to many things to learn. I just use a platform called prepros

[@cboisvert2]

@@calebprenger3928 you were going fine until you named your own favourite thing for me to learn.

[@ThalesJ]

WTF?! Why do u have hair now? Hahhaha