Let's code a neural network in plain JavaScript Part 1

[@thomaslinssen1426]

Damn, im learning how to make an AI by Tony Stark himself.

[@JKCD]

Underrated comment 😂lol

[@sandman7155]

Says 'No libraries. beginner mindset, etc' First line of code: R = require('ramda')

[@DivjotSingh]

I love your videos. Even though they're longer and usually with less content per minutes, the very human nature of your videos makes the concept much more easy to grasp and is remembered for long. It's like we're sitting in a cafe and having a legit conversation about stuff to do with a keyboard and a screen. 💜

[@funfunfunction]

Thank you for noticing this! The human aspect of it is very much intentional and I don't spend a lot of time trying to condense the subject, as there are so many videos doing that already. We're taking it slow instead and talking through how it feels and muse about it here. :)

[@Eoliann]

Definitely LOVE that too

[@kalm42]

+1 Because liking this comment doesn’t do enough.

[@Jemmeh]

I really like this about your videos, I can watch them for hours without getting a headache because of the pacing. :D

[@smreason]

Divjot Singh up

[@Bhuvaneshbabu0712]

Observable makes it kinda hard to follow along on.

[@funfunfunction]

How so?

[@Bhuvaneshbabu0712]

Probably the way of writing bottom to up instead of top to bottom. I think I might get used to it eventually.

[@Bhuvaneshbabu0712]

And thanks a lot for the wonderful video.

[@re8nifle]

Really dig your profile pic my dude.

[@hatrick3117]

that intro will never gets old

[@robincEPtion]

absolutely! Never thought to skip forward

[@CodingByMyself]

This is MPJ with much POWER and motivation! Damn good! :) Im watching at the end of 2019 and im watching it with smile on my face.

[@Saiphes]

Great energy! I really think you have the ratio right: Two Funs for every Function.

[@chickenzwings]

This is going to be a great series! This has always been a topic that terrifies me but I think i'll give it a go now that I have you and Dan holding my hand through it!

[@mserrano]

Can’t wait for the “build your own Skynet” episode!

[@ninojovic]

I've been following both you and Shiffman for years now, this feels like favorite TV shows crossover...

[@aronkoffler]

Hell yeah, the very best channels I've ever watched. Yours and shiffmans!

[@codeblobsTV]

Hey MPJ great video - just a note that there is native javascript way of creating a "range" like you did here which is not actually a range per se as you change the values so the following code works as well ----> Array(200).fill(0).map(_ => rand(-1, 1))

[@funfunfunction]

Nice!!!

[@dimitarmarinov3584]

Yes you are right, when you just need something for the map to enumerate on this would also work: [...Array(200)].map( _ => rand(-1, 1))

[@codeblobsTV]

yeah thats even better :)

[@blackbird9854]

Well theres even better way... Array.from({length: 200}, () => rand(0,400)) And there are milion more ways you can do that ;p

[@grafgrantula6100]

@@blackbird9854 Your way is a more obscure to me than codeblobTV's, because I have to know that Array.from uses the length attribute of the object literal. It's interesting though, thanks for sharing!

[@MudassirZulfiqar]

I love the way you teach us problems. Very nice idea. Please continue this new series

[@patrickstival6179]

I find this is familiar and intuitive. Please continue the series

[@JordanShackelford]

I am using a neural network for a problem which requires that I get 3 outputs (x, y, and z coordinates) instead of 1 output and they should be actual coordinates in the range of 0 to 5000 for example, rather than a probability value between 0 and 1. Any ideas on how I could implement this?

[@bruhmode283]

how do you create ai that is based off a api and takes action based on the api and the variables or functions you set BTW im new to javascript kindof i know it but i lost all my knowledge on it. and really am trying to create a really awesome program.

[@johnjelatis2033]

Just ran into this series and shared with a friend or seven- best one I have seen, simple enough and no libraries are absolutely required, plus easily portable to languages like C because generating an array is easy to do (still don't understand why you used that require though, I followed along with a for loop and it was just as easy). Thanks for making this, I have subbed now :D -- Will edit typos because phones auto-carrot is a pain to deal with, sorry about that

[@johnjelatis2033]

john-jelatis.github.io/Neural-Names/ Thanks for this tutorial! Can’t thank you enough for explaining it enough for me to understand! My implementation would probably need a bit more training data, but it works seemingly well right now so :D

[@manualvarado2212]

Where do the rand function came from? ): I get an error when trying to use it and it does not seem to come from the ramda library. Observable is quite cool by the way!

[@funfunfunction]

I paste it in at one point in the video - you can find it in the observable notebook. It was written by an audience member during the Twitch stream of this video.

[@ZahidIrfan]

Its Math.rand() function being used not from ramda library. Also the rand function is written by him which takes high and low and returns a random number between those limits.

[@manualvarado2212]

Thank you MPJ and Zahid! I tried to implement my own, but i am not used to Observable so i did it wrong hahaha. I will do it again later.

[@manualvarado2212]

Could you please tell me if this is a good implementation of that function? const rand = (start, end) => { const decision = Math.random(); if (decision

[@MichielVermandel]

Can you / someone please explain why the guess() function calculates the sum as it does? How do you get to that formula? Is there some rule to create such function?

[@durchschnittlich]

The first minutes of this video were really motivating. It's uplifting to hear that a former Spotify dev doesn't know much about all the mathy things I thought will be essential for a university degree in software engineering

[@michaeladams4999]

where are you writing the html svg and declaring x_max and y_max? do we initializing x_max and y_max in the neural network file? this weird text editor is too new for me. Why is the radius the only thing displaying in chrome when trying to load html in chrome

[@GameFunHQ]

The intro alone deserves a thumbs up

[@Gol_D_Roger_The_Pirate_King]

I love how MPJ try to reason out how to build your own Nueral Network from scratch. TO really understand thing you must build them by yourself. There are some pitfalls but you can learn from it.

[@Harshavardhan-gd4eu]

I was waiting for this MPJ :) . Becoz of you i started learning JavaScript and now Machine Learning :)

[@MistaT44]

features should be the word that classifies something and should be distinct and useful. Excellent video once again MPJ!

[@renecouture3719]

Remember to add mini-batching to the training function. The training examples have to be randomly mixed up and re-used many times.

[@chrisdel1483]

I'm not using observable. What is rand? That doesn't work for me. It's undefined.

[@sarahyiskah3147]

Can you do a tutorial on making a simple app using the spotify api and react-native please? I've been trying to figure out how to use them together for a while but I can't find any tutorial for this...

[@mathgag555]

11:31 "[...] there! ... and then it didn't work, at all, it not whatsover, shit!"

[@thorleach6988]

Lol! Such a true and honest moment at 14:00 minutes. Made even funnier by the fact that I was just thinking exactly the same thing 🙂

[@mengyangchen8563]

Some of my colleagues hate Machine Learning and Neural network, they are frontend developers, but I feel that's the most interesting thing now, javascript developers now have the chance to get on the train since Tensorflow has support written in Javascript, and for Nodejs soon

[@estranhokonsta]

Nice topic. I hope you give us more of your journey. As for intuition and since, in this example, you are doing Logistic Regression (a straight line), I would like to give my two cents. You could consider explaining the concept in this manner or another approximation to it (I have not seen the future videos, but I think it is similar to your approach and I hope it help someone in any way): • There is 2 kind of Points (blue and red) in the 2d plain and one want to divide/classify them with a line. How can we achieve that? First let us consider how one can construct a line. • A simplified definition: a line is defined by a Slope and a Height. So we end with just two variables to discover. (Spooky math. Do not look at it: Slope * x + Height) • We will call the Slope as an Weight (w) and the Height as a Bias (b) • A Neuron (in this simplified case) has two algorithms. First the Forward Propagation, where one try a random line (random Weight and Bias) and compare it to a Point with a rule: is the Point red or blue and is the Point above or below the line. • This comparison will help us create a Loss value. A normal Loss function in this case (many others can be used) is simply a binary (not binary in a strict mathematical or computing definition) yes or no (1, 0) case (i.e. is it wrong?). The sum of the Loss for each Points will be called Cost (divided by the number of points tried by the neuron). • The second algorithm is called Back Propagation (Gradient Descent in this case) and is simply an iterative process where one will try step by step to minimize the Loss by adding or subtracting some values/gradients (math: a derivatives) to the Weight and the Bias. • In more general terms (just an approximate intuition in multi layers neural networks), one can see the process of a neural network as an iterative algorithm to construct multi-dimensions surfaces (manifolds) that divide the space in regions, so that the ‘Points’ in each regions have distinct characteristics. • One very important concept not mentioned in the above intuition is the Activation function. They are a way of adding non-linearity (curves and complexity) to the construction of the surface because just adding ‘lines’ to each other will only give ‘lines’ as a result. • The idea is that each neurons in a multi layers neural network will help construct this surface with a little more of a certain ‘curve’ form each other. These definitions and intuitions are not to be taken too seriously. For that, the only way is passing by the different theories and math. This field is in a constant evolution and grows. One point to consider: why are the matrixes (with libraries such as NumPy in Python) used instead of arrays? Mainly because of the possibility of exploiting the power of parallel computing of the CPU or the GPU which give tremendous performance gains.

[@yngve1993]

Great comment, I only have one nitpick, which is that Numpy does not use any parallelism (or rather it only uses one CPU core). The reason for its efficiency is solely from minimising cashe misses and using SIMD (Single instruction multiple data optimisation). This will, however, still give you an immense speedup in comparison with a naive implementation.

[@estranhokonsta]

Thanks for the reply. Re-reading my comment, i have also many nitpick on it and some of them more than nitpick :) But i will leave them for other people to contribute if they wish. It seem, to me, as more interesting, than editing the original comment. As for Numpy and the 'parallel computing', it does gives a wrong image of multi-threading and more. What i was aiming at, was the SIMD part that you mentioned and its vectorization power. Vectorization can be considered as parallel processing by the CPU. Much more could be said about Numpy but, i consider myself just as a layman on it...

[@shpleemcgert]

What a shame to discover your channel so long after you have left KZbin. Your approach to teaching is one of the best I’ve seen in all my years as a student.

[@slugger777]

How does literate programming sound with ramda. We have been using this quite a lot and it's been great for reading code almost like sentences.

[@OscarRieken]

I think you would be able to create the same type of collection with ``` Array.from({ length: 100}, _ => ({ x: rand(-1,1), y: rand(-1,1) })); ```

[@cintron3d]

Loving this! Can't wait for Monday!

[@francispires]

i love it, please continue this subject!

[@mathgag555]

Observable HQ seems to be a LOT useful for that kind of work, it reminds me of Jupyter Notebook (for Python) and RStudio (for R) that I used in the past. Really useful to see your data and charts at the tip of the hand :)

[@DanielRamBeats]

awesome I coded along the first part. Really great explanations. I've been studying math -> calc, lin algebra, geometry along with learning neuroscience to understand machine learning and I was missing this block that is like a bridge in terms of bringing them together! I noticed when you mentioned the guess function which uses the weights and being able to guess how wrong it was and then nudging itself slightly closer to the right value, this can be called to 'learning rate' or perhaps the 'gradient descent', or at least I think that is what it refers to? A larger learning rate will still end up wrong and a smaller learning rate will require the neural net significant amount of time before it is accurate, but a learning rate that itself is progressively and dynamically adjusting its weights can then set a 'state' of the brain that knows more accurately how to predict the data, at least I think so? I agree with you that the terms and the math language used to describe all this... it is very challenging because they are truly within their own domains of understanding and web development is often separate from them. Unless it's within your own neural network of understanding the terms of knowledge of the topics, the hurdle to learn ML practically can be incredibly difficult. I think the way you have approached it, is perfect and very helpful!

[@FPChris]

How many parts?

[@NyaowYeon]

Excited for the next episode!

[@lithiumyacht]

Awesome video - love observable - and what you have done on this. please do more machine learning stuff

[@slimhmidi6398]

Very interesting video . What do you think about realizing graph application exercice?

[@Xy-gx8ou]

You know Daniel Shiffman? Subbed.

[@tom-snively]

Awesome. Can't wait for next week.

[@ryuhaneda]

You, sir, rock. Thank you for posting this!

[@falconmick]

Are you looking forward to the “not using a js framework? Well then why rambda” questions?

[@funfunfunction]

Yeah, programming tends to attract people that a very literal and have a hard time interpreting intent from what is expressed. :) To be incredibly, overly clear, for people that cannot do this: I meant that we're not going to use machine learning libraries like tensorflow, or math libraries. I don't think it would add anything to the video if we wrote a range function.

[@falconmick]

Fun Fun Function I dno! Ranges and random numbers sound pretty mathy to me. Watch out the fun fun fun police are on their way!

[@funfunfunction]

Good point, I'll keep that in mind!

[@NilsWesthoff]

It was a little weird because you JUST stated you wouldn't use libraries, but I agree with your sentiment otherwise. Everybody understands what the random functions do, so it doesn't obscure the logic of the actual neural network.

[@falconmick]

Nils Westhoff NPM is javascripts standard library in a way.

[@calebprenger3928]

I'm curious, Why would you use ramda instead of something like...I dunno. Math.random()?

[@funfunfunction]

Huh? We’re not using Ramda to replace Math.random in the video?

[@calebprenger3928]

Sorry I misunderstood video.

[@thomashoffmann9349]

That feeling when you finally catch up to the latest episode and have to wait a whole week for the next one :(

[@anonymousannoyed385]

I hate that feeling

[@zyst6]

Thanks MPJ I'm really looking forward to next week! I would love to see a video (series?) on SVG at some point! It's one of those things I've been telling myself I'd learn for the last two years or so, and I have never gotten around to actually doing it! Or is there a particular resource in particular you could point to which you would recommend? Thanks again! Have a great day.

[@funfunfunction]

Honestly, I just started using it, it's so simple that it doesn't really need much reading unless you're doing very complicated things. developer.mozilla.org/en-US/docs/Web/SVG

[@zyst6]

Thanks! I'll check it out.

[@navroze92]

Yes Yes. ML lets do it. I believe you are the only person who can teach ML in a interesting way. I don't know why I am so excited but lets do it!!!! Also I believe you can collaborate with Dave since he is a proper engineer and must be knowing a shit ton of Maths. Also I would like to know if you would be diving into ML with Maths and Stats in the future

[@VladTrishch]

Hey MPJ, you are quite good at explaining the "why" part

[@funfunfunction]

Thank you so much!

[@mayankbhaisora2699]

Can you please make a neural network in plain JS not using observable?

[@lithiumyacht]

@mpj didn't see the link to the observable ??

[@funfunfunction]

You're not super good at looking

[@maxvlaarhoven]

Hey! I am an AI student, and I find your explanation really good! Like how you visualize the weigths to a brain state! Great metaphor! Neural networks are so interesting!

[@lilbahr]

An idea for some episodes: Take some product with source code and break it apart by going through it. It can even take forever to go through, I would watch it just for fun, how you do it.

[@UnchartedThoughtsMusic]

Wunderbar!

[@rockywright4237]

what is the weird html?

[@funfunfunction]

Its just normal html. Perhaps you’re new to JavaScript template literals?

[@rockywright4237]

I am. That's pretty cool

[@volimsir]

I was just thinking about getting into this. You read my mind!

[@rborissov]

Hello MPJ, What do you think about Pattern Matching? Is it good idea? Great videos!

[@shurabhrastogi8778]

Hey hi, recent interview, interviewer as me sum(3)(2), sum of 2 number through closure. Can u give me some more javascript functional programming tips.

[@funfunfunction]

Yes, watch my functional series. :)

[@samsteele5139]

I've done this whole software development thing completely backwards. Nailed the machine learning early on at university now battling to figure out how websites work every day ._.

[@AbhishekKumar-mq1tt]

Thank u for this awesome video

[@dmh20002]

I had just learned what a tagged template literal is right before seeing you use it with the html function. Coincidence? I think not.

[@funfunfunction]

fate

[@esplanode1027]

how many paper towels did it take to clean up the coffee

[@sinyolinuxozy]

thank you MPJ :), i owe you big then let me pay it by dedicating myself on learning more javascript. Thanks sir

[@marcocastellano2451]

rambda is cool, but it isn´t needed. const rndPoints = new Array(100).fill(Point(Math.random(-1), Math.random(1))) works just fine :D

[@darshanmehta7057]

Great job mpj . By the way you look more like rdj (Robert Downey Jr.) 's Tony stark

[@exxzxxe]

My god- I stumbled into the Barnum and Bailey clown circus!

[@JurajPecháč]

My comment to video kzbin.info/www/bejne/eJSbhnaVopyAlbs :You can use javascript observables instead of console ( similar to jupyter notebook).

[@kensyjolicoeur]

who is here for the first time 🤣🤣🤣🤣, i enjoy this intro so much

[@ganeshumashankar]

great video. so, when are getting started with Skynet?

[@loyaldoge7948]

10/10 intro

[@SublimeSimplicity]

If anything happens to MPJ this week, it's because the would-be creator of skynet learns AI from part 2 of this series.

[@rkosak237]

Greetings MPJ! on your videos move looks a bit unnatural - are you rendering out videos above 24fps?

[@funfunfunction]

It’s 50fps.

[@rkosak237]

I seldom sent feedback to creators, but I enjoy your content and if you would want to compare the difference between 24fps and e.g 60fps - here is link kzbin.info/www/bejne/iYG9iYVqqN2ZiMU Human eye recognizes 24fps as more natural - that's why in cinema we can see movies in this preset. Take care!

[@funfunfunction]

One of my biggest pet peeves: When people use "natural" when they actually mean "familiar". Nature has nothing to do with film production. In a couple of years, 60fps will feel "natural" and people will be complaining that 120 fps feels "unnatural". Also, if you believe that 24FPS is what is used in cinemas because it's natural to the human eye, you should do some reading on the subject.

[@tiennes2403]

such a cliffhanger now I have to wait a whole week. buhuhuh

[@foundev]

so its not plain javascript

[@koukiadem]

you got a new follower

[@RocketLR]

Pretty cool but is there a dark mode on that site? Its so damn white and bright I cant tell if im dying and seeing heaven or my eye sockets are getting fried xD

[@funfunfunction]

Programmers seem to have some interesting belief that dictates that white background is harmful for the eyes, which contradicts all research on the subject, which pretty consistently concludes that black text on white background is the least straining on the eyes. It seems to be very specific to programmers, I've never heard a writer say this, for example, and I'm fascinated as to why. What does your workspace look like? Is it well lit? Also, do you wear prescription eyeglasses/contacts? If so, is the prescription recent?

[@RocketLR]

Fun Fun Function I have no idea how the brightness affects overall health of the eyes. I just have a hard time looking at super white screens. Unless it's reaaaally bright around me. At work and home I have everything dark mode, EEVVEERYYTHING xD

[@KnThSelf2ThSelfBTrue]

I think one improvement you could make is the way you refer to classical coding versus machine learning coding. The way you talk about it here, it's like you're claiming that machine learning is not code, when truly it is. I think a more clear and precise distinction would be defining a function with a classic coding, versus approximating a classic custom function with ML coding.

[@dimitricharles9784]

Glad I am not the only one procrastinating about ML

[@pascovicidinmoscova]

This is fucking great A lot of us (or at least our kids, if we decide to have them) will be doing these kinds of things, we all have to start somehow

[@nickmarks6693]

I think you are great man

[@IgorHaraSerafim]

MPJ Do you have any Brazilian ancestry?

[@blasttrash]

the real stuff starts at 6:21

[@wcdeich4]

Is there a github repo for this little silly example?

[@coreywilson353]

finally someone who dosnt use libraries

[@zanzibarmystic]

When did Guy Fieri learn to code?

[@PierreSoubourou]

Shifmann does great zooming everything, please think of people watching on their phones

[@funfunfunction]

Zooming is an interesting idea, I'll see what I can do.

[@Gingercats69]

starts at 6:10

[@chen_eason]

Hi MPJ, what is your major in college?

[@funfunfunction]

Did not go to college. Year 10-12 of school was theatre.

[@chen_eason]

I think college is useless because full of bad professor who asking his student to do research for their own benefit.

[@theja63]

Christoph Waltz!!! Is that you?

[@andriann-rak]

I see Tony Stark in your face, dude

[@developmentwithroman]

With your coffee, you can make math better than neural network!

[@PhillipSenn]

A mathematician is a machine for turning coffee into theorems.

[@developmentwithroman]

Phillip Senn haha, exactly!

[@Fellc0de]

i want that light from you wall in the back D:

[@funfunfunction]

It's a nanoleaf aurora!

[@re8nifle]

Beginning of the video: "no libraries".... 6 minutes later, the first line of code: require('ramda') .......

[@IsfhanAhmed]

Observable make hard to understand

[@shubhamkamble68]

You look like Tony stark

[@Djzaamir]

yes yes yes, Neural Networks In JS