Neural Networks Explained from Scratch using Python

[@BotAcademyYT]

Please share this video if you know somebody whom it might help. Thanks :) edit: Some people correctly identified the 3Blue1Brown style of the video. That is because I am using the python library manim (created by 3Blue1Brown) for the animations. Link and more information in the description. Huge thanks for all the likes and comments so far. You guys are awesome!

[@walidbezoui]

WOW FIRST TIME TO KNOW HOW 3Blue1Brown Work Awesoome

[@jonathanrigby1186]

Can you plz help me with this .. I want a chess ai to teach me what it learnt kzbin.info/www/bejne/hZCxmJ-PprWoasU

[@spendyala]

Can you share your video manim code?

[@twanwolthaus]

Incredible video. Not because of your insight, but because how you use visuals to represent the information as digestible as possible.

[@danielcurrin3882]

If anyone can tell me what neural networks on my phone is please let me know. Bye the way, I've been hearing voices that say they are going to kill me, they started at Granville medical centre, and pretty much seems like everyone working for Granville county is in the know and I'm not knowing sht

[@blzahz7633]

I can't say anything that hasn't been said already: This video is golden. The visualization, explaining, everything is just so well done. Phenomenal work. I'm basically commenting just for the algo bump this video rightfully deserves.

[@hepengye4239]

As an ML beginner, I know how much effort and time is needed for such visualization of a program. I would like to give you a huge thumb! Thank you for the video.

[@xcessiveO_o]

a thumbs up you mean?

[@EagleMasterNews]

His thumb is now massive

[@Ibrahim-o3m7m]

I dont think he want no thumbs

[@abdulrafaynawaz1335]

He will be in pain if you will give him such a huge thumb... just give him a thumbs up

[@magitobaetanto5534]

You've just explained very clearly in a single video what others try to vaguely explain in series of dozens videos. Thank you. Fantastic job! Looking forward to more great videos from you.

[@ejkitchen]

FANTASTIC video. Doing Stanford's Coursera Deep Learning Specialization and they should be using your video to teach week 4. Much clearer and far better visualized. Clearly, you put great effort into this. And kudos to using 3Blue1Brown's manim lib. Excellent idea. I am going to put your video link in the course chat room.

[@danielcurrin3882]

If anyone can tell me what neural networks on my phone is please let me know. Bye the way, I've been hearing voices that say they are going to kill me, they started at Granville medical centre, and pretty much seems like everyone working for Granville county is in the know and I'm not knowing sht

[@craftydoeseverything]

I know I'm watching this 2 years after it was released but I really can't stress enough how helpful this is. I've seen heaps of videos explaining the math and heaps of videos explaining the code but this video really helped me to link the two together and demystify what is actually happening in both.

[@Transc3nder]

This is so interesting. I always wondered how a neural net works... but it's also good to remind ourselves that we're not as clever as we thought. I feel humbled knowing that there's some fierce minds out there working on these complicated problems.

[@danielcurrin3882]

If anyone can tell me what neural networks on my phone is please let me know. Bye the way, I've been hearing voices that say they are going to kill me, they started at Granville medical centre, and pretty much seems like everyone working for Granville county is in the know and I'm not knowing sht

[@GaithTalahmeh]

Welcome back dude! I have been waiting your comeback for so long Please dont go away this long next time :) Great editing and audio quality btw Reminds me of 3b1b

[@BotAcademyYT]

Thanks! I'll try uploading more consistently now that I've finished my Thesis :)

[@pythonbibye]

I can tell you put a lot of work into this. You deserve more views! (also commenting for algorithm)

[@ElNachoMacho]

This is the kind of video that I was looking for to get beyond the basics of ML and start gaining a better and deeper understanding. Thank you for putting the effort into making this great video.

[@pisoiorfan]

That's it! Comprehensive training code loop for a 1 hidden layer NN in just 20 lines. Thank you sir!

[@mici432]

Saw your post on Reddit. Thank you very much for the work you put in your videos. New subscriber.

[@cactus9277]

for those actually implementing something, note at 12:08 the values in the hidden layer change back to how they were pre sigmoid application

[@BotAcademyYT]

good point! Must have missed it when creating the video.

[@robertplavka6194]

yes but wasnt the value before sigmoid in the last cell 9 ? precisely I got something like 8.998 If I missed something please explain I want to know why is that

[@hridumdhital]

As someone beginning machine learning, this video was so useful to really getting a deep understanding on how neural networks work!

[@OrigamiCreeper]

Nice job with the explanation!!! I felt like I was watching a 3blue1brown video! A few notes: 1.)You should run through examples more often because that is one of the best ways to understand a concept. For example. you should have run through the algorithm for the cost function so people understand it intuitively. 2.)It would be nice if you went more in depth behind backpropagation and why it works. Things you did well: 1.)Nice job with the animations and how you simplified them for learning purposes, the diagrams would be much harder to understand if there was actually 784 input layers. 2.)I love the way you dissect the code line by line! I cant wait to see more videos by you I think this channel could get really big!

[@BotAcademyYT]

Thank you very much for the great feedback!

[@eldattackkrossa9886]

oh hell yeah :) just got yourself a new subscriber, support your small channels folks

[@angelo9915]

Amazing video! The explanation was very clear and I understood everything. Really hope you're gonna be posting more videos on neural networks.

[@gonecoastaltoo]

Such a great video -- high quality and easy to follow. Thanks. One typo in Additional Notes; (X,) + (1,) == (X, 1) -- this is shown correctly in the video, but in the Notes you show result as (1, X)

[@BotAcademyYT]

Thank you very much for pointing out the inconsistency. You're right, it is wrong in the description. I just corrected it.

[@mrmotion7942]

Love this so much. So organised and was really helpful. So glad you put the effort into the animation. Keep up the great work!

[@photorealm]

Excellent video and accompanying code. I just keep staring at the code, its art. And the naming convention with the legend is insightful, the comments tell the story like a first class narrator. Thank you for sharing this.

[@michaelbarry755]

Amazing video. Especially the matrix effect on the code in the first second. Love it.

[@vxqr2788]

Subscribed. We need more channels like this!

[@BlackSheeeper]

Glad to have you back :D

[@AVOWIRENEWS]

It's great to see content that helps demystify complex topics like neural networks, especially using a versatile language like Python! Understanding neural networks is so vital in today's tech-driven world, and Python is a fantastic tool for hands-on learning. It's amazing how such concepts, once considered highly specialized, are now accessible to a wider audience. This kind of knowledge-sharing really empowers more people to dive into the fascinating world of AI and machine learning! 🌟🐍💻

[@danielcurrin3882]

If anyone can tell me what neural networks on my phone is please let me know. Bye the way, I've been hearing voices that say they are going to kill me, they started at Granville medical centre, and pretty much seems like everyone working for Granville county is in the know and I'm not knowing sht

[@andrewfetterolf7042]

Well done, i couldnt ask for a better video, Germans make the best and most detailed educational videos here on youtube. The pupils of the world say thank you.

[@dexterroy]

Listen to the man, listen well. He is giving accurate and incredibly valuable knowledge and information that took me years to learn.

[@Lambertusjan]

Thanks for a very clear explanation. I was doing the same from scratch in python, but got stuck at dimensioning the weight matrices correctly, especially in this case with the 784 neuron input. Now i can check if this helps me to complete my own three layer implementation. 😅

[@bdhaliwal24]

Fantastic job with your explanation and and especially the animations. All of this really helped to connect the dots

[@doomcrest8941]

awesome video :) i did not know that you could use that trick for the mse 👍

[@brijeshlakhani4155]

This is really helpful for beginners!! Great work always appreciated bro!!

[@Darth_Zuko]

This is one of the best explained videos i've seen for this. great job! Hope this comment helps :)

[@jimbauer9508]

Great explanation - Thank you for making this!

[@eirikd1682]

Great Video! However, you say that "Mean Squared Error" is used as loss function and you also calculate it. However "o - l" (seemingly the derivative of the loss function) isn't the derivative of MSE. It's the derivative of Categorical Cross Entropy ( -np.sum(Y * np.log(output)), with Softmax before it). Anyways, keep up the great work :)

[@gustavgotthelf7117]

Best video to this kind of topic on the whole market. Very well done! 😀

[@mateborkesz7278]

Such an awesome video! Helped me a lot to understand neural networks. Thanks a bunch!

[@chrisogonas]

Superbly illustrated! Thanks for sharing.

[@kousalyamara8746]

The BEST video ever! Hats off to your efforts and a Big Big Thanks for imparting the knowledge to us. I will never forget the concept and ever. 😊

[@Lukas-qy2on]

This video is pretty great, although i had to pause and sketch along and keep referring to the code you showed, it definitely helped me understand better how to do it

[@v4dl45]

Thank you for this amazing video. I understand the huge effort in the animations and I am so grateful. I believe this is THE video for anyone trying to get into machine learning.

[@devadethan9234]

yes , finally I had found the golden channel thanks budd

[@malamals]

Very well explained. I really liked it. making noise for you. Please make such video to understand NLP in the same intuitive way. Thank you :)

[@napomokoetle]

Wow! Thanks you so much. You rock. Now looking forward to "Transformers Explained from Scratch using Python" ;)

[@susakshamjain1926]

Best video of ML so far i have seen.

[@kenilbhikadiya8073]

Great explanation and hats off to ur efforts for these visualisation!!! 🎉❤

[@DV-IT]

This video is perfect for beginners, thank u so much

[@ThootenTootinTabootin]

"does some magic." Great explanation. Thanks.

[@jonnythrive]

This was actually very good! Subscribed.

[@jordyvandertang2411]

hey this was a great into! Gave a good playing ground to experiment with in increasing the nodes of the hidden layer, changing the activation function and even adding an addition hidden layer to evaluate the effects/effectiveness! With more epochs could get it above 99% accuracy (on the training set, so might be overfitted, but hey_)

[@asfandiyar5829]

You create some amazing content. Really well explained.

[@neuralworknet]

12:40 why dont we use derivative of activation function for delta_o? But we used derivative of activation function for delta_h. Any answers???

[@hidoxy1]

I was confused about the same thing, did you figure it out?

[@itzblinkzy1728]

Amazing video I hope this gets more views.

[@Scronk03]

Thank you for this. Fantastic video.

[@EnglishRain]

Great content, subscribed!

[@kallattil]

Excellent content and illustration 🎉

[@dormetulo]

Amazing video really helpful!

[@saidhougga2023]

Amazing visualized explanation

[@maxstengl6344]

at 14:32 you use the updated weights (to the output layer) to calculate the hidden layer deltas. I never saw anyone doing it this way. Usually, the old weights are used and all weights are updated after backprop. I don't think it makes a large difference but I wonder if this is intentional or I am missing something.

[@FlyingUnosaur]

I also think this is a mistake. Andrew Ng emphasized that the weights must be updated after calculating the derivatives.

[@neuralworknet]

​@@FlyingUnosauryou are talking about the derivative of activation function right?

[@appliiblive]

Thank you so much for posting this comment, i was wondering why my model was losing accuracy with every epoch. With that little change my accuracy jumped from 20'000 / 60'000 to 56'000 / 60'000

[@Maxou]

Really nice video, keep doing those!!

[@Hide310122]

Such an amazing video with lots of visualization. But I don't think you can simplify delta_o to "o - l" with whatever mathematical tricks. It needs to be "(o - l) * (o * (1 - o))".

[@Kuratius]

I think you're right, but for some reason it seems to work anyway

[@neuralworknet]

yess i have been trying to understand this for weeks 🤯

[@quant-prep2843]

intuitive video on the whole planet, likewise can you come up with a brief explanation on NEAT algorithm as well ?

[@BotAcademyYT]

Thanks! I‘ll add it to my list. If more people request it or if I‘m out of video ideas, I‘ll do it :-)

[@quant-prep2843]

@@BotAcademyYT Nooo, we cant wait.... i shared this video across all discord servers, and most of em asked , wish this guy could make a video like this on NEAT or hyperNEAT. because there isnt much resources out there. Hope you will make it!

[@Michael-ty2uo]

The first minute of this video got myself asking who is this dude and does he make more videos explaining compicated topics in a simple way. pls do more

[@neliodiassantos]

Great work! thanks for the explication

[@VereskM]

Source text Excellent video. Best of the best ) i want to see more and slowly about backpropagation algorithm. It is most interesting moments.. maybe better to make the step by step slides?

[@johannesvartdal624]

This video feels like a 3Brown1Blue video, and I like it.

[@morty_squared]

Great video, really interesting!

[@HanzoHasashi-bv7rm]

Video Level: Overpowered!

[@LetsGoSomewhere87]

Making noise for you, good luck!

[@curtezyt1984]

you got a subscriber ❤

[@miguelhernandez3730]

Excellent video

[@Ibrahim-o3m7m]

How would you do the 50000 samples for training? Great video by the way!

[@onlineinformation5320]

As a neural network, I can confirm that we work like this

[@ziphy_6471]

Well , your brain is basically a complex neural network Plus, our body isn't us; our brain is us. We are just a complex meat neural network controlling a big fleshy, meaty and boney body.

[@nomnom8127]

Great video

[@ThePaintingpeter]

Fantastic video. I really appreciate the effort u_tubers put into great videos like this one.

[@yoctometric]

Algy comment right here, thanks for the wonderful video!

[@_Slach_]

11:31 What if the first output neuron wasn't the one with the highest value? Does that mean that the neural network classified the image incorrectly?

[@BotAcademyYT]

Exactly :)

[@0xxi1]

you are the man! My respect goes out to you

[@georgeseese]

What do neurons represent? You say "just numbers" @1:39. That may be true of the input layer (pixel values) and bias. But don't the neurons in other layers represent functions?

[@alizaka1467]

Sad to see a valid question get zero likes and replies for over 8 months

[@2wen98]

how could i split the data into training and testing data?

[@NikoKun]

What are you referring to when you talk about "defining the matrix from the right-layer to the left-layer" @ 2:35 ? I'm sure I'm just missing something obvious, but I can't seem to figure out what that's referring to in the code..

[@wawan_ikhwan]

Where is the batch size parameter sitting on?

[@rverm1000]

Thanks. I wonder if I could train it for other pictures?

[@oliverb.2083]

For running the code on Ubuntu 20.04 you need to do this: git clone github.com/Bot-Academy/NeuralNetworkFromScratch.git cd NeuralNetworkFromScratch sudo apt-get install python3 python-is-python3 python3-tk -y pip install --user poetry ~/.local/bin/poetry install ~/.local/bin/poetry run python nn.py

[@khalil_stuff]

but why we can't write :delta_o = (o-l)* (h * (1 - h)) 14:30

[@viktorvegh7842]

11:32 why are you checking for the highest value I dont understand when the highest is 0.67 its classified as 0 can you please explain? Like what this number has to be for example for input to be classified as 1

[@pu3zle]

Great content! I can't wait for more of this stuff

[@hynesie11]

for the first node in the hidden layer you added the bias node of 1, for the rest of the nodes in the hidden layer you multiplied the bias node of 1 ??

[@rejeanto6508]

I have a data set with the same size, how do I change the data set? I have tried to change it but failed. BTW thank you this video really helped me

[@cryptoknightatheaume6462]

awesome man. Could you please tell me how do you realise this neural animation? It's really nice

[@noone-du5qu]

bro how did u make the first layer know how much number of color scale should be used on the img

[@enriquefernandezaraujo3943]

TKU for this excellent video👌

[@jnaneswar1]

extremely thankful

[@tanvir-tonoy-programmer]

Hey do you use manim ? I was curious should I use manim or Aftereffect to visualise math concepts like those ???

[@cocoarecords]

Wow amazing

[@waterspray5743]

14:32 Where did the equation "delta_h = np.transpose(w_h_o) @ delta_o * h * (1 - h)" come from? I thought we were using gradient descent of the cost function?

[@BotAcademyYT]

Hey. When calculating the delta values for the hidden layers, we are using the derivate of the activation function. And the Derivate of the sigmoid function (h) is h * (1 - h) We just use the derivate of the cost function for delta_o which is already simplified to “o - l” here due to some mathematical tricks that can be used for the mean squared error cost function. I have to admit that this video is optimized to explain the concept and not the mathematics. To get a deep understanding of the mathematics, I’d recommend the free Stanford Machine Learning course by Andrew Mg on Coursera

[@waterspray5743]

@@BotAcademyYT Hello, I have a problem. Isn't the "derivative of sigmoid(h) wrt h" = "sigmoid(h) * (1 - sigmoid(h))" instead of "h * (1 - h)"?

[@BotAcademyYT]

@@waterspray5743 if you do sigmoid(h) you've applied sigmoid twice. Your statement gets correct if you swap 'h' for 'h_pre'. You can also see this in the code, pay special attention to line 24 and 25. Does it make sense now?

[@hoot999]

great video, thanks!

[@parthpatwari3174]

a neural network with only 2 for loops is underrated

[@ipsdon]

The forward propagation has 2 sigmoid functions for hidden and output layers, however the gradient calculation for output layer does not compute the sigmoid. The entire calculation only computes gradient of sigmoid on the hidden layer. Am I missing something ? But even that, it seems the network is able to compensate.

[@ramazandurmaz3012]

Yeah that got my attention too. Instead of MSE using categorical cross entropy for this is much more preferred but then it could be complicated. Let A2 be the activation of last layer for a single neuron and A1 the previous activation L = 1/2(Y_hot - A2)^2 and dL/dW = dL/dZ*dZ/dW. dL/dZ = dL/dA2*dA2/dZ Now dL/dA2 = (A2-Y_hot)*A2*(1-A2) Then calculate dZ/dW which is A1 Putting it all together: dL/dZ = (A2-Y)*A2*(1-A2) dL/dW = dL/dZ@A1^T So definitely the derivative of sigmoid needs to be included. The mathematical trick he mentioned works for sigmoid or softmax combined with cross entropy loss function. There delta simplifies to A-Y but in MSE it does not simplify so.

[@payola5000]

I really loved your video, it's so clearly explained. I have a kind of big question. What if you had a data frame where all the columns are related to each other, but there are different functions for certain parts of it? I'm trying to make a neutral network that is meant to understand the functional parts of proteins, in order to create new proteins

[@BotAcademyYT]

Thanks! That's a really hard one :D If there is some temporal difference in the data, you'd need a recurrent NN like an LSTM. But I think its not the case for proteins. So if they are related to each other I guess you'd flatten the data frame and use it as input. If the input dimension is too large, I think you need some other feature extraction technique before applying a NN. But I am just guessing here tbh. There might be better approaches directly for proeteins (there are surely some good papers out there because its a topic with quite some research behind it)

[@alangrant5278]

Gets even more tricky at 50 metres one handed - weak hand!

[@xXxxSharkLoverxXx]

I can't find tutorials for Java Script, so I am using this. How do I not use any external downloads, or with my own data that I gather later?