Backpropagation explained | Part 1 - The intuition

[@danluba]

This video deserves to be on television at dinner time.

[@parthasarathybalaraman1800]

I have just completed watching all the 27 videos in the list. All are excellent videos!! Thanks for uploading. Please continue the good work!! Thanks!

[@deeplizard]

Hey Parthasarathy - That's great to hear that you've gone through the entire playlist! I'm glad you're enjoying them!

[@maedehzarvandi3773]

your videos are actually great in expressing the concepts in possibly quickest time . i havnt seen your other playlists yet. but this one really helped me so much.

[@husseinwassouf2743]

this course is so nice you are simplifying complex ideas in a really nice way, I think this series should be binge watched like a netflix series to refresh your memory on all these concepts XD

[@justchill99902]

There is no limit to the awesomeness of her explanation!

[@luna-kr3zc]

there's a whole team behind this she is just the voice (but she does a great job at it)

[@deeplizard]

Backpropagation explained | Part 1 - The intuition kzbin.info/www/bejne/jnaWnKWcaKiEotU Backpropagation explained | Part 2 - The mathematical notation kzbin.info/www/bejne/aJ62qqaIrZJkmZI Backpropagation explained | Part 3 - Mathematical observations kzbin.info/www/bejne/fWbFZZ2Id7CBrtk Backpropagation explained | Part 4 - Calculating the gradient kzbin.info/www/bejne/kKOYp5x3j6yhmqc Backpropagation explained | Part 5 - What puts the “back” in backprop? kzbin.info/www/bejne/rnTPfJKVeNaNpLM Note, at 7:44, I misspoke when I stated that the updated values we get for the weights are the the corresponding derivatives of the loss function with respect to each weight. Actually, the updated values themselves are *not* the derivatives. Rather, after calculating the derivatives, the weights are updated to their new values, which are calculated *using* the derivatives we obtain. This process of updating the weights is covered in more detail in the following video. This particular detail is mentioned at 1:26: kzbin.info/www/bejne/lX-YnKOJgqmZatEm26s Machine Learning / Deep Learning Fundamentals playlist: kzbin.info/aero/PLZbbT5o_s2xq7LwI2y8_QtvuXZedL6tQU Keras Machine Learning / Deep Learning Tutorial playlist: kzbin.info/aero/PLZbbT5o_s2xrwRnXk_yCPtnqqo4_u2YGL

[@GirlKnowsTech]

00:00 Intro 01:23 Recap stochastic gradient descent (SGD) 04:42 What is backpropagation, the intuition 08:28 Summary

[@deeplizard]

Added to the description. Thanks so much!

[@RaghavendraBoralli]

Yes i want to know on the math behind backprop

[@kareemjeiroudi1964]

Me too!

[@piyushchauhan343]

Jan ke kia karo ge

[@zw9423]

Omg Thank You! All these books make it so complex and include a bunch of unnecessary equations before a noob like me can actually understand the process. I am interested in the math as well but first I have to know the concept, otherwise I often find myself understanding these complex details and be like, why exactly are we doing this? This video actually makes sense and now I can dive deeper into it. Nice animations too. Props to you guys

[@deeplizard]

You're welcome, Zi! Glad you found it helpful. The math explanations are in the following videos of this playlist.

[@hkrishnan91]

Bless you for making the world a better place. Keep up the good work!

[@ThePRINCEBARPAGA]

I cannot thank you enough. I don't know how many videos I watched to clear this concept, nothing helped but your video. You explained it in a very detailed and clear way. Thank You so much!

[@nicholasscotto3712]

after watching sooooooo many videos on this topic yours was EASILY the most helpful by far. It is accurate, simple, and just all around perfect. the image is very simple. I love your voice and the explanation is so good.

[@RohanPaul-AI]

Just can not THANK YOU enough for this Greatest of videos on the topic. Just Brilliant.

[@tymothylim6550]

Thank you very much for this video! It was fantastic that I could finally understand how SGD uses back propagation to calculate the gradients to minus off!

[@ismailelabbassi7150]

YES YES YES YES YES YES WE NEED MATH TOPICS

[@mohamedmahdy969]

I already know the math behind back propagation; however, I will watch your videos in order to see how you are going to present it. Your way of giving the information is awesome. I want to see how it will work with the math complexity.

[@deeplizard]

Thanks, Mohamed! Let me know your thoughts after you finish the following math videos.

[@mohamedmahdy969]

@@deeplizard To be honest, as I mentioned earlier, I already know the math behind the back propagation, yet your videos were a good refresh to me. You used the same mathematical notation and the same methodology of my teacher. he started with the last hidden layer; after that, he generalized to any hidden layer. Thanks a lot for your videos.I already finished the math videos and I will finish this list today. In the future, I am going to watch tensorFlow.js series

[@deeplizard]

Great to hear, Mohamed! Thanks for letting me know. Would love to hear how your progression is going in the TensorFlow.js series as well once you start!

[@yzyangliang]

This is an amazing video and I understand the deep learning instantly!

[@cedricrogers8102]

This was a very clear explanation of what happens during back propagation. The next step is to provide the math. Thank you for your effort. Great Job.

[@deeplizard]

Thanks, cedric! The full math for backprop is covered in the videos following this one! :) They are currently #28 - 31 in this playlist: kzbin.info/aero/PLZbbT5o_s2xq7LwI2y8_QtvuXZedL6tQU

[@cedricrogers8102]

deeplizard I will watch them thanks again

[@deepaksingh9318]

Yess.. Adding a specific Playlist with Maths behind all the functionality with an example (Like uh showed for max pooling which was visible to see hoe things are working with input) would be really great.. Bur still appreciate your efforts in helping students by such a great videos..

[@deeplizard]

Thanks, deepak. The next four videos in this playlist show the full math behind backpropagation. Let me know how it goes as you progress through them!

[@qusayhamad7243]

thank you very much for this clear and helpful explanation. Words fail to express my gratitude.

[@javierCi]

Thanks you very much for making this great videos. And I want to know more about maths

[@paragjp]

Hi, Thanks for super clean way to explaining basic concepts. Keep it up. Request you for following video series 1. Complete Maths and Stats for ML/Deep Learning. 2. Pl have additional series on calculus on backpropogation. 3. Make complete new series on ML/Deep Learning Practical projects rather than Housing Price Prediction, Titanic, iris, Hand written digits ....etc

[@MsStoCa]

Hey! What's going on everyone? :D Great content!

[@TheMeltone1]

You are truly a fantastic teacher :)

[@rajuthapa9005]

yes math too please.

[@DM-py7pj]

These are excellent videos. I do worry about biases not being discussed in later videos as these are also being updated.

[@dan7582]

At 8:00 you say that the update values of the weights are equal to their derivatives (gradient). But since we are talking about a loss function, they should be equal to de NEGATIVE gradient instead?

[@flamboyanz]

Thank you so much for this! Its concise and well explained.

[@deeplizard]

You're very welcome, Parvez!

[@travel7517]

great..explaination...simple and effective

[@robertoooooooooo]

Amazing, love your videos, will recommend it to all my friends

[@deeplizard]

Thank you, Robert!

[@faraazmohammed3693]

please make a series on videos explaining math behind machine learning and deep learning in particular. Thanks for Amazing videos.

[@deeplizard]

Hey Faraaz - You're welcome! Did you see the next four videos in the playlist that cover the full math for backprop?

[@richarda1630]

such an awesome super simple explanation! and how you bring up the math, it's almost like a teaser :) now I want to see the Math! =)

[@1sankey2]

Great video I wanna see the math. Thank you for uploading this one.

[@deeplizard]

Great :D It's in the following episodes. Starting with this one: deeplizard.com/learn/video/2mSysRx-1c0

[@Arcaerus]

I would greatly appreciate if you could make a series on how to take the derivative of a bunch of layers! Thank you for these videos!

[@samaryadav7208]

Wow that was exciting Hi. Thanks for sharing these videos. And Also we want to see the maths.

[@fahimmahmud3115]

{ "question": "Since the derivative of the loss function is calculated with respect to the weights of the model during backpropagation, what characteristics should the loss function have?", "choices": [ "Continuous", "Discontinuous", "Constant", "Zero" ], "answer": "Continuous", "creator": "Fahim Mahmud", "creationDate": "2020-02-12T04:10:35.701Z" }

[@deeplizard]

Thanks, Fahim! Just added your question to deeplizard.com/learn/video/XE3krf3CQls :)

[@mjain2172]

Yes, please explain math behind of this calculation, thanks for explaining deep neural network concepts

[@deeplizard]

Great :D The math is included in the following episodes. Starting with this one: deeplizard.com/learn/video/2mSysRx-1c0

[@krishnasaibiradar]

Thanks for the awesome video on backpropagation Yes i want to know the math behind backpropagation as the whole logic lies in and around math.So kindly make a video on the math behind the Backpropagation.

[@deeplizard]

You're welcome, krishnasai! There are 4 episodes following this one that explain the math :) And they all have corresponding blog articles along with the video! The next one starts at the link below, and the following 3 are directly after. deeplizard.com/learn/video/2mSysRx-1c0

[@loganmay2105]

Never thought I'd hear the words "chain rule" outside of my old high school AP calc class

[@deeplizard]

🤓

[@krishnaik06]

Math too please

[@avibcci1297]

interesting

[@omargonalfa]

Awesome video. please a brief example of the math behind.

[@deeplizard]

The math is shown in the following four videos of the series :D

[@fritz-c]

I noticed a broken link in the article for this video, near the words: "shown in an [earlier post]." It looks like part of the url got duplicated.

[@deeplizard]

Fixed, thanks Chris! :D

[@nourelislam8565]

Amazing... Before asking about the math courses for the backpropagation. Is it essential to study the math behind deep learning in order to work with Keras or any other APIs? Thanks for these fruitful videos!

[@generalzeedot]

would this backpropagation technique, therefore, only be usable in supervised learning?

[@jesilmohammed7926]

What is the default height and width of a conv_2d filter ?

[@vantongerent]

great video - yes I want to see the math! 🙂

[@richardme9928]

Any links to videos that do the complete math calculations ?

[@deeplizard]

Hey richard - They are the following videos in this playlist. Check out the full series in order here: deeplizard.com/learn/playlist/PLZbbT5o_s2xq7LwI2y8_QtvuXZedL6tQU

[@sanketkachole9639]

yes we need it

[@deeplizard]

You got it - Math starts in the next episode :D kzbin.info/www/bejne/aJ62qqaIrZJkmZI

[@VISWESWARAN1998]

Thank you so much

[@GelsYT]

Hello :D Is the output of the activation equal to the weight ? i mean is the activation the weight? of every neuron firing? right? cause i think that's what I remember about your video on activation function so basically the output of the activation function is going to be the weight? thank you so much great tutorial ! you explain it at the simplest way :))

[@rohtashbeniwal9202]

great videos,,saveed,subscribed ,

[@deeplizard]

Awesome, thank you!

[@jabrams3643]

i'm confused about the difference between the gradient and derivative? Is it called a gradient of the loss function when you average out all of the derivatives of the loss wrt to a certain weight? Meaning that its called a derivative when its just dloss/dweight for a single sample, and its called a gradient when its the average dloss/dweight for all of the samples? thank you for this amazing series btw it is the best on the internet

[@GelsYT]

sooo gradient descent is the one who's updating the weights? thank you soo much

[@deeplizard]

Yes it is!

[@abail7010]

Something isn't clear tome. Do we update the bias for each neuron or is this example without using bias? Apart from that, I love your videos and the way you explain things! Please keep up since this is a highly interesting thing. :)

[@deeplizard]

Thank you, Xi! I've left bias out of this example, but they get updated at the same time and in the same fashion as the weights.

[@abail7010]

Thank you for the reply! I've not seen your video to the bias which cleared my question when I commented. :)

[@herohari27]

In which video you were talking about SGD?

[@deeplizard]

Hey Hari - These two: deeplizard.com/learn/video/sZAlS3_dnk0 deeplizard.com/learn/video/_N5kpSMDf4o

[@zrmsraggot]

Why cant we just change the weights from pink to blue nodes since we just want activation in blue nodes to change to be efficient

[@ojasvinnagpal9472]

I have a similar question: Why can't we simply change the weights between the blue and the yellow nodes and arrive at the optimal values for the hidden layer units?

[@ojasvinnagpal9472]

Is it the case that in order to change the nodes in each layer, we need to change BOTH the nodes in the previous layer as well as the weights between the previous layer and the current layer?

[@meetayan15]

hi, do you have any lecture on SGD ?

[@deeplizard]

Hey Ayan - The following two videos (in the order listed) talk about SGD: 1. kzbin.info/www/bejne/qYuknYZplMmhoZI 2. kzbin.info/www/bejne/lX-YnKOJgqmZatE Note that SGD uses backpropagation during training, which is where most of the "grunt work" comes into play. So after generally understanding what SGD is doing from the above videos, this 5-part backprop series, starting with the video we're currently commenting on, gives all the details for what backprop is doing during the training process.

[@xeroxSoldier]

Love your videos, really helping me with my master thesis! :) You're talking about SGD but there are also other optimizers. I'm interested especially in Adam, and how it differs from SGD, maybe you got some paper or article recommondations? - In my understanding Adam is doing the exact same thing as SGD just using another algorithm, is that correct? - You're talking about backpropagation referring to SGD, is Adam also using backpropagation? In my understanding, backpropagation ist just the general term of changing the weights after every forward propagation.

[@torbjornstorli2880]

Is the backpropagation being applied only once per epoch? So, if you have 20 epochs you will perform backpropagation 20 times, once per epoch ?

[@deeplizard]

Hey Torbjorn - It occurs at each batch. The details for this implementation are covered in the backprop videos that come after this one in the playlist.

[@stydras3380]

I can only agree with the other comments, knowing the math is neat :D Is there some sort of follow up video of yours?

[@stydras3380]

Ok, I have found the videos discussing the math :D

[@rajuthapa9005]

r u bringing RNN tut too?

[@deeplizard]

Hey Raju - Yes, I have RNNs on my list to cover in future videos.

[@aamir122a]

same here.

[@sprajapati2011]

7:17 'ie' is pronounced as 'that is' not i e itself

[@aliasgarzakir4779]

Yes please, math is fun.

[@deeplizard]

All the math is in the episodes following this one :D

[@torbjornstorli2880]

That is per batch per epoch

[@ahdm1319]

please explain the maths behind this in seperate videos

[@deeplizard]

The math is shown in the next four videos after this one. Let me know what you think!

[@saranshtayal2526]

i want to know about the math

[@deeplizard]

It's in the following episodes after this one :)

[@danluba]

And yes, I could go for some math.

[@deeplizard]

The math for backprop starts in the next video in the playlist! Here are the full details for the backprop series: Backpropagation explained | Part 1 - The intuition (this video) kzbin.info/www/bejne/jnaWnKWcaKiEotU Backpropagation explained | Part 2 - The mathematical notation kzbin.info/www/bejne/aJ62qqaIrZJkmZI Backpropagation explained | Part 3 - Mathematical observations kzbin.info/www/bejne/fWbFZZ2Id7CBrtk Backpropagation explained | Part 4 - Calculating the gradient kzbin.info/www/bejne/kKOYp5x3j6yhmqc Backpropagation explained | Part 5 - What puts the “back” in backprop? kzbin.info/www/bejne/rnTPfJKVeNaNpLM

[@danluba]

Yeah - I found it. Awesome stuff. Thank you!

[@deeplizard]

Awesome, you're welcome!

[@josegregorioperezmagallane3211]

We want to Sep the math

[@deeplizard]

The math explanation is in the following four videos of the series :D

[@urvashidang6083]

i want to know the maths behind

[@deeplizard]

Great! It's in the following episodes. Starting with this one: deeplizard.com/learn/video/2mSysRx-1c0

[@wesleyrademaker5167]

math please!

[@deeplizard]

Hey Wesley - The math is in the following four videos after this one in the playlist!

[@yvesa6959]

give maths pls

[@deeplizard]

The math comes in the episodes that follow this one :D

[@MrFranciscoooooo]

A video for this a video for that!! Just teach or do a brief sum of what this is, because if I want to search more I will do it from other videos not just yours.

[@lando6583]

i don't want to know about the math.

[@deeplizard]

Warning: Don't watch parts 2 - 5 of the backprop videos 😜

[@akshatsahu2637]

The background is so bad. The image and background blend

[@deeplizard]

I agree the image and the background don't have enough contrast with each other. The background has changed in later videos.

[@akshatsahu2637]

@@deeplizard rest all the videos are very very good. The way everything is explained is awesome!. Better that course era. One last things, are there any videos in which the programming part is explained or the dimensions of the error or Delta, and the dimensions of theta?

[@jesilmohammed7926]

What is the default height and width of a conv_2d filter ?