Tutorial 6-Chain Rule of Differentiation with BackPropagation

[@debtanudatta6398]

Hello Sir, I think there is mistake in this video for backpropagation. Basically to find out (del L)/(del (w11^2)), we don't need the PLUS part. Since here O22 doesn't depend on w11^2. Please look into that. The PLUS part will be needed while calculating (del L)/(del (w11^1)), there O21 & O22 both depend on O11 and O11 depends on w11^1.

[@alinawaz8147]

Yes brother there is mistake what is said is correct

[@prakharagrawal4011]

Yes, This is correct. Thank you for pointing this out.

[@aaryankangte6734]

true that

[@vegeta171]

You are correct concerning that, but I think he wanted to take derivative w.r.t O11 since it is present in both nodes of f21 and f22, so if we replace w11^2 in the equation by O11 the equation would be correct

[@byiringirooscar321]

it took me time to understand it but now I got the point thanks man but I can assure you that @krish naik is the first professor I have

[@ksoftqatutorials9251]

I don't want to calulate Loss function to your videos and no need to propagate the video back and forward i.e you explained in such a easiest way I have ever seen in others. Keep doing more and looking forward to learn more from you. Thanks a ton.

[@tarun4705]

This is the most clear mathematical explanation I have ever seen till now.

[@moksh5743]

kzbin.info/www/bejne/f6nPZKGvoLB6b68

[@AmitYadav-ig8yt]

It has been years since I had solved any mathematics question paper or looked at mathematics book. But the way you explained was damn good than Ph.D. holder professors at the University. I did not feel my away from mathematics at all. LoL- I do not understand my professors but understand you perfectly

[@RomeshBorawake]

Thank you for the perfect DL Playlist to learn, wanted to highlight a change to make it 100% useful (Already at 99.99%), 13:04 - For Every Epoch, the Loss Decreases adjusting according to the Global Minima.

[@vishnukce]

But for negative slopes loss has to increase know to reach global maxima

[@being_aadarsh]

@@vishnukce For negative slopes weights need to be increased instead of a loss

[@OMPRAKASH-uz8jw]

you are no one but the perfect teacher,keep on adding playlist

[@ganeshvhatkar9040]

one of the best videos, I have seen in my life!!

[@namyashah3173]

No one has ever explained like you did.hatts off!!

[@VVV-wx3ui]

This is simply yet Superbly explained. When I learnt earlier, it stopped at Back Propagation. Now, learnt what is in Backpropagation that makes the Weights updation in an appropriate way, i.e., Chain rule. Thanks much for giving clarity that is easy to understand. Superb.

[@rajeeevranjan6991]

simply one word "Great"

[@manateluguabbaiinuk-mahanu761]

Deep Learning Playlist concepts are very clear and anyone can understand easily. Really have to appreciate your efforts 👏🙏

[@aj_actuarial_ca]

Your videos are really helping me to learn Machine learning as an actuarial student who is from a pure commerce/ finance background

[@TheMainClip-t1h]

You have saved my life, i owe you everything

[@VIKASPATEL-of2sy]

i guess differentiation done at 11:26 is bit wrong, r u sure about? i mean why do we have to addan extra term of delta loss by delta w12

[@debasispatra8368]

yes correct. It seems a mistake. addition part will come when we will calculate derivative of w11 for layer 1, not for derivative of w11 for layer 2.

[@RajatSharma-ct6ie]

Yes you are correct !!

[@bhavyaparikh6933]

@@debasispatra8368 but why we dont have to add for layer 2 and add to layer 1

[@mranaljadhav8259]

@@bhavyaparikh6933 same question here....if you got it, can you explain.. I have just started deep learning.

[@nikitlune9526]

@@debasispatra8368 Hi, can you just tell how initially weights are assign and how many hidden layers and no. of neurons on each layer should be there?

[@nishitnishikant8548]

Of the two connections from f11 to the second hidden layer, w11^2 is affecting only f21 and not f22(as it affected by w21^2). So, dL/dw11^2 will only have one term instead of two. Anyone, pls correct me if i am wrong.

[@sahilvohra8892]

I agree. i dont know why others didn't realized this same mistake!!!

[@mustaphaelammari1128]

i agree, i was looking for someone has the same remark :)

[@ismailhossain5114]

That's the point I am actually looking

[@saqueebabdullah9142]

Exactly, cause if I solve the derivative of two terms it results d/dw11^2 *L = d/dw11^2 *L + d/dw12^2 *L , which is wrong

[@RUBAYATKHAN89]

Absolutely.

[@shaan2522]

great explanation of the chain rule in backpropagation.. all my doubts are cleared!! thankss

[@saritagautam9328]

This is really cool. First time samjh aaya. Hats off Man.

[@varunsharma1331]

Great explanation. I was looking for this clarity since long...

[@adityashewale7983]

hats off to you sir,Your explanation is top level, THnak you so much for guiding us...

[@abhishek-shrm]

This video explained everything I needed to know about backpropagation. Great video sir.

[@mranaljadhav8259]

Well Explained sir ! Before starting the deep learning, I have decided to start the learning from your videos. You explain in very simple way ...Anyone can understand from your video. Keep it up Sir :)

[@hashimhafeez21]

first time i undestand very well by your explanation.

[@shrutiiyer68]

Thank you so much for all your efforts to give such an easy explanation🙏

[@mohammedsaif3922]

Krish your awesome finally I understood the chain rule from you thanks Krish again

[@manikosuru5712]

Amazing Videos...Only one word to say "Fan"

[@ruchikalalit1304]

@ 10:28 - 11:22 krish do we need both the paths to get added . since w11 suffix 2 is not affected by lower path ie w12 suffix 2? please tell

[@amit_sinha]

The second part of the summation should not come in the picture as it will come only when we will be calculating (dL/dw12) with suffix as 2.

[@SiMsIMs-1]

@@amit_sinha i think that is correct.

[@niteshhebbare3339]

@@amit_sinha Yes I have the same doubt!

[@vishaldas6346]

Not required, its not correct as w11^2 is not affected by lower weights. The 1st part is correct and summation is required , when we are thinking about w11^1.

[@grownupgaming]

@@vishaldas6346 Yes!

[@someshanand1799]

great video especially you are giving the concept behind it, love it.. thank you for sharing with us.

[@aditideepak8033]

You have explained it very well. Thanks a lot!

[@kamranshabbir2734]

the last partial derivative of Loss we have calculated w.r.t. (w11^2) is that correct how we have shown there that it is dependent upon two paths one w11^2 and other w12^2 ......... Please make it clear i am confused about it ??????

[@wakeupps]

I think this is wrong! Maybe he wanted to discuss about the w11^1? However, a forth term should be add in the sum. Idk

[@imranuddin5526]

@@wakeupps yes, i think he got confused and it was w11^1

[@Ip_man22]

assume he is explaining about W11^1 and youll understand everything. From the diagram itself, you can see the connections and can clearly imagine which weights are dependent on each other . Hope this helps

[@akrsrivastava]

Yes, he should not have added the second term in the summation.

[@gouravdidwania1070]

@@akrsrivastava Correct no second term needed for W11^2

[@MrityunjayD]

Really appreciable the way you taught Chain rule...awesome..

[@chartinger]

OP... Nice Teaching... Why don't we get teachers like u in every institute and college??

[@deepaktiwari9854]

Nice informative video. It helped me in understanding the concept. But i think at end there is a mistake. You should not add the other path to calculate the derivative for W11^2. Addition should be done if we are calculating the derivative for O11. w11^2(new) = (dl/dO31 * dO31/dO21 * dO21/dW11^2)

[@grownupgaming]

Yes deepak, I noticed the same thing. There's a mistake around 12:21. no addition is needed.

[@anupampurkait6066]

yes deepak you are correct. I also think the same.

[@albertmichaelofficial8144]

Is that because we are calculating based on o3 and 03 depends on both output from second layer

[@uddalakmitra1084]

Excellent presentation Krish Sir .. You are great

[@channel8048]

Thank you so much for this! You are a good teacher

[@sundara2557]

I am going through tour videos. You are Rocking Bro.

[@sundara2557]

Your*

[@punyanaik52]

Bro, there is a correction needed in this video... watch out for last 3 mins and correct the mistake. Thanks for your efforts

[@aaryamansharma6805]

your right

[@ZaChaudhry]

❤. God bless you, Sir.

[@tanvirantu6623]

love you sir, love ur effort. love from Bangladesh.

[@hokapokas]

Loved it man... Great effort in explaining the maths behind it and chain rule. Pls make a video on its implementation soon. as usual great work.. Looking forward for the videos. Cheers

[@shivamjalotra7919]

Hello Sunny, I myself have stitched an absolutely brilliant repository explaining all the implementation details behind an ANN. See this: github.com/jalotra/Neural_Network_From_Scratch

[@kshitijzutshi]

@@shivamjalotra7919 Great effort. Starred it. ⭐👍🏼

[@shivamjalotra7919]

@@kshitijzutshi try to implement it yourself from scratch. See george hotz twitch stream for this.

[@kshitijzutshi]

@@shivamjalotra7919 Any recommendation for understanding image segmentation problem using CNN? resources?

[@manjunath.c2944]

clearly understood very much appreciated for your effort :)

[@skviknesh]

Thanks ! That was really awesome.

[@good114]

Thank you Sir 🙏🙏🙏🙏♥️☺️♥️

[@dnakhawa]

You are too Good Krish , nice Data science content

[@chandanbp]

Great stuff for free. Kudos to you and your channel

[@devgak7367]

Just awsome explanation of gradient descent.

[@gunjanagrawal8626]

Could you please recheck the video at around 11:00, W11 weight updation should be independent of W12.

[@SiMsIMs-1]

Awesome Mate. however, I think you got carried away for the second part to be added. read the comments below and correct, please. W12 may not need to be added. But it all makes sense. A very good explanation.

[@mohamedanasselyamani4323]

Same remark concerning W12, good job Krish Naik and thank you for your efforts

[@ravikumarhaligode2949]

Hi Both, I also have same query

[@vishalshukla2happy]

Great way to explain man.... keep on going

[@kavinvignesh2832]

for the dL/w11^3 it should be dL/w11^3 = (dL/dO31 * dO31/dO31(before activation) * dO31(before activation)/dW11^3) right?

[@aminzaiwardak6750]

thank you sir, you explain very good keep it up.

[@grownupgaming]

Isnt the dL/dw2-11 independent of dL/dw2-12? At 12:21 why is dL/dw2-11 those two terms added up? dL/dw2-11 is the first line of additions, and dL/dw2-12 is the second line of additions.

[@yedukondaluannangi7351]

Thanks a lot for the videos it helped me a lot

[@rajshekharrakshit9058]

sir i think one thing you are doing is worng. as w^(3)11 impacts O(31) , here is one activation part. so the dL/dw^(3)11 = dL/dO(31) . d0(31)/df1 . df1/dw^(3)11 I might be wrong, can you please clear my query ?

[@sekharpink]

Very very good explanation..very much understandable. Can I know how many days ur planning to complete this entire playlist?

[@arpitdas2530]

Your teaching is great sir. But can we get some video also about how we will apply these practically in python?

[@mdmuqtadirfuad]

I can't understand( 11:09) dL/dw^2_11= 1st term + 2nd term... We are updating w11. But how w12 make impact (2nd term)?

[@viveksm863]

Im able to understand the concepts you are explaining, but I dont know that from where do we get values for weights in forward propgation.Could you brief about that once if possible.

[@sekharpink]

Hi Krish, Please upload videos on regular basis. I'm eagerly waiting for your videos. Thanks in Advance

[@krishnaik06]

Uploaded please check the tutorial 7

[@sekharpink]

@@krishnaik06 thank you..please keep posting more videos..I'm really waiting to watch your videos..really liked your way of explanation

[@sandeepganage9717]

Brilliant explanation!

[@pranjalgupta9427]

Nice 👍👏🥰

[@amitjajoo9510]

Best video on back proportional on internet

[@jontyroy1723]

In the step where dL/dw[2]11 was shown as addition of two separate chain rule outputs, should it not be dL/dw[2]1 ?

[@omkarpatil2854]

thank you for great explanation, i have a question, with this formula which generates for ( diff(L) / diff (W11)) is completely same for ( diff(L) / diff (W12)) i am i right? does both value gets same difference in weights while back propagation ( though W old value will be different

[@SunnyKumar-tj2cy]

Same question. What I think, as we are finding out the new weights, the W11 and W12 for HL2, both should be different and should not be added, or I am missing something.

[@abhinaspadhi8351]

@@SunnyKumar-tj2cy Yeah, Both should not be added as they are diff...

[@spurthygopal1239]

Yes i have same question too!

[@varunmanjunath6204]

@@abhinaspadhi8351 its wrong

[@rede_neural]

11:17 are you sure we have to sum them? It doesn't seems like the the two sides are equal when we "cancel" the chain

[@maheshvardhan1851]

great effort...

[@ThachDo]

10:44 you are pointing to w1_11, but why the formula on board is the derivative w.r.t w2_11?

[@winviki123]

That's correct. Even I was wondering the same

[@dipankarrahuldey6249]

I think this part dL/dw11^2 should be (dL/dO31 *dO31/O21 *dO21/dO11^2). If we are taking derivative of dL w.r.t w11^2 then,w12^2 doesn't come into play. So,in that case, dL/dO12^2= (dL/dO31 *dO31/O22 *dO22/dw12^2)

[@raj4624]

agree...dw11^2 should be (dL/dO31 *dO31/O21 *dO21/dO11^2). not extra afte addition

[@waynewu7763]

how do you take the derivative of d(O31)/dO21? what kind of equations are those?

[@saygnileri1571]

Nice one thnks a lot!

[@pranjalbahore6983]

so insightful @krish

[@sivaveeramallu3645]

excellent Krish

[@meanuj1]

Nice and requested to please add some videos on optimizer...

[@ga43ga54]

Can you please do a Live Q&A session !? Great video... Thank you

[@krishnaik06]

Let me upload some more videos, then I will do a Live Q&A session.

[@cynthiamoricordova5099]

Thank you so much for all your videos. I have a question respect of the value to assign to bias. This value is a random value? I will appreciate your answer.

[@camilogonzalezcabrales2227]

Excellent video, I'm new in the field, could someone explain me how the O's are obtained. Are that O's the result of each neuron computation? are the O's numbers equations?

[@chaitanyakumarsomagani592]

krish sir, is it w12^2 is depends on w11^2 then only we can do differentiation. w12^2 is going one way and w11^2 is going another way.

[@axelrocco2760]

Sir, I have a doubt , how will we calculate del(o31)/del(o21) , both are functions

[@mikelrecacoechea8730]

Hey Krish, god explanation I think there is one correction. In the end, you explained for w11^2, what I feel is, it is for w11^1.

[@tintintintin576]

so helpful video :) thanks

[@nikhilramabadran2959]

for calculating the loss function wrt W112 why do you also consider the other branch leading to the output ?? Kindly reply

[@nikhilramabadran2959]

it's mentioned clearly that it's wrt only W112 - the reason I'm asking this question

[@utkarshashinde9167]

Sir , If to every single neuron in hidden layer we are giving same weights and features with bias then what is the use of multiple neurons in single layer?

[@aswinthviswakumar64]

Great Video and a Great initiative sir from 12:07 if we use same method to calculate dL/dW12^2 it will be the same as dL/dW11^2. is this the correct way or am I getting it wrong thank you!

[@siddharthdedhia11]

Skip to 3:50 If you've watched the previous videos

[@anshuyadav24]

How do we know that we reached to a global mimima and we don’t need to update weights?

[@bsivarahulreddy]

Sir, O31 is also impacted by weight W11(3) ryt? why we are not taking that derivative in chain rule?

[@tabilyst]

Hi Krish, can you pls let me know, if we are calculating the derivative of W2 11 weight then why we are adding derivative of W2 12 weight in that. ? pls clear

[@hope2251]

10:30 i dont think w112 is effecting o22, so the plus oart should not come

[@grownupgaming]

Yes, that is what i feel too!

[@jerryys]

Great job! Does the last derivative need the second part? I do not get it.

[@kartikesood8242]

d(O22) will also be differentiated but with respect to w11, thus it will come out to be zero. Hence take it or not, result will be the same

[@shindepratibha31]

Hey Krish, your way of explanation is good. I think there is one correction. In the end, you explained for w11^2, what I feel is, it is for w11^1. It would be really helpful if you correct it because many are getting confused with it.

[@aneeshkalita7452]

I think the same.. But great method of teaching.. there is no doubting that

[@utkarshdadhich771]

@krish naik Correction at 13:05.. I guess Loss should be dcecreasing not increasing with to every epoch.

[@sandipansarkar9211]

yeah I did understand chain rule but being a fresher please provide some easy to study articles on chain rule so that i can increase my understanding before proceeding further.

[@bibhutiswain175]

Really helpful for me.

[@satishkundanagar3237]

"Why" back propagation works in learning weights of the neural networks? What is the intuition behind using back propagation to update the weights? I know that we are trying to make corrections w.r.t the predicted value if the predicted value has some errors when compared to the actual value.

[@shashishankar1352]

so depending on output predicted and output expected, we derive our loss function or cost function. By any mean if we can minimize overall loss of network our predicted output and expected output will reach closer which we want. Now think what we have in our hands to tweak so that loss can be minimized: 1. model hyper parameters (learning rate, no of layers, units in layer), 2. weight and bias for units across all layers. For option 2, we use back propagation where we take partial derivative of loss function w.r.t to unit weight and adjust that in unit weight. When we say partial derivative on loss function that actually mean drawing gradient(literal meaning slope in multidimensional plane), that gradient could be in upward direction or downward direction, so if gradient is negative then we are walking in downward direction which mean we are minimizing the total loss per loss function.

[@satishkundanagar3237]

@@shashishankar1352 Thanks for your reply. BGD/SGD is used to solve the optimization problem at hand and back propagation is a technique that is used in sync with Gradient descent for tuning weights and bias. Whatever you explained are all facts that have been researched, documented and the same are being used in implementing solutions across various fields. I'm looking for a mathematical and geometrical explanation as well as proof on why back propagation works.

[@pratikgudsurkar8892]

We are solving supervised learning problem that's why we have loss as actual-predicted , what in case of unsupervised where we don't have y actual how the loss is calculated and how the updation happen

[@benvelloor]

I don't think there will be back propogation in unsupervised learning!

[@aravindvarma5679]

Thanks Krish...

[@Philanthropic-fg8xx]

Then what will be the formula for derivative of loss wrt w12^2 ?

[@Skandawin78]

Do u update the bias during backpropagation along with weights? Or does it remain constant after the initialization?

[@krishnaik06]

Yes we have to update the bais too

[@kasimidrisi7602]

His sir i think there is something wrong wrong because the w11 to the suffix 2 is not impacted with the w12 to the suffix 2..! But this playlist is really helpfull to me thankyou sir...:)

[@ravikumarhaligode2949]

Hi Kasim, I am also having same query

[@enquiryadmin8326]

in the back propagation, calculation of gradients using the chain rule for the w11^1, i think we need to consider 6 paths. please kindly clarify.