Backpropagation in CNN - Part 1

Backpropagation in CNN is one of the very difficult concept to understand. And I have seen very few people actually producing content on this topic.
So here in this video, we will understand Backpropagation in CNN properly. This is part 1 of this tutorial, and in this is we will just look at Backpropagation for Convolutional Operation. In part 2, we will see how the gradients propagate backward in the entire architecture.
All the frameworks used for Deep Learning automatically implement Backpropagation for CNN. But as we humans are curious, we want to know how it works and not let it be implemented automatically.
So buckle up! And let's understand Backpropagation in CNN.
➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖
Timestamp:
0:00 Intro
1:49 What to obtain
4:22 dL/dK
11:46 dL/dB
13:20 dL/dX
18:51 End
➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖
📕 PDF notes for this video: bit.ly/BackPropCNNP1
➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖
Follow my entire playlist on Convolutional Neural Network (CNN) :
📕 CNN Playlist: • What is CNN in deep le...
At the end of some videos, you will also find quizzes 📑 that can help you to understand the concept and retain your learning.
➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖
✔ Complete Neural Network Playlist: • How Neural Networks wo...
✔ Complete Logistic Regression Playlist: • Logistic Regression Ma...
✔ Complete Linear Regression Playlist: • What is Linear Regress...
➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖
If you want to ride on the Lane of Machine Learning, then Subscribe ▶ to my channel here: / @codinglane

[@PranjS]

You will be remembered by many future developers as the person who helped them clear their concepts and probably also the person who helped them crack their job in this field. A big thanks for your work!

[@CodingLane]

Wow… so glad to hear this Pranjal. Thank you so much for your words 😇

[@waleedrafi7977]

Even though I have already watched your videos but due to your hard work and spending a lot of time on research, I also watch the whole video from my other accounts just to support you to increase your watch time, etc. so you don't stop making videos & I will highly encourage you to continue your hard work, one day you have a big audience.

[@CodingLane]

I don’t even how to react to this great comment 😄. Probably the best coment I have seen. Thank you so much. It really means a lot to me. I will keep on making such videos!

[@daniteka]

I wish I could show my appreciation by liking your videos multiple times, but unfortunately, the system limits us to a single like per video. Thank you so much for sharing your knowledge and expertise!

[@muskanmahajan04]

Prolly the best explanation on CNN Backprop there is on yt , thank you Jay!

[@CodingLane]

Your welcome. Glad to help!

[@user-pg4ui4us1u]

An absolutely perfect playlist that provided a wealth of insights - a huge thanks!

[@nadiaaouadi4266]

thanx for the whole series, I hope that you continue uploading videos in the future because they are amazing! *I never comment on videos but this was totally worth it*

[@devvratsahai2068]

you're a life saver man!!

[@arashroshanpoor1682]

This is the best explanation I looking for. thx

[@avisinh7249]

Best example and explanation I have seen. Thanks!

[@jordiwang]

again bro, its quiet clear and I am loving it appreciate it

[@mariap.9768]

Excellent work, I like how you get to the point quickly.

[@locostineverything4810]

Thanks for all the contents so far , Appreciate it !

[@shrotkumarsrivastava2441]

This is awesome and rare. Really appreciate the work you have put in here. Thanks for it. Keep rocking.

[@bevel1702]

Very down to the point and descriptive, didn't think I'd be able to understand this but the way you described it made it crystal clear. Good work!

[@CodingLane]

Thank you 😇

[@WilliamJung98]

This is incredibly helpful. Thank you so much.

[@jaiminjariwala5]

Best Explanation Ever! Thank You so much Brother!

[@user-ob5ts8yc7v]

I'm Korean subscriber, you contents is very helpful for me. Thank you!

[@Pierredefermatteee]

Thank you for the excellent content that you provided for no cost. Please continue making such precious videos, I know you will rock it.

[@CodingLane]

Thank You for your kind words!

[@MrAMerang]

BEST youtube on this topic, Thank you very much.

[@Maciek17PL]

Awesome video, crystal clear explanation!!!

[@nurabba3273]

Hello brother. I am not the first I know to get enlightened by your educative videos. But I will say this, you have save my days by giving me so much insight on machine learning stuffs. Thank you

[@emanuel8418]

I just can't describe how much you helped me with this video. Thank you so much.

[@CodingLane]

Glad to help 🤗!

[@jackrozmaryn7905]

Jay, Excellent, Excellent, Excellent and excellent thank you!

[@josephj1643]

Hey, Thank you for making this video. Most of the people on KZbin have explained CNN, but have not explained how Back Prop works in CNN saying it is similar to DNN. I was looking for this video, since a week, it really helped. Do continue making such videos!!!

[@CodingLane]

Glad it was helpful! Sure, will keep uploading more videos. 🙂

[@kumkumupreti]

Thank you so much...u really make every concept crystal clear ❤

[@thienlu7011]

Thank you a lot! Very intuitive explaination.

[@naseeruddin832]

At last, found one the real backprop of CNN. Thanks buddy

[@CodingLane]

Your welcome! Glad it helped.

[@lz-ym5eq]

Thank you for an explanation. It helped me a lot

[@shahnewazchowdhury4175]

This is a fantastic video. Keep up the great work! It's surprising this does not have several hundred thousand views already.

[@CodingLane]

Thanks, will do! 🙂

[@AnkurKumar-dc3db]

Finally after watching this video I now fully understand the backpropagation through CNNs. Thanks man for creating this video.

[@CodingLane]

Happy to help! 🙂

[@farrugiamarc0]

Very well explained despite this being a very complex topic and very challenging to teach. Well done!

[@alexsemchenkov5740]

The best explanation on the internet! Thanks!

[@CodingLane]

Thank you. Much appreciate your words!! 🙂😄

[@tnmyk_]

Amazing lecture! Very well explained! Keep up the good work, man!

[@CodingLane]

Thank you

[@BlackmetalSM]

Even Chat GPT was not as clear as you. Great job!

[@user-oq7ju6vp7j]

Thank you for your videos!They are very helpfull for those of us, who are making NN's from srcatch.

[@anshulsaini5401]

Man, this is prolly the best video I have seen so far. Literally thanks for this video I have been struggling with backpropagation for 2 weeks and today I feel like ik everything lmao. Hats off to you brother keep grinding, keep rising. Definitely sharing this playlist (especially this video) in my circle!!

[@CodingLane]

Thank you so much for your comment. I feel very good seeing this. Glad to see that it helped you. Comments like these keeps me going and make me create more such content 😇.

[@vandanvirani167]

Dude really great I was searching whole over internet to find perfect explanations of cnn back propagation but on one explain like you even best teachers on KZbin

[@CodingLane]

Thank you. Glad I could help 😇

[@adelzier4264]

You are the best bro ! Hope you all the best

[@CodingLane]

Hey… thank you!

[@SharathsooryaBCS]

Sir, Really wonderfull explanation sir. I got the concept at the very first attempt. Thank you so much sir.

[@ce108meetsaraiya4]

This is best explanation of backpropogation in CNN

[@CodingLane]

Thank you!

[@mukandrathee]

U made it so easy. Thanks

[@CodingLane]

Glad to help!

[@Aca99100]

Hey Coding Lane! This tutorial was a life saver for me and just what I was looking for. As you said there arent many sources on the internet that explain backprop in CNN's especially in this depth. Thank you for this video, you got yourself likes on both parts and a new sub! Keep doing what youre doing!

[@CodingLane]

Happy to help! And thank you for the comment 😇

[@learningfoundation6601]

Amazing content keep uploading

[@sushantregmi2126]

your content is unreal, thank you very much...

[@CodingLane]

Thank you so much. Glad you find it helpful! 😀

[@user-do3mw8yx4y]

Thank you so much!!

[@jameshaochi0824]

Thank you so much.

[@getisbhai]

amazing video bhai

[@RH-mk3rp]

You are an AMAZING PERSON, THANK YOU.

[@CodingLane]

You are so welcome! Thank you! 🙂

[@RH-mk3rp]

@@CodingLane I retract my statement. Your convolution backpropagation assumes batch_size=1 and channels=1, perhaps to simplify the problem. but in practice this is never the case. If you explain all that in another video then I apologize, but thus far I have found none.

[@CodingLane]

@@RH-mk3rp Hi, it would have been difficult to understand the backpropagation if batch_size is taken greater than 1. The purpose of this video was to help viewers understand how backpropagation works. But I hope you find and understand the solution where batch_size > 1 and channels > 1 😇

[@enricollen]

thanks sir, appreciate it

[@omerihtizaz9043]

Great video, Keep up the good work!

[@CodingLane]

Thank you!

[@ummayhaney4162]

Thank you❤💙

[@shreedevisindagi888]

Thank you .

[@VR-fh4im]

Brilliant.

[@cobblin_gock]

awesome work. well explained.

[@CodingLane]

Thank you!

[@gauravshinde8767]

Doing Masters in AI in Ireland. Lectures: Understood nothing Coding Lane videos: Understood everything Teaching is a skill, you got that bro

[@firstkaransingh]

Good explanation bruh

[@rimeblb573]

thank you

[@jgg0207]

preciate this!! Very helpful

[@CodingLane]

😇

[@joelrcha3368]

Great explanation

[@CodingLane]

Thanks

[@qaiserali6773]

Lovely!!

[@CodingLane]

Thank you! 😊

[@dhawalpatil7779]

Hey coding lane I have only 2 words to say YOU ROCK 🎉

[@CodingLane]

Hahaha… Thanks a lot! Cheers 🎉

[@a.k.103]

bhai tere liye mere pass koi shabd nahi h ( I don't have any words for you), I have never seen content like this. i can write like this comment on every video but it will also be so less against your hard work. keep this up I always support you. Love you, bro.

[@CodingLane]

Thank you so much 😊. I am really glad that my content is this much valuable. This is One of the most loving comment i have seen.

[@waleedrafi1509]

Great Video.

[@CodingLane]

Thank you!

[@DangNguyen-yh5mm]

Amazing job, hope you have an example on simple CNN with updating weights on these layer(conv->relu->pool->... just a few layers), thank you for your this video.

[@CodingLane]

Yes... that is coming in the next video

[@punamkhandar2678]

informative

[@gajendrasinghdhaked]

insane content

[@fitrinailahanwar4102]

Clear explain, thank you sir, but please add subtittle in order can be easily to understand , thank you if you read this suggest

[@CodingLane]

Hi, KZbin somehow faced error to add subtitles in this video. Sorry for the inconvenience. But glad you found it helpful!

[@John-wx3zn]

thank you. why is del L/del X being found when backpropagation only updates the kernel weights matrix and the bias scalar?

[@lodemrakesh7092]

Good one

[@CodingLane]

Thanks

[@ness3963]

Thank you bhai 🙁

[@CodingLane]

Welcome!

[@user-lv2pg5so2j]

nice video sir

[@CodingLane]

Thank you!

[@user-fz3ui4zp6h]

I cought this logic due investing lots of time. However, I can't understand one aspect: In the full connected model: The model is received the "INPUT" we make the forward then to get the loss and based on it we fix each weight in the model using backpropagetion. During this process we never deal with "INPUT", because the "INPUT" could be different but we need to train our model to predict. As I realise in a convolutional model a kernel plays the same role as weight in the full connected model and during training our goal to configure the kernel for better prediction What I can't really understand why fix the INPUT data in the convolutional model convolutional model INPUT is alwais changed but kernel stay constantly. Based on this logic we need only to configure the kernel. Could you explain, please?

[@John-wx3zn]

thank you. why didn't you show the activation functions step?

[@ASdASd-kr1ft]

why appear the sum in the chain rule for the derivati of a matrix respect to another one? you know someany source where i can figure out how this work? Thanks

[@harryt5878]

This is super useful thank you! However I am slightly stuck with finding dl/dk as for example in my neural network the previous layer has output 14x14x20 and the convolutional layer uses 20 3x3x20 filters so the dl/dk needs to be of size 3x3x20x20, but by applying the convolution(X, dl/dz) the output is 3x3x20 as dl/dz has size 12x12x20) how do I fix this?

[@user-oq7ju6vp7j]

HI. Did you find out how to solve it? I have the same problem

[@spyder2374]

Ultra Nice explanation ... 👍 Q - why we update kernels while back propagation, kernels should be fixed right ? Let if a kernel is an vertical edge detector, after back propagation through it, what it will be ???

[@CodingLane]

Hi, Kernels should not be fixed. Because you don't know which kernel is detecting what kind of features. The vertical edge detection was just an example to show that kernels detects edges, but in any model, we don't know which kernel is detecting what kind of edges. The model decides it itself. And that is why we update kernels through backpropagation, so that they automatically take appropriate values to identify features. Manually setting kernel values of so many kernels will be very tedious job. Let say, if you change your model and add new kernels, then you will have to set those values as well. Hope the answer helps. And sorry for this really late reply.

[@arpit743]

hi jay! can you please explain the Einstein summation convention part? My issue is that the convention says Mij= sigma k (Aik*Bkj), but in the video you mentioned dl/dkmn = sigma( dl/dzij * dzij/dkmn), is this equation consistent with Einstein convention? as it has both as ij ??

[@CodingLane]

Hello... yea it is still consistent with Einstein convention. Convension just says that if you have ij appearing at cross position (numerator of one and denominator of other) then it implies to do summation over those terms.

[@rahulkumarjha2404]

Great video. I just have one doubt, Why are we calculating dL/dX I mean using backpropogation we only update weights, biases and the terms in filter matrix. Please answer.

[@CodingLane]

Hi, yes the end goal is to update weights and biases. dL/dX helps us calculating dL/dW and dL/dB of previous layer. Checkout 3:27 timestamp. dL/dX will become dL/dZ to the previous layer.

[@siddhanthsridhar4742]

hi at 15:30 equation of del L/del X12 is written wrong-it should be del L/del Z11*k11 insread of del L/del Z12*k11

[@mounmountain141]

I would like to ask if there is any material to learn when encountering situation， when stride != 1 or have dilation

[@CodingLane]

I found one article which showed that… but unfortunately I have lost it now. May be you can find it on google.

[@mounmountain141]

​@@CodingLane OK, Thank you for this very helpful video.

[@axitamohanta6743]

and what about delL/delZ

[@ramchandhablani9834]

He Jay at 3:34, I think you have written wrong equations Z11=X11K11+X12K12+X21K21+X22K22+B, B is a 2x2 matrix, you can not add to scalar Z11😣

[@puchaharinathreddy5556]

can you give us some example

[@samueljohanes5219]

what is L?

[@haidarrmehsen]

am I missing something or he didn't mention what is L?

[@gamerx3582]

It's loss function

[@surender_kovvuri]

bro can you provide the python code for this CNN model

[@Jeffrey-uw8un]

MATHHHHHHHHHHHHHHHHHHHHH TOOOOOO MUCH MATTTTTTTTTTTTTHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH

[@Jeffrey-uw8un]

BTW awensome video

[@malanosi2869]

for gods sake please buy a mic i love ur lectures but can hear a damn thing in some clips

[@CodingLane]

Hey thanks. I have got one, will use it when create new videos 😊

[@berwinamir5325]

thank you a lot, this is really very very helpful 🤩🙌

[@CodingLane]

You're welcome. Glad I could be of help! 😄🙂