PyTorch Tutorial 04 - Backpropagation - Theory With Example

New Tutorial series about Deep Learning with PyTorch!
⭐ Check out Tabnine, the FREE AI-powered code completion tool I use to help me code faster: www.tabnine.com/?... *
In this part I will explain the famous backpropagation algorithm. I will explain all the necessary concepts and walk you through a concrete example. At the end we will see how easy it is to
use backpropagation in PyTorch.
- Chain Rule
- Computational Graph and local gradients
- Forward and backward pass
- Concrete example with numbers (Linear Regression)
- How to use backpropagation in PyTorch
📚 Get my FREE NumPy Handbook:
www.python-engineer.com/numpy...
📓 Notebooks available on Patreon:
/ patrickloeber
⭐ Join Our Discord : / discord
Part 04: Backpropagation - Theory With Example
If you enjoyed this video, please subscribe to the channel!
Official website:
pytorch.org/
Part 01:
• PyTorch Tutorial 01 - ...
Linear Regression from scratch:
• Linear Regression in P...
Code for this tutorial series:
github.com/patrickloeber/pyto...
You can find me here:
Website: www.python-engineer.com
Twitter: / patloeber
GitHub: github.com/patrickloeber
#Python #DeepLearning #Pytorch
----------------------------------------------------------------------------------------------------------
* This is a sponsored link. By clicking on it you will not have any additional costs, instead you will support me and my project. Thank you so much for the support! 🙏

[@fuma9532]

Clear, simple, with an easy example... Just brilliant! Thank you so much, I wish more people made things simple :D I'm sharing this!

[@patloeber]

thanks so much!

[@fuma9532]

@@patloeber Thank you man: you're doing a great job!

[@nicholastaylor9743]

This is wonderful. I appreciate the simple example and translating to PyTorch. Too few videos connect the two. Thank you!

[@vaibhavtiwari1084]

This is the best video I've seen for backprop, forward pass and how the values get updated. Classic!!

[@tanmaybhagwat3650]

7:30 That's the craziest 1 I have ever seen.

[@kaveen_93]

That's how we write 1 in Germany, assuming the tutor is from Germany. I believe this is done to ensure the clear difference visually between, 1 (one), I (uppercase i), l (lowercase l) etc,.

[@Adarsh256]

you're a genius, I've been stuck on this step since ages. Thanks and keep going!!

[@patloeber]

Glad you like it!

[@jayceonsantana1405]

i guess it's kind of off topic but does anybody know a good website to stream new tv shows online ?

[@leonidaspreston3310]

@Jayceon Santana Flixportal :D

[@jayceonsantana1405]

@Leonidas Preston Thanks, I signed up and it seems like a nice service :) Appreciate it !

[@leonidaspreston3310]

@Jayceon Santana No problem xD

[@amitg2k]

You just made it look so simple. One of the best tutorials in a long time. Many thanks.

[@patloeber]

Glad you liked it!

[@sugamtyagi9144]

I found your explaination of backpropogation very intutitive. I'm loving this playlist.

[@patloeber]

Thank you 😊

[@djchrisi]

Da hast du mich wirklich an die Hand genommen und alles ganz super erklärt! danke schön!

[@Vikram-wx4hg]

Excellent clarity and explanation!

[@ericohana9650]

Short, sweet and to the point, thank you!

[@hpaghababa8111]

That was a great tutorial and I enjoyed it. Before watching this video I used to calculate gradients through a painful operations!!! It would be interesting if you could have other examples with a bit complicated functions like multi-rule functions (say absolute value function for example). Moreover, if you give an exercise to do and attach a solution, it would be great for learners to practice, I guess.

[@davidcordova1773]

this is the best explanation of backpropagation on youtube, thank u so much

[@patloeber]

Glad to hear thar!

[@johnhildenbrand9867]

First time I finally understand the chain rule in detail....thx!

[@hamzawi2752]

Thank you so much for your extraordinary explanation. Please keep up this informative video.

[@josepoktopus8924]

Simple and precise, go ahead with your great work.

[@choco4429]

Your video is clearer than my teacher's lecture. I subscribe directly

[@patloeber]

thanks! glad yo hear that

[@arslanali900]

Amazing explanation! good job man.

[@imdadood5705]

I have always noticed. Even the complicated topics can be easily explained when we simple numbers without going with the alphabets. Thanks, Python Engineer!

[@patloeber]

glad to hear that :)

[@kevinliao9938]

This is really helpful. Thank you!

[@castilTube]

Great explanation!. Thank you very much for this video. Your channel is fantastic!

[@patloeber]

thanks for the feedback :)

[@dfdiasbr]

😱 what an explanation! Thanks!!

[@xinyi9015]

It's really helpful! Thank you!

[@Kimthepowerfuljong-un]

the best video i could've ever find! thanksss!

[@HieuTran-rt1mv]

Great! So easy to understand. Thank you so much

[@patloeber]

Glad you like it!

[@greenmanhulk]

The best so far

[@Andrey-dh6sx]

Thanks, very detailed explanation, helped a lot!

[@patloeber]

Great to hear!

[@justlasse7773]

Thank you so much!

[@dwhdai]

great videos, thanks for doing these!

[@patloeber]

glad you like it!

[@maithilijoshi796]

thankyou for the explanation

[@alirezamohseni5045]

that was very useful, thank you so mush

[@jacobmoore8734]

One quick note about linear regression @4:46, gradient methods are almost never used in practice because there is a closed form solution with pure linear algebra.

[@albertog2196]

Thanks. Good explanation

[@patloeber]

Thanks :)

[@sb-tq3xw]

thanks a lot for this video, keep going!!!

[@patloeber]

Glad you like it :)

[@adeelahmad1066]

Thanks, I like this tutorial

[@alirezaasadi9109]

Keep on going ... Very very useful ...

[@patloeber]

thanks :)

[@adityasahu96]

extremely well explained.

[@patloeber]

Thanks 😊

[@yangthomas5064]

love it ,good

[@vipulritwik]

Very well taught

[@patloeber]

thanks!

[@shrawansahu9500]

It was Brilliant can you please add playlist for Pytorch with NLP

[@InfoArduino]

Very good!

[@patloeber]

thanks!

[@ashishpondit8183]

thanks man big help

[@patloeber]

Glad it is helpful!

[@monsterpets5899]

love it

[@muntasiradnan6321]

so after backward pass 1st iteration, first we need to reset gradient, right? then start a new iteration like update weight, forward pass and so on?

[@jorgemercadog]

Thank you!!

[@patloeber]

glad you like it!

[@manideepgupta2433]

hey!! your videos are very clear! but i have a small question in this, if x,y are 2 separate variables, then dz/dx=x'y+yx' ? why did you consider other variable as a constant? at 2:55

[@patloeber]

Thanks for watching! In this toy example the function is x*y, so df/dx=y. We are using partial derivatives here, that means the other one is constant. But we also have to do the partial derivative for y then

[@nikcru]

i request you to make a Udemy course out of these video series, very nicely taught.

[@patloeber]

thanks! maybe in the future! What kind of course would you be interested in?

[@thomasgeudens7131]

@@patloeber machine learning from zero to hero?

[@lambsauce5445]

@@patloeber a full course. Please. It'd go really well. Please make it free tho...

[@robosergTV]

@@lambsauce5445 a free course is the one you watch LOL

[@TheOraware]

Hi Patrick , you are great thanks for it. At 6:35 doing backward pass you mentioned dloss/dy^ , i think it should be ds/dy^ isnt? Please clear my confusion

[@patloeber]

No what I mentioned is correct. Maybe it gets clearer at minute 9 when I explain the steps in more detail. While doing the backward pass we need dloss/dy^ as an intermediate result for the chain rule

[@Kenbomp]

Nice to have a use for calculus

[@patloeber]

Indeed!

[@whaysdsdsd973]

Thank you so much (T^T), it made me understand.

[@mykun8737]

thanks, How simple everything is when I hear you draw pictures with a pen and explain

[@rohit2761]

I have installed torch in my system. But in VS code, it shows "No module named torch". If i create virtual env and then pip install torch, it works. But Not In VS Code. Please help Thanks

[@panosparaskevopoulos5043]

im 13 yo and i can confirm that some experiments with pytorch, help of chat gpt and a lot of thinking i manage to follow along (to be noted: i havent even been introduced to calculus or anything similar and im not a native speaker as you can see)

[@ezdul2404]

Great video, nice coding examples, but at 7:57 ds/dy^ should be -1 and not 1

[@rodrigosalazar6112]

It's 1. In this case it is taking the derivative with respect to y^, not y. This means y is treated as a constant here with regards to the derivative. The y gets dropped and the y^ turns into just it's coefficient which is 1.

[@ahmetcihan8025]

Hi Patrick my friend could you do c++ opencv tutorials also ? Thanks.

[@patloeber]

Good suggestion! Maybe in the future :)

[@sandroalexormeno]

This is a very useful tutorial. But i need a little bit explanation. when you said (4:47) dLoss/ds =ds^s/ds = 2s, but it,s not simple for me, why it's equal to 2s ??? Please give me a way or a route to understand it.

[@sandroalexormeno]

OH, I found my own answer

[@patloeber]

Glad you like it!

[@pokemonXD1]

Could either of you explain this to me please? I don't understand how we'd get 2s.

[@pokemonXD1]

Oh wait. dloss/ds where loss = s^2 so it's dloss = s^2 ds => dloss = 2s.

[@steven9492]

great video! the only thing complicated is this guys "1".

[@user-lx6lq7xb7c]

8:10 I need learn some calculate. to figure out why the answer is 1.

[@JohnFerrier]

Why is your 1 an upside down v? It makes things difficult to follow sometimes

[@gastroacid1971]

Strange it makes you disable gradients with no_grad() in order to actually subtract them. The software doing the calculus for you makes it much easier than writing the code manually in C++.

[@lucyfrye6723]

I did just that. It helps if you use a good linear algebra library, without it I got completely lost in endless for loops and keeping track of indices. By using the Eigen library , as if by magic, all the equations reduced to about 8 lines of code, it was one of the most gratifying pet projects I ever did, seeing four screens of unreadable code get reduced to a golden nugget. It took me about 8 days, including doing it the wrong way and learning the new library. I can recommend it!

[@minipy3164]

This is the best explanation for backpropagation in the entire KZbin

[@patloeber]

thanks so much!

[@AKSHAY99552]

why the calculation of d (y_pred - y)/d y_pred = 1 why not -1 ?

[@muhammadbilalhaneefqureshi48]

due to square, (-1)^2 = 1

[@sanjaykrish8719]

PyTorch should have this as their official tutorial..

[@patloeber]

thanks!

[@vatsal_gamit]

Can anyone please tell me how to remove that extra path in output terminal in VS code?? How to get clean output

[@patloeber]

Can be found in the settings for the code runner extension

[@shaheryarshafi]

i didn't understand backward pass clearly how we got -2

[@akashverma4280]

it is based on the chain rule. Can you explain exactly what you didn't understand? I will try to help.

[@user-lx6lq7xb7c]

9:53 final gradient = -2 dLoss/dw = -2 12:36 w.grad mean dLoss/d w h.grad dLoss/d h g.grad dLoss/d g

[@gorkemvids4839]

It could be better with proper naming

[@my_opiniondemocracy6584]

your one dosent seem to be a one...it seems like a big lamda... please try to correct it