How Gradient Descent Works. Simple Explanation

[@lovekildetoft5658]

All other videos on gradient descent are atleast 20 minutes long. This one is five, and made me understand more than any of those videos. Thank you!

[@DataScienceGarage]

Thank you for watching! Hoping it was useful.

[@blendaguedes]

Sometimes we just need two loops to understand a whole. Thank you!

[@uncoded0]

Thank you! Many hours of trying to understand gradient decent, now I finally get it, thanks to this video. Thank you!

[@anamikabhowmick6322]

This is one of the best and easiest way to learn and understand gradient descent, thank you so much for this

[@DataScienceGarage]

Glad you liked it! :)

[@impzhu3088]

That’s the way to explain a concept! Example with detailed steps. Thank you so much!

[@DataScienceGarage]

Thanks for watching! :)

[@hemantsah8567]

It is easy... spent my 2 days on learning gradient descent.... then I came to your video... Thanks bro

[@DataScienceGarage]

If you found useful in this video I highly recommend to check other related ones: -- Calculate Convolutional Layer Volume in ConvNet (kzbin.info/www/bejne/aaaolWN7p9Z6sLc) -- Adam. Rmsprop. Momentum. Optimization Algorithm. - Principles in Deep Learning (kzbin.info/www/bejne/j5LGgXh5pK5oibs) -- Numpy Argsort - np.argsort() - function. Simple Example (kzbin.info/www/bejne/bIibhnuso52Wock) -- Python Regular Expression (RegEx). Extract Dates from Strings in Pandas DataFrame (kzbin.info/www/bejne/e2XEp3WOl7OCfcU)

[@riki2404]

Thank you for such a clear explanation. Short and precise. no unnecessary talk.

[@DataScienceGarage]

Thanks for watching! Hoping it was useful :)

[@yaminikommi5406]

We can take any number as intial parameters and learning rate

[@wtry0067]

It's very short and very useful.. I get clarity what I was looking for. Thanks once again.

[@aalishanhunzai]

Bro thank you so much for your efforts, couldn't find a more simple explanation of gradient descent than this one.

[@DataScienceGarage]

Thanks for such feedback! :)

[@abdelrahmane657]

Oh my god, you are excellent. You make the difference on KZbin. Thank you so much. 🎉🙏👏🙌👌👍✌🏼

[@DataScienceGarage]

Thanks you for such feedback, appreciate it! :)

[@Elementiah]

Thank you so much for this! This is the perfect explanation! 😄

[@sukanya4498]

Love this video ❤️, Very simple and precise! Thank you !

[@DataScienceGarage]

Thanks for watching! :)

[@Alex-pd5xc]

wow dude, very clearly explained and you made it simple for me to understand. cheers man

[@DataScienceGarage]

Thanks for such feedback, appreciate!

[@Ziad-Ahmed-Mohamed]

2 hours in DL course i didnt get it 5 min made my day this is how actually learning should be

[@DataScienceGarage]

Glad it was helpful for you! :)

[@samvanoye]

Perfectly explained, thanks!

[@DataScienceGarage]

Thanks for watching! :)

[@phaniraju0456]

I bow to your for this great clarification ..loved it

[@muhammadhashir7949]

Thank you so much your work was practical and I loved it alot and underestood gradient descent. Before that I spent lots of time but didn't understood it properly

[@TingBie]

Thanks for this example, simple and spot-on!

[@murat2073]

thank you Sir! you are a HERO!!!

[@DataScienceGarage]

Thanks a lot! :)

[@josephsmy1994]

Awesome explanation! straight to the point

[@DataScienceGarage]

Thanks for such feedback! :)

[@zafarnasim9267]

You made it so simple. Great Job!

[@DataScienceGarage]

Thanks a lot! :)

[@ajaykushwaha4233]

Best explanation ever.

[@nawaab9275]

thanks for saving the semester

[@eramitvajpeyee85]

Thank you so much for explaining it in short and easy way!! Please keep uploading content like this.

[@DataScienceGarage]

Thank you for watching! Glad you enjoyed! :)

[@mohamedelkhanche707]

ohhhhh wonderful i was chocked this is insane thank you from all my heart

[@DataScienceGarage]

Thanks a lot for such feedback, appreciate!

[@omkarkadam5715]

Thanks mate, Finally Enlightened.

[@DataScienceGarage]

Thanks for watching! I hope it was useful :)

[@glenfernandes253]

how do you know, how many iterations to run before reaching the global/local minimum, what if it reaches the minimum and starts climbing on the other side ?

[@yasamannazemi6706]

It was so simple and helped me a lot :) Thanks👍🏻

[@DataScienceGarage]

Thanks!

[@alexbarq1900]

I get the idea but any reason for not doing simple math to find the local min? dy/dx = 2(x+5) If we want to find the min, we just do dy/dx=0.. then: 0 = 2(x+5) x = -5

[@hindbelkharchiche1654]

Thank you .. the explanation is as simple as useful .

[@abdellatifmarghan7521]

Thank you. grateful explanation

[@DataScienceGarage]

Glad it was useful! :)

[@9891676610]

Awesome explanation . Thanks a lot !!

[@DataScienceGarage]

Thanks for watching! Hope it was useful!

[@sanurcucuyeva7040]

Hi, thanks for explanation. If our function is hard, at what point in the iteration should we stop to find the minimum point

[@pwan3971]

Thanks a lot, really appeciate the video, this makes so much sense now

[@DataScienceGarage]

Thank you for you feedback! Glad it was useful for you :)

[@praneethcj6544]

Simple and clear ... Yet need more detailing ...!!!!

[@radhar5349]

Great explanation. Easy to get the concept

[@DataScienceGarage]

Thanks for feedback! :)

[@fmikael1]

Thanks for the great explination. everyone always complicates it

[@DataScienceGarage]

Thanks for feedback, glad it was helpful! :)

[@mbogitechconpts]

Beautiful video. I have to like it.

[@DataScienceGarage]

Thanks for feedback, inspiring!

[@abdanettaye8217]

good starting, thank you

[@dennisjoseph4528]

Great job of explaining this as simple as possible Sir

[@colton3000]

How do we find learning rate?

[@basheeralwaely9658]

Well done sir, very easy to understand

[@george4746]

Thanks, It was very clear and concise.

[@DataScienceGarage]

Thanks!

[@ydkmusic]

Great video! There is a typo around 3:50. The bottom equation should be x_2 = .... instead of x_1.

[@michaelscott8572]

What I don't get is: When we use this method in neural net, we don't know the Errorfunction. We just have some point. So how can I build the derivative?

[@AJ-et3vf]

Very useful! Awesome ❤️

[@DataScienceGarage]

Thanks for watching! :)

[@twicestay6683]

Thx a lot!!! But I'd like to ask why the learning rate=0.01? is it a random number? Thx

[@SuperYtc1]

This is a good video.

[@DataScienceGarage]

Thanks!

[@luisurena1770]

Coñazo siempre hay un indu que me ayuda a entender todo🔥🔥🔥🔥

[@Kay12234]

Wonderful Video!!! Thank You!

[@DataScienceGarage]

Thanks for feedback!:)

[@eliashossain4327]

Best explanation.

[@DataScienceGarage]

Thanka for such feedback! :)

[@blinky1892]

How do we know what the y value is of the parabole at any given x?😊

[@RayhanAhmedsimanto]

Amazing Practical Explanation. Great work.

[@DataScienceGarage]

Thanks Rayhan!

[@smurfNA]

hey! so do we choose the learning rate? and the gradient is simply just the function right ?

[@karthiklogan9384]

really helpful sir.thank you so much

[@DataScienceGarage]

Happy that was useful.

[@mastan775]

Very good explanation...thanks a lot.

[@Snetter]

Nice work! thanks

[@DataScienceGarage]

Thanks for feedback, glad for this! :)

[@bhavikdudhrejiya4478]

Very good video. I appreciate your hard work. Keep uploading more videos.

[@DataScienceGarage]

Many thanks for such comment!

[@machinelearningid3931]

Thanks, this give me the light in darkness.

[@davidbarnwell_virtual_clas6729]

How do we choose the learning rate? Good video but it's things like that I'd love to know

[@DataScienceGarage]

Hi! Choosing learning rate often is not easy task. I usually makes experiments on model performance with multiple learning rate (manual, Grid search hyperparameter tuning, Bayesian search, etc.).

[@davidbarnwell_virtual_clas6729]

@@DataScienceGarage Ahh...ok...I get you...it's very interesting.

[@Hasasinful]

Thanks just what i needed

[@DataScienceGarage]

Hope it was useful. Thank you!

[@MuditDahiya]

Very nice explanation!!

[@DataScienceGarage]

Thanks!

[@ericklestrange6255]

didnt explain how do calculate the direction we are moving to (the minus), why the derivatives etc

[@ak-ot2wn]

That's what I am looking for already for several days and nobody mentions this. Anyways I still think, that it is trivial. If your derivative is negative, you have to "move" to the right side (in case of 2 variables). If it is positive, you have to "move" to the left.

[@debayondharchowdhury2680]

he also didn't talk about loss calculation. why do we need to calculate loss at all if we can simply use the gradient descent on the function.

[@blendaguedes]

@@debayondharchowdhury2680 Your loss function is the one pointing the difference between your output and your "y". You calculate the gradient to your loss function. At his example, he shows something that looks like a ' mean squared error' as loss function to me, and he is doing a linear regression with only one input "x". I recommend you the Andrew Ng classes on Coursera. have a good time

[@blendaguedes]

@@ak-ot2wn I totally agree with what you are saying, the only matter is when you are programming you don't see witch direction your vector is going. So basically if the error is going down: keep going, if it starts to increase go back. You can just stop, or you can make your learning rate smaller to increase your accuracy

[@rssaiganesh]

I think the comments thread is looking for the math behind the formula for the gradient descent. Apologies if I misunderstood. But here is a link that helped me: towardsdatascience.com/understanding-the-mathematics-behind-gradient-descent-dde5dc9be06e

[@suezsiren117]

2:29 "Put the walrus in the correct places."

[@DataScienceGarage]

In 2:29 we are having x0 as - 3, graph representing it. Values are in correct places in formula. Could you clarify what is wrong there? Thanks!

[@pearlsofwisdom2416]

Good explanation but would have been better if you elaborated its formula of why it is used to reach next step. Why is derivative multiplied by learning rate and why it is then substracted from first point value

[@blendaguedes]

The learning rate makes the "decay slow". At his first interaction, the result would be: -3 -4 = -7. Can you see where this is going? As he goes slow he will keep dropping his "y", until he get to as close as possible to -5. Sometimes to get at the minimum you have to make you learning rate smaller while computing your weights .

[@sandipmaity2687]

Amazing Explanation :) Really simple and to the point 😀

[@muhammadhilmirozan1266]

thx for explanation!

[@DataScienceGarage]

Thanks for watching! :)

[@darkman8939]

thanks, very hhelpful.

[@kronlogic2408]

For the Iteration 2, shouldn't the second line be x2= and not x1= ?

[@denisplotnikov6875]

How to use this example for Stochastic Gradient Descent?

[@harshithbangera7905]

How we know -5 is global minimum...is there when gradient or derivative become 0

[@explovictinischool2234]

Hello, better now than never. Let's assume we have reached -5 at step Xn. However, we don't know that we have reached the local minimum. We perform another step Xn+1 with the formula, which gives: Xn+1 = Xn - (learning_rate) * (dy/dx) Xn+1 = -5 - (0.01) * (2 * (-5+5)) Xn+1 = -5 - (0.01) * 0 Xn+1 = -5 And so, we have Xn+1 = Xn which means we can not progress anymore and which means we reached the local minimum.

[@adscript4713]

@@explovictinischool2234 So we know whenever the result is not getting any smaller?

[@bharatcreations7154]

Can we compute same thing without getting into learning rate??

[@shankaks7217]

Why did we choose 0.01 as the learning rate?

[@diegososa5280]

Thank you very much!

[@DataScienceGarage]

Thanks! Hoping it was useful. :)

[@mattk6182]

using x as your means of showing multiplication is confusing, makes it looks like you took the derivative wrong with 2x(x+5)..maybe in future videos either leave the x out so the multiplication is implied.

[@boniface385]

Hi, why the learning rate are 0.01? Can it be any random learning rate? For example 0.2, 0.02 or any. I appreciate it for thee fast reply, thank you😊🙏🏻🙏🏻🙏🏻

[@DataScienceGarage]

Hello! Thanks for watching this video, I'm glad it was useful for you. While modelling ML system, you can specify random Learning rate. However the good practice is to use 0.1, 0.01, 0.001, or 0.0001. Each ML model has its own architecture, and different training data, hyperparameters, etc., so learning rate can be adopted separately for each case. Here, I used 0.01 just for demonstration purposes.

[@boniface385]

@@DataScienceGarage thank you so much for the explanation. 🫶🏻

[@مروةمجيد-ت1خ]

Well done 👏

[@DataScienceGarage]

Thanks! :)

[@supantha118]

Thank you so much

[@codingtamilan]

How you draw that curve can be fixed as -5 ? Always it is centre as -5 ?

[@blendaguedes]

First you decides witch will be his loss function. On his case it was (5+x)^2, or x^2 + 10x + 25. Nos you program the gradient descent to find the minimum of the function. It depends of your function.

[@codingtamilan]

@@blendaguedes thq... pleasure to meet you

[@thankyouthankyou1172]

Useful thank you

[@DataScienceGarage]

Thank you for watching!

[@tevinwright5109]

GREAT VIDEO

[@DataScienceGarage]

Thanks for watching this!

[@govardhan3099]

Great explained...

[@DataScienceGarage]

Thanks!

[@jimyang8824]

Good explanation!

[@kombuchad1237]

2:30 - is anybody else getting -12.04 for X1? Im realising know that if I can't do the arithmetic I shouldn't be trying to understand machine learning haha

[@moazelsawaf2000]

Thanks a lot sir

[@dtakamalakirthidissanayake9770]

Thank You So Much. Great Simple Explanation!!!

[@chenxiaoyu4379]

maybe change the x that represents multiplication into a dot? It is confusing to see in the dy/dx

[@AlfredEssa]

Good job!

[@arvinds7182]

On point👏

[@DataScienceGarage]

Thanks!

[@bernardaslabutis5098]

Ačiū, padėjo!

[@DataScienceGarage]

Džiaugiuosi! :)

Perfect !

[@preetbenipal1034]

its simple short and easy here ..thank you :)

[@prakritisinha9344]

Thank you!

[@DataScienceGarage]

You're welcome!

[@gireejatmajhiremath6751]

Thank you very much sir for clearing my concept.

[@RK-ro4br]

Why the learning rate = 0.01 ? it chosen randomly ? or what?

[@DataScienceGarage]

Hi! For this video - yes. It is chosen randomly for demonstration purposes only.

[@alidakhil3554]

The direction of minus got be on the opposite

[@DataScienceGarage]

Hi, could you expand your response?

[@grinfacelaxu]

Nice!

[@DataScienceGarage]

Thanks!

[@tremaineification]

The first iteration results in -3.03