Machine Learning | Gradient Descent (with Mathematical Derivations)

[@RanjiRaj18]

For notes👉 github.com/ranjiGT/ML-latex-amendments

[@gabelster3729]

You just helped me understand hundreds of web pages that talked about topics with no order. Thank you

[@sudiptodas6272]

What is great about this particular video is these concepts are explained well in many places like scattered dots , you connected the dots to paint the whole picture . an example for gradient descent included - very helpful .

[@RanjiRaj18]

Thank You for your valuable feedback 😊

[@donaldngwira]

You are such a great teacher. Concepts are clearly explained beginning with the basics and slowly easing into the most advanced level. Thank you

[@RanjiRaj18]

You're very welcome!

[@bibekitani8401]

You are the best. Thank you! Awesome interpretation. To anyone confused, when we should be minimizing based on slope v intercept. Here are the reasons: • Minimization based on slope happens when the angle or steepness of the line needs adjustment to better fit the data. • Minimization based on intercept happens when the line is generally in the right direction but needs to be shifted up or down. In most real-world scenarios, both slope and intercept are optimized together during each iteration of gradient descent. The cost function depends on both parameters, and gradient descent adjusts both to achieve the best fit.

[@yashdhawade5341]

The best and clear explanation I've ever listened about Gradient Descent. Keep up the good work!🙌

[@RanjiRaj18]

Awesome, thank you!

[@subramaniarumugam6902]

My god you are perfect I think your work should reach more audience your best and clear than the renowned ML yputubers. Applause Ranji

[@aryandeshpande1241]

this might be the most underrated explanation on youtube

[@amarnammilton]

its a very good description, The way you teach is humble and appreciatable.

[@RanjiRaj18]

Thank you very much!

[@sumanmondal5276]

Your hard work made the concept very easy to grasp. Kudos.......

[@rajapal9736]

Hi, Your video is helpful for beginners to understand the concept. One suggestion: In the very beginning of the video when you write the equation of your predicted line remember to mark it as y(cap) = mx(i) + c. It is not y(i) which is the actual data point.

[@Sagar_Tachtode_777]

Everything is so easy on this channel, great work Man!

[@RanjiRaj18]

Glad you like them!

[@alhassanturay7233]

Truest your the best. You solve my long time machine learning challenge.

[@vedanthbaliga7686]

This is what I pay my internet bill for! Thanks a lot!

[@chinmaysrivastava3212]

I was unable to understand this topic tried many videos but this was the most useful video thankss

[@kvv6671]

When my ML teacher teaching this , I felt I am learning some rocket science ,but you are teaching it felt very easy , thank you Sir😊

[@RanjiRaj18]

Glad to hear that

[@swaroopthomare7237]

Hey thank you so much for this content since I started studying regression using your videos , I became huge fan of yours

[@RanjiRaj18]

Thank you for your comment. Glad you like it ;)

[@maneetsaluja]

Great explanation with an example. This is the way to explain such concepts.

[@RanjiRaj18]

Thank you for the comment. Happy Learning!

[@classy_Girl8920]

perfect video for core concept understanding , amazing.. I love the explanation.. thankyou so much

[@sundar8147]

Thanks for the clear explanation sir

[@manishjain9703]

I am a beginner and as a beginner I was struggling understand the gradient decent concept. I have seen many videos on gradient decent however all of them skipped explaining the derivative part however you explained it very well both (total and partial) with solution . Thanks!

[@RanjiRaj18]

Great to hear!

[@vishaldas6346]

Man, you've won my heart, you kept it so simple, best way of explaining Gradient Descent. Can you please help me in using learning rate in the equation and number of steps used in gradient descent with an example.

[@theysigankumar1671]

Sir thank you very much, this has been so helpful since my course will only get tougher from here onwards and u helped me understand the basics

[@sonnyarulanandam]

Very good explanation of gradient descent

[@aarib8594]

Great explanation, thank you.

[@vigneshwar2897]

will the sign ( direction ) for calculating m and b at last change, from addition to subtraction if we take "y-pred - y" instead of "y - y-pred" lke you have done in cost function ? i saw at few articles where this was mentioned but it was not clear.

[@tusharsub1000]

As far as I know, gradient descent doesn't talk about solving 'm' and 'c' directly by putting them into two equations like the way you did it here..Because sometime the expression that we get after differentiation becomes so complex especially with logistic regression, random forest and other complex model in deep learning that solving 'm' and 'c' directly becomes extremely tricky and time complexity becomes very high...So Gradient descent talks about solving 'm' and 'c' by some trial and test method. Starting with some dummy value of 'm' and 'c' and putting those values in the equation of differentiation and check if the value of differentiation(say D0) for those 'm' and 'c' becomes 0 or close to 0. If not, then subtract that differentiation value(D0) from the old value of 'm' and 'c' and get a new 'm' and 'c' and again check if the value of differentiation (say D1) for those new 'm' and 'c' comes close to 0. Continue like this till you find that old 'm' - value of differentiation is very very close to 0. That 'm' becomes your actual 'm' and similarly the same thing to be done for 'c'.

[@RanjiRaj18]

This video was an intuitive way for understanding gradient descent for beginners. Anyways appreciate your time to quote about your understanding of GD.

[@ArunKumar-yb2jn]

@@RanjiRaj18 Good explanation. I think you have shown derivation for Ordinary Least Squares method. As far as Machine Learning is concerned it has to be slightly adapted.

[@maxpatrickoliviermorin2489]

He solved m and c in this case because it wasn't a very complex example. Only one independent variable. In a multiple linear regression it would have been much more complex.

[@anushadevi4937]

Thank you so much, I got a clear picture of the topic now.

[@RanjiRaj18]

You are welcome!

[@BADURELGADIR-dd2ck]

simple and useful lecture.. thanks

[@RanjiRaj18]

You are welcome

[@bhamidimaharshi]

explanation is simply awesome.....

[@MrAmarSindol]

killer explanation !! amazingly amazing !! thank you bro

[@anarkaliprabhakar6640]

sir u explained so well

[@lingadevaruhp5576]

Really amazing, thank you so much sir, keep rocking

[@RanjiRaj18]

Always welcome

[@shaun2201]

Hi @ranjiraj , at 21:37 u have given a wrong explanation in the partial derivative w.r.t C... d/dC (-C) will be -1 ,then why are you treating it as constant whereas in d/dc mx should be taken as constant.

[@manasagowrikottur8242]

Thanks for this tutorial sir..made it very easy and simple.

[@vasachisenjubean5944]

you earned a subscriber my man

[@Venomus658]

Thank you! Wish me luck on exam about it!

[@RanjiRaj18]

Good Luck 👍

[@helloworld2740]

really nice approch to teach thank you sirji

[@salmansayyad4522]

Thanks a lot sir! It was really helpful. Excellent explaination.

[@jayaprakashs4412]

Very good explanation. It would've been good if you could've explained the usage of learning rate usage to find a minimum point.

[@aaryan3461]

Great video man. Loved it.

[@trendhindifacts]

Well explained bro ❤ just bring another video for statistics and linear algebra 🎉

[@Shivendra7277]

Thankyou sir , Make more videos on machine learning concepts .

[@RajSingh-ik3og]

great explainnation

[@sarasijbasumallick4036]

can you please tell me why the curve is much sharp when you draw the graph with respect to j and c ? please tell

[@1234wellwell]

Thanks so much for the video. It helped me a lot.

[@Rambabukatta-ox6tc]

very nicely explained Bro

[@RanjiRaj18]

Glad you liked it

[@pursuitofgrowthwithtr]

your videos are really nice, good content and presentation...keep it up sir.

[@RahulTiwari-oe1ww]

Well explained

[@suhasrewatkar9001]

Best explanation sir

[@patelraj3140]

Thank you so much sir for such a perfect explanation....🙏🙏👏👏👏

[@shivammodi1105]

Lovely explanation

[@sherifbadawy8188]

one of the best

[@ramnarayan3323]

Thanks ...very well explained

[@OpeLeke]

excellent video

[@alirezasoleimani2524]

very nice explanation

[@praveenkumar-nh5qs]

Nicely explained.

[@dheerajverma189]

Sir linear regression is not used for classification as you said in satrting of video while explaining

[@ayushsingh-qn8sb]

great explaination

[@suryakrishna760]

What should i do if i am to apply learning rate of some value?

[@chaithanyack]

What type is it batch gradient?

[@umermehboob5630]

That is a explanation. I have one question, where would the learning rate be actually used in computation. Like in your numerical example. We found the outputs and calculated corresponding m and c. How does the learning rate is catered. Secondly, when we multiply learning rate with derivative, what does it gives us ?

[@sanusimuhammad7466]

i have this video over and over again, it the most satisfying video i have seen, in as much as gradient decent is concer, but i have questions, 1 what happen to the 2 tha became the multiple of the function as chain rules implies, then what happen to the no in the cost function. i know its mean squared errored thing. in my small assumption either of the values cant be thrown away just like that mathematically. please help with explanation.

[@naytunaung-y9v]

At 20.37, you discarded n/2. I would like to know why?

[@pBERA0_0]

if 2x = 0 , then it simplifies to x = 0/2 . i.e. x = 0

[@mahmoodapurbo5537]

Thanks bro.

[@sudhansumtripathy]

hi, sir do you have the python code using tensor flow or do you have any recordings of ML using TF

[@nileshpandey5724]

thank you so much sir

[@danielsehnoutek2016]

Absolutely the best explanation

[@danielsehnoutek2016]

If I got it your last example is analytical solution, but it couldn't been done everytime, then we use iterative solution with alpha learning rate?

[@nchoreanthony4294]

what about when someone is working with the learning rate?

[@dineshlogu9368]

can you please explain me why we are squaring the step at 4.36 . everything is clear to me except this one squaring step I cant able to understand..

[@sabeenao.m7388]

Thankyou sir 😊

[@nightsky5037]

why do we set the derivative equal to 0? i mean the gradient at the minima might not be equal to zero for all curves

[@testenma5155]

Hi Ranji Sir, i have a doubt. In 17:00 you have mentioned that we need to take derivative because we have two variables. And mentioned variables as x and c. But I think you were supposed to say m and c. later in 17:27 you mentioning about two parameters m and c. Please verify whether it correct or not. If i have pointed out wrong please apologize me.

[@RanjiRaj18]

Yes you are correct we have to take derivative wrt m and c

[@Nudaykumar]

Hi one question here: First derivative: xsquare =2x Second derivative = 2 ( replaced in same location) Third derivative = 0 While applying same in mean square error formula First derivative= I understood square to 2/n(•••) Second derivative: with respective to slope It should be 2/n I=1 to n (yi -xi -c) here I applied second derivative replacing. Since -mxi converts to -xi. But In ur explanation instead of replacing, you brought second derivative to starting as below: 2/n I= 1 to n -xi(yi-mxi-c) In the same way for intercept. One more At the end , what happened to 2/n? Please correct me if I am wrong.

[@RanjiRaj18]

2/n Σi=1 to n -xi(yi-mxi-c) this comes from chain rule watch this part carefully again in the video, this (y-mxi-c) is the result of (yi-mxi-c)^2 and now since we want to differentiate wrt to slope m again you take the derivative now you treat the y and c as constants and what's left is -mxi so you get -xi that's what you get and you multiply with this(y-mxi-c). 2/n is a constant say for any number, n=5; 2/5 =constant you eventually equate it to zero so it vanishes away. Hope now you understand!

[@aerogrampur]

keep up the good work !

[@OpeLeke]

can this method work for an equation with multiple slopes?

[@SrikantBhusan]

between the timestamp 21:39 to 21:45 you told that partial derivative of y(i)-mx-c with respect to c is 0 so only take minus sign which is wrong it will be -1 because of here c is not constant.

[@pavankumar8673]

Linear regression for classfication at 0:20???

[@aqharinasrin7002]

dear sir, I still cannot connected what is the purpose we do m = m - lambda * dJ/dm and c = c - lambda * dJ/dC

[@leninfonseca7129]

Yess .. exactly....plz explain the proof of these 2 equations

[@AdityaSingh-lf7oe]

Hi Ranji sir, I wanted to ask that if there are if our line is of the form M1*(feature1) + M2*(feature2).... Mn*(feature n) + c, do we have to follow same steps and calculate dJ/dm for all M1, M2...Mn?

[@RanjiRaj18]

Yes, one by one

[@iramarshad700]

So gradient descent is our cost function to calculate the error

[@suryatej839]

is it a sweet, in the middle of a hyperplane?

[@apoorva3635]

Why do we need partial derivative when we have the total derivative?

[@RanjiRaj18]

When there are relatively larger coefficients in your model, taking total derivate would be a diffuícult task and also to estimate the optimal parameter. Partial derivatives reduces the workload by keeping one parameter as constant and determine the other.

[@rameshthamizhselvan2458]

Excellent...

[@mihirnaik3383]

Thanks Buddy :)

[@mayank265memories]

Amazing lecture. x^n, will not have a 3rd order derivate to be 0, it will be n+1 order derivate.

[@dineshlogu9368]

Thanks you so much but I have small clarification regarding differentiate. why we are differentiate with respect to m , c & why we should not differentiate with respect to x to find out the value of y..

[@RanjiRaj18]

Because m, c are the weights that we want to determine which will give the best equation for curve fitting.

[@dineshlogu9368]

@@RanjiRaj18 thanks you so much for spending time to respond to my comment..

[@mariawilson6807]

Sir my maths is quite weak i want to start my career in data science i know that i can make my math strong but how should i start to learn maths for data science

[@RanjiRaj18]

Hello Maria, you can refer to websites like www.mathsisfun.com/ to learn the basics. Hope it helps!

[@mariawilson6807]

@@RanjiRaj18 thanks sir

[@mariawilson6807]

@@RanjiRaj18 Sir its very low level mathematics I am in sybsc it

[@sirajmotaung6930]

Thank you so much..quick question, when/how do we use the learning rate in this regard?

[@RanjiRaj18]

If I understood your question correctly then: When? learning rate is made use for convergence, it should not be neither too large nor too low just optimal, so that your traning process is complete. How? You can use learning rate schedule or can use optimizers like ADAM.

[@sirajmotaung6930]

@@RanjiRaj18 Yes, Alright thank you so much. Your vid was really helpful.

[@testenma5155]

How did the 2/n gone from equation when dJ/dm and dj/dc was assigbed to 0

[@RanjiRaj18]

2/n is a constant say you take n =5 so it becomes 2/5 so it's derivative is zero

[@testenma5155]

@@RanjiRaj18 Thank you Ranji

[@sandhu6355]

bro please ans this question why we are taking summation of c in one equation and not in other ---> one is 5C why

[@sandhu6355]

Plase ans

[@debrajnath6031]

The explanation of mathematical formula is absolutely fantastic. The explanation was about with single feature. But if we have multiple feature, what to be changed in the equation? Can you please let us know that. Thanks very much, and we will love to see this kind of videos shortly.

[@RanjiRaj18]

In case of multiple features or weights we have to conisder them individually by taking the partial derivtaive. This video is just a general idea of gradient descent. Hope it answers your question.

[@abhiaaron1715]

why at the end multiply the 2 nd equation with 5

[@RanjiRaj18]

To make equation balance on both sides for cancellation. Those are basic algebraic rules.

[@fpl8648]

thank you!!!

[@fpl8648]

It was very helpful, in writing a thesis. Could you also indicate some bibliography for citations

[@mariawilson6807]

Which level of maths is required 11th and 12th or degree level mathematics ?

[@RanjiRaj18]

Personally both Derivatives, differential equations, Matrices and vector concepts

[@mariawilson6807]

@@RanjiRaj18 thanks

[@shafiqahmad9057]

Sir can you recommend a book for machine learning with mathematical background please

[@RanjiRaj18]

You can refer the book by `Tom Mitchell`

[@shafiqahmad9057]

@@RanjiRaj18 sir please what is book name and if you share pdf link it will be better

[@RanjiRaj18]

@@shafiqahmad9057 You can check on google it is open source

[@shafiqahmad9057]

@@RanjiRaj18 thank you for very fast respomse

[@mohamedelbatoty]

Thanks Bro :)

[@mohdhashim2321]

sir explane krne ke baad thoda side ho jaya kijiye screenshot lena rehta hai

[@divyamohan6113]

thanks a lot..... :)

[@laodrofotic7713]

J = 1/(2*m) * sum (h(x)-y)^2. being h(x) the hipotesis and y the accurate value... at 3:37 you got them mixed up right? damn man.. no wonder people get confused