3.4: Linear Regression with Gradient Descent - Intelligence and Learning

[@FANSasFRIENDS]

This is the amount of enthusiasm I need from my professor. Keep up the good work, sir!

[@josephgigot8827]

You are a really great teacher. Watching you, we are feeling that you re-discover what you already knows with us ! I think it is the perfect way to learn people knowledges !

[@Kevin-ex9vr]

man, this series with both board and coding together is really the best from yt, congrats

[@luisa534]

got stuck on gradient descent from the andrew ng coursera course, so as always, I'm back here for more digestable explanations. love your teaching style!

[@LearnWithYK]

Excellent. Love the way you present - enthusiastic, excited, but totally at ease.

[@sagauer]

Hey, I am watching your channel for the first time and I am amazed how good you explain things! I am a teacher myself and I find you very inspiring!

[@anubratanath5342]

This is the most intuitive explanation of linear regression. Thank you sir!

[@christophersheppard3249]

You single handedly made me go into cs. Thank you for your inspiration.

[@NickKartha]

2:35 spoiler for Avengers: Infinity War

[@Thronezzz]

Keep up the good work. Your teaching is the best, especially when it comes to complicated topics.

[@sues4370]

This was a great visual representation of SGD, thank you!

[@franciscohanna2956]

Great videos Daniel! Thank you! I started a IA course at college this semester (it's almost ending now), and this helped me to settle what I was studying. Keep it up!

[@niharika7631]

Dan i love how you get so excited to explain things.. So much to say! 😅 super cute. Plus so informative. I m glad I found this channel.

[@mkalicharan]

How awesome is this explanation, theory + programming is the way to go Coding train

[@mohammedsaeed7241]

Dude thank you so much for the intuition! many ppl don't bother going through that

[@Manojshankaraj]

Really awesome video! Thank you for making machine learning and math so much fun!!

[@niklasheise]

Its incredible when you display the error and guess values, my next try is to make a learning rate which changes depending on the numbers behind the comma. This tutorial is awesome!!

[@sarangchouguley6292]

Thank you Dan. Really you made this topic so easy to understand. Keep up the good work.

[@josephkarianjahi1467]

You are hilarious man! Best teacher on youtube for machine learning

[@kingoros]

Thank you for making these! Very informative!

[@TheCodingTrain]

You're welcome!

[@st101k]

I agree ;)

[@solomonrajkumar5537]

you are really incredibly awesome teaching Sir!!!!... there is no words say....

[@renelalla7799]

Thank you for your awesome and easy to understand explanations! :) But I have a question regarding the code from 18:08 Why can we see the line moving instead of being just in its final position? So far, as I can see it in the code, the drawline() method is called after the gradientDescent() method. What am I missing here?

[@juliekell9454]

Thank you for this. I was taking a coursera course on machine learning and got stuck on week one (incredibly frustrating!!) because half the math instructions didnt make sense. I had no idea it was so simple! I just passed week one. Thank you.

[@kwajomensah940]

Thank you so much! I've been wanting to go over statistics to start diving into ml and mv you've just made my day!

[@TheCodingTrain]

I'm so glad to hear, thank you!

[@gracelungu3646]

This channel is really an amazing place to learn high programming algorithm. thank you for the videos Mr shiffman.

[@TheCodingTrain]

Thank you!

[@aeroptical]

Sooo impressed by the white board being magically erased! I watched the live stream and thought it would be a total disaster; well, I'm beyond impressed - some fine editing there! :) Loving the ML series so far Dan.

[@arzoosingh5388]

I must say i like the way you teach . You're a nice man God bless .

[@SharonKlinkenberg]

Great videos Dan keep up the good work. The code really helps getting a handle on the theory.

[@TheCodingTrain]

That's great to hear.

[@miteshsharma3106]

the snap was cool ..... but we saw the truth in livestream lol😁

[@teja2775]

Awesome cool..... What a teaching style I really love it you made my day by understanding linear regression with simple story really love you man

[@syedabuthahirkaz]

Shiffman is always nice man. Love you Guru !

[@fernandonakamuta1502]

That is an awesome use of DOM man!

[@zhimingkoh1029]

Hey Dan, thank you so much for making all these videos (: You're amazing!

[@francescozappala8822]

Hi, I love your videos...I think they are amazing! I'm Italian and don't understand many words😕 you are great!

[@TheCodingTrain]

Thank you! I need to get more language subtitles!

[@Algebrodadio]

Are you going over gradient descent because it's used by the back propagation algorithms for neural networks? Because I can't wait to watch you do stuff with NN's.

[@TheCodingTrain]

That's right!

[@benjaminsmeding8966]

Hi Dan, I really enjoy your movies, I'm a self thought programmer and your movies give a real good insight in different kind of algorithms. Maybe nice to know... I'm actually a railtrack (P-Way) engineer and we use for example the least square method quite a lot. Keep up the great work! Ps. If your interested in some actual train datasets (from the Dutch Rail Network) leave a message.

[@TheCodingTrain]

Oh yes, that could be good!

[@varalakshmi3932]

Great videos! You are good at making videos by just being yourself and explaining in the best way possible. :))

[@crehenge2386]

thank you for showing me how to implement multivariable calculus in programming!

[@capmi1379]

Wow! machine learning!.. you gave understanding how they work and how they write by line by line without package unlike package like tensor flow XD Wow..thank u

[@junaid1464]

wonderful. nobody can teach better than you.

[@TheCodingTrain]

Thank you so much!

[@matteoveraldi.musica]

you're the boss. Very good explanation, loved it!

[@wengeance8962]

Dan is wearing a funky t-shirt! looks good!

[@rajcuthrapali800]

you are like my coding guru lol thanks so much mr dan for your help!

[@ElBellacko1]

great explanation

[@lakeguy65616]

velocity in this example doesn't mean speed? but instead means heading?

[@Contradel]

So my guess on an explanation on these lines: m = m + (error * x) * learning_rate; b = b + (error) * learning_rate; First line: think about the question "when I change m, how does that affect y?". This is what calculus is used for, more specifically differentiation. The answer to the question is written in math as dy/dm, if our line expression is defined as: y = m * x + b. dy/dm = D(m * x + b, m) = x. This is why the error should by multiplied by x. For the second line same thing! Change of y when changing b? dy/db = D(m * x + b, b) = 1. We could multiply error by 1, or leave it out as Shiffman did. What does the D function do? It differentiates the expression with regards to the second parameter passed. To calculate this you can either use a calculator, use a lookup table of rules or derive the answer yourself following the proof.

[@troatie]

This isn't quite right I don't think? Shouldn't you divide by x? Let's say your error was 1. So you want to change y by 1. If you change m by 1 you'll get a change of x! out of that. If you change m by 1/x you'll get the 1 out that you want. Or maybe written out... e1 = y - m1 * x - b1 e2 = y - (m1 + m_change) * x - b1 if you want e2 to be 0, then you get 0 = y - m1 * x - m_change * x - b1 = y - m1 * x - b1 - m_change * x = e1 - m_change * x m_change = e1 / x

[@Contradel]

I'm not sure I'm following you. But if, for one of the datapoints, the error is 1, you want to adjust the parameters (m and b) a small amount (learning_rate), weighted by error, so that for all your datapoints you get closer to a best fit.

[@amitbansode]

All videos by you are rocking

[@adammontgomery7980]

Would you have two separate learning rates for m and b? Seems like weighting the slope change higher could be beneficial.

[@vengalraochowdary4712]

Really superb explanation of Gradient Descent. Is there any book which you refer or suggest us for Machine Learning ?

[@darek4488]

You need separate learning rates for m and b. Then set the learning rate for b higher than the one for m so it would rotate faster, but move up and down slower.

[@8eck]

So the steer on the graph, it would be vertical line between Yguess and Yactual as a difference?

[@8eck]

So the so called steer is the delta of the weights? So called the change of weights in each iteration/epoch?

[@jairajsahgal5062]

you are a good man. thank u

[@tigerspidey123]

Would it be possible Applying PID control scheme to the learning rate, so it will accelerate our learning process?

[@adaptine]

What you're describing here is effectively a kalman filter?

[@nnmrts]

Hey Dan! I really like your videos, but sometimes you seem so lonely in that studio. :D Wouldn't be something like a co-op coding challenge awesome?

[@TheCodingTrain]

Hah, love this idea!

[@BinaryReader]

great stuff Dan, this stuff is invaluable for anyone starting out in ML. top stuff.

[@stefanoslalic2199]

can you host me?

[@PatrickPissurno]

You're really amazing! Thank you so much. Really enjoyed the way you explain things.

[@dukestt]

It worked yay haha. I was waiting for it. I was watching at the time though.

[@jadrima8640]

Nice tutorial channel!

[@gonengazit]

hey, nice video. could you explain why you normalize the values between 0 and 1 and what it does? i tried not normalizing them and i got some really wacky results using gradient descent even though it worked fine with the Ordinary Least Squares method. do you know why that happens?

[@gonengazit]

Julian atlasovich but it didn't work without normalization

[@yogeshpandey9549]

Would you please elaborate the implementation of Gradient Descent Algorithm using vectorization method in python?

[@TheCodingTrain]

Our Coding Train Discord is a great place to get help with coding questions ! discord.gg/hPuGy2g - The Coding Train Team

[@iftikhar58]

Cost function in this video is mean sqaure error?

[@YauheniKisialiou]

Hey! Great Video! But...how is it possible that the line is self adjusting...According to the code..

[@arijitdebnath4480]

why you multiply error * x by learning_rate??

[@thehappycoder3760]

Very helpful

[@gozumetaklanlar9274]

Hi Dan, greate video. I had watch most of your videos and I would be glad if you could make video about addEventListener and what advantages and disadvantages over onclick, onblur, onmouseover... thank you in advance

[@souravsarkar5724]

Dear sir, if you give any suggesion to understand that formula : "DELTA_m = error * x " , I will be very greatful .

[@ImtithalSaeed]

why you said x=data[i]*x and y=data[i]*y at 12:20

[@OneShot_cest_mieux]

Hello, there is a traduction of your description and your title in french. I live in France and I can't disable this, how to do it please ?

[@massadian75]

Very interesting !

[@TheNikhilmishras]

Great videos! :D You are the best! Do you recommend going with "Intelligence and Learning" sessions after p5.js introduction for someone who wants to get into Machine learning?

[@vishwajeetsingh6766]

Can someone explain why this is correct ? m = m + (error * x) * learning rate; I mean how is it dimensionally correct ? Shouldn't error be divided by x so that m can be added to something that is of type m.

[@michaelho9388]

I agree with you, I feel confused at this part as well.

[@nkemer]

yep I do not understand either.

[@nkemer]

oh it is in the next video.

[@PeterGObike]

The derivative from first principles allows for that. 2error*x= (error*x)*2(or any small number epsilon)

[@user-vl2oi3yr8q]

Hey cool video, I have an unrelated question, how can I transform strings or single letters no numbers, like A to 65 in p5.js?

[@gonengazit]

Felix Berg var s = 'A'; console.log(s.charCodeAt(0));

[@user-vl2oi3yr8q]

Thank you!

[@gonengazit]

Felix Berg you're welcome!

[@bellindaakwa-asare4442]

great video!! where can I get the code?

[@TheCodingTrain]

Apologies that it is missing, please file an issue here! github.com/CodingTrain/thecodingtrain.com/issues

[@jasdeepsinghgrover2470]

GREAT Video .. Thanks a Lot

[@jasdeepsinghgrover2470]

had a small doubt, shouldn't the change in slope be error/x instead of error*x?as it is rise / run

[@hunarahmad]

Your snap has inspired Thanos :D

[@blackdedo93]

Hey Thanks for The awesome video, i dont understand why not calculate the correct line directly ?

[@DaSodaPopCop]

The reason for this is because he is not simply writing a program that finds the correct line. He specifically is writing this program in such a way that implements and showcases the idea of back propagation. Calculating the line directly would be the most efficient way to write this program, but that's not the point of the video. There will be instances with much higher dimensional data where prognostication is much more efficient than doing what you suggest, such as in a Neural Network.

[@DaSodaPopCop]

look at 19:14 for his explanation

[@blackdedo93]

makes sense Thanks. but can u give examples or reference on why would i need this learning process

[@williamobeng4703]

Well explained. it will be nice to see the code. Cant find it on github

[@TheCodingTrain]

github.com/CodingTrain/website/tree/master/Courses/intelligence_learning/session3 (Need to figure out a way for things to be more findable!)

[@nikhilnambiar7160]

Make a video on lasso regression without library as did for linear regression

[@sujansonly007ify]

is the desired velocity given ? , when i already know which direction is my target.. why would i choose other side and steer it ? Could someone please shed some light on desired velocity

[@himannamdari7375]

I love this video Great Tnx Nice logo on your shirt

[@anuraglahon8572]

Where is the code ??i am not finding it in github?

[@Bena_Gold]

That "come back to me" ... hahahahaha

[@aakashkamalapur8510]

can anyone help me out in what way should I solve a system of over determined non linear equations?

[@iftikhar58]

Brother , Do you have any slack Channel or discord?

[@stefanoslalic2199]

But once again i don't understand how does NN know what the desired output is? You are calculating the Loss function based on desired output that you explicitly writing in the system? If you explicitly write out the numbers you are more or less telling the Neural Network what to do, isnt't the whole concept of neural network to find the way by itself?

[@TheCodingTrain]

Apologies for not making this clear. The technique I'm applying is called "supervised learning" where you have a set of training data with known outputs! The neural network learns how to reproduce the correct results with the known outputs so that it can (hopefully) produce the correct results also with data that doesn't have the answers paired with it. I think I cover this more in my 10.x neural network series.

[@stefanoslalic2199]

Got it! Thank you for your time and determination, see you in next episode (:

[@ac11dc110]

what is the best book for machine learning?

[@manojdude3]

2:41 Thanos of blackboard writings.

[@namanbhardwaj8230]

which platform does he using?

[@tejasdevgekar1154]

Really rookie right now... Gotta progress fast!

[@mohammadpatel2315]

The concepts in this are very similar to the perceptron model

[@toastyPredicament]

Sir, I wanted what they seem to have

[@akashsrivastava5963]

I need a code for linear regression for n variables in java

[@sreekrishnanr1812]

I think you are awesome 😊😊

[@NattapongPUN]

What if multiple regression?

[@charbelsarkis3567]

I would love to see the snapping of the fingers live :pp

[@adarshr3490]

You have done the snap even before Thanos have done that =D

[@frisosmit8920]

Maybe it would be cool if you made an AI for a simple game like noughts and crosses with a minimax algorithm

[@darkcaper703]

is this stochastic gradient descent?

[@TheCodingTrain]

Yes, indeed!

[@lucafilippini1348]

PID ? As always thx Dan...