Linear Regression and Multiple Regression

[@xavierfournat8264]

Fantastic work. Usually all tutorial videos about linear regression or multiple regression are simply giving the formulas out of nowhere, without explaining the rational in the background. Thanks for taking the time for diving through the underlying maths :)

[@martinsahmed9107]

This exposition is timely. I have battled over the disappearance of Y transpose Y in the matrix approach of a least squares for months until I came across this video. This is awesome. I am speechless.

[@speakers159]

Pretty amazing, especially since nobody really covers the mathematics behind ML, really appreciate the math based content.

[@CodeEmporium]

Yesss! Math is underappreciated

[@joelwillis2043]

this is a very well made video but this is always covered in statistics

[@arpitbharadwaj8799]

after multiplying and opening the brackets at 9:00 third term of the resultant should have transpose of B hat and not just B hat

[@chachajackson_original]

Correct

[@physicsfaith]

yes just a typo....

[@sidharthramanan6780]

This is a great video - I was looking for the math behind calculating the co-efficients in multiple linear regression and this explains it perfectly. Thank you!

[@CodeEmporium]

Thanks Sidharth! Glad it helped! Mind sharing the video to help others like you? :P

[@sidharthramanan6780]

Thank you for the video! And I'd love to share it with others :) Also, you just got a subscriber! Let's see you get to 1K soon !

[@CodeEmporium]

Thank you! Much Appreciated! I'm trying to upload more regularly than I have done in the past. There should be a lot more where that came from very soon.

[@sidharthramanan6780]

Yup! Any platform I can network with you on by the way? Quora for example?

[@CodeEmporium]

Sidharth Ramanan Quora is good. I'm under the name "Ajay Halthor".

[@sanathdas4071]

very well explained. I have been searching such video for many days. Now, the concept is crystal clear.

[@yuxiongzhu4249]

Wow! This is the best video to quickly understand the derivation of linear regression formulas!

[@bluejays440]

Excellent video, highly illuminating to finally see a comprehensive explanation of things that are too often left unexplained. I wish far more people, books, and videos explained statistics in similar detail.

[@ledzeppelin1212]

Needed some refresher on a math class from grad school, and this really hit the spot. Thank you!

[@amogh2101]

Is there something wrong at @8:58? Shouldn't B(hat) be B(hat)(Transpose)?

[@Anonymous-ho1mt]

I have tried many ways to find a decent derivation for multiple regression, I found the key term is understanding matrix derivation rules which I was missing all those times. this is first time I got the clear understanding of the formula. Thanks a lot.

[@jean-michelgonet9483]

In minute 10:27: X is mx1 and A in mxn. The 3rd differentiation rule is about y = X*A. But, given the sizes of the matrices, how can you multiply X*A?

[@noelcastillo3369]

I have the same question. Were you able to clear it up?

[@xiaodafei4738]

here the differentiation rule should be: let scalar y=x^T A, then dy/dx = A^T It's nice that the video shows some matrix differentiation rules, but I recommend the more serious propositions in: atmos.washington.edu/~dennis/MatrixCalculus.pdf

[@kerolesmonsef4179]

after week of searching . finally i found you . Thank you so much great explanation . keep going on

[@CodeEmporium]

The search is over. Join me in turning this world into -- nah just kidding. Glad you Finally found me. Hope you stick around

[@leochang3185]

At 11:10, the quadratic form of matrix differentiation should be x^T(A^T + A). Under the condition of A being symmetric could the derivative be 2 x^T A (as being used in the last term of d(RSS)/dx).

[@alphatakes]

this is the best video on multiple linear regression

[@CodeEmporium]

Thanks so much!

[@donaldngwira]

One of the best explanations on this topic. And the presentation is superb

[@rajeshsoma143]

9:40 How does the third formula work? Here, the dimensions of xA do not satisfy the condition for matrix multiplication

[@kuntsbro4856]

Thanks a lot. This is the most comprehensive regression video on KZbin.

[@CodeEmporium]

Kunt's Bro Thanks! Regression is an important topic, thought I'd take time explaining it

[@anandachetanelikapati6388]

Excellent explanation with precise terminology!

[@hyphenpointhyphen]

I am binging the concepts and might forget to like - great channel.

[@jugglingisgreat]

You explained this 1000000000000000000000000000x better than my professor. Thank you!

[@CodeEmporium]

Ryan Smith Thanks! So glad it was useful!

[@Rigou27]

There's a mistake at minute 9:00, the third term of the expanded version of RSS is -(beta' x' y)

[@trackmyactivity]

Amazing, thanks to the map you just drew I feel confident to learn the deeper concepts!

[@mustafizurrahman5699]

Splendid and now words are sufficiently enough for such lucid explanation

[@CodeEmporium]

Thanks for the compliments! :)

[@areebakhtar6422]

This is one of the best videos

[@JAmes-BoNDOO7]

Finally a video which makes perfect sense. Thanks a lot bro.

[@CodeEmporium]

Making sense is what I do :)

[@surajJoshiFilms]

9:44 Actually it should be 2AX if A is a symmetric matrix, am i correct ??,help me anyone please

[@MrStudent1978]

Very nice explanation! Very clear! I was looking for exactly the same.

[@ellenma3523]

beginning from 8:54 the RSS should have the third term as -(β_hat)^T X^T y instead of -(β_hat) X^T y, the transpose sign is missing here

[@nguyenkuro3827]

You're Right. And I think it should be:" y=x^T.A => dy/dx = A" .

[@CK-vy2qv]

Excellent! Very nice see the scalar and matrix approach :)

[@kastenkarsten]

Incredible video for the derivation!

[@FindMultiBagger]

Hats of for your efforts ! Really Fun way to learn algorithms, Please post more videos of other machine learning algo.

[@gcumauma3319]

Excellently explained. Very lucid

[@CodeEmporium]

Glad this is useful. Thank you :)

[@souravdey1227]

Extremely clear. Bang on!

[@ashlee8140]

This is a great video and explained things so clearly! Thanks!

[@CodeEmporium]

Thanks for the compliments! :)

[@capnic]

Wonderful video! very useful and clear!

[@juanitogabriel]

Nice video. What software do you use for writing that math expressions? I mean, is it editor equations from ms word? Thank you.

[@anujlahoty8022]

This video just made my day. Absolutely loved it...

[@elishayu8002]

Super helpful and very clear! Thank you so so much!

[@satyendrayadav3123]

Hands down my dawg❤️❤️ Very well explained

[@mike_o7874]

Great video, exactly what i was searching for, how did they get that matrix equation was exactly what i needed! thanks a lot man!

[@shyamyadav6842]

can anyone tell why (p-1)^2 operation instead of 2*(p-1) at 6:16

[@tuscarorastreet1564]

and shouldn't it be p instead of p-1 also?? since the B goes from B0 to Bp... which is p+1 number of beta's

[@user-gn7op1nq3d]

Thanks! You just saved my life!

[@CodeEmporium]

Raquel Morales Anytime. Saving lives is what I do.

[@swapnilborse3150]

Thanks....put more videos on regression analysis

[@CodeEmporium]

Glad you enjoyed it! Will think of more Regression based videos in the future

[@k_anu7]

1 question. The method that you described above is of normal equation as of andrew ng machine learning course. The other way to find coeff. are gradient descent, BFGS, L-BFGS etc. Correct me if I am wrong.

[@MrDiggerLP]

My friend, you Saved my Bachelorpresentation.

[@gaiacampo5502]

Minute 8:54. How can the third term be minus beta-hat X-transposed y?? I thought it should've been minus beta-hat-TRANSPOSED X-transposed y... can you help me?

[@CodeEmporium]

You're mostly right. Might have missed that transpose out. There is a lot to keep track of here. If matrix multiplication works out, then that's good :)

[@gaiacampo5502]

@@CodeEmporium thank you so much! your video is pure gold to me :) lots of doubts finally solved :)

[@CodeEmporium]

Glad to help :)

[@tsehayenegash8394]

really nice explanation you have deep knowledge. hoa can we minimize the error term?

[@carolyneatieno1927]

This was really helpful. I'm taking a unit on Data mining with no statistics background. Thank for sharing your knowledge 👊

[@cecilia1300]

wow good luck!!!

[@Dra60oN]

Hey, in 2nd example, you got y = xA, how can you even multiply those two when dimensions don't match? (m x 1) * ( n x m) , thus 1 != n Similar for 4th example where you got y = transpose(x) Ax ... I think A should be square matrix in this case (mxm).

[@CodeEmporium]

2nd example: y = Ax, not xA. 4th example: You're right here. x^T A x has shape (1 x m) *(n * m)*(m*1). This is true if n = m i.e. A is a square matrix. Good catch! Should have mentioned that. In the derivation, we use it with X^T X -- which is square.

[@Dra60oN]

Hey, sorry my typo,I was referring to the 3rd example, y = xA, not the 2nd one. And also are you sure that the last term B^T * X^T * X * B is the case of your 4th example. Because you can rewrite that expression as (X*B)^T * (X * B) and then it's a norm squared of matrix, and you say g(X) = X * B, and then you can apply derivative with respect to beta given by this formula: 2 * g(X)^T * d(X*B) / dX, which in this case would yield the same result, so after all you might be correct as well. All the best.

[@andrewzacharakis8583]

great job man !

[@baa3124]

thank you so much from korea

[@youcefyahiaoui1465]

In your logistic regression, I am not sure how you came up with the two exponents when you formed the two product of the product of p(x) and 1-p(x)

[@krishnachaitanyakr1237]

Very well explained. Thank you.

[@raihanahmimi8233]

27.6K Subscriber on 13 July 2020... is that close enough from your prediction?

[@amineelfarssi3902]

Very clear explanation … better than doing it by considering the projection on the model space and using the projection formula (t(X)X)^-1t(X)Y

[@benisbuff]

Great explanation

[@MOTIVAO]

Amazing work

[@rajinfootonchuriquen]

Something i don't understand is this: in simple linear regresión, you take the mean of square of error, but in múltiple regresión, what happend with taking the mean? X and y in the result fórmula have components with the mean?

[@akzork]

Can you upload a pdf of these formulae you shows in this video?

[@cghale3692]

when u removed the bracket...what happened to B transposition and X transposition while multiplying with Y? B transposition is not there just B is there ...the last line of simplification ?

[@xofjo6239]

why is that, I am very confused

[@P3R5I3dark]

At 8 56 last line, 3rd term shouldn t it be BTXTy instead BXTY? It also doesn t make sense bcs matrix sizes dont fit

[@CodeEmporium]

Nevermore You're right. Someone mentioned that in the comments as well. When you take the derivative wrt Beta, you get the same result. So that part of the video is incorrect, but the rest is fine.

[@P3R5I3dark]

Yea the thing is -yTXB=BTXTy so you obtain deriv(RSS)=deriv(-2BTXTy+BTXTXB)=2XTXB-2XTY=0. Wich lead us to b=XTY*(XTX)^(-1) . Ty anyways for video, it helped to understand better this method. Tomorrow i have exam from this method's aplication in System's models.

[@CodeEmporium]

Death Xeris yup. You're right again. Still, glad the video was helpful. Good luck with your exam!

[@anmoltariqbutt8987]

@@P3R5I3dark -yTXB = BTXTy? How? If we use this equality then the whole term will be eliminated

[@ahmedmarzouki3991]

thank you very much for this amazing video, it was really helpful do you have any other videos about : polynomial regression and non linear regression ?

[@NEOBRR]

Thanks, this is an amazing video. It was very helpful.

[@CodeEmporium]

Many thanks!

[@DM_musik-01]

Thank you very much it was very helpful

[@tomarkhelpalma138]

Way too cool!!! I am enjoying this video!

[@runethorsen6230]

Is it possible that there is a little error at 12.54 min. 3'rd term of RSS: beta should be transposed?

[@lucaslopes9907]

Thank you! It helped me a lot.

[@zachc5261]

Great video, thanks for your effort 😁 I just have two questions: 1. in the last RSS equation, why is T removed from beta_hat in the third term 2. how is y = xA feasible given x has dimension (m x 1) and A has dim (n x m) Appreciate your help please. Thanks!

[@MrKaryerist]

Great video! Probably the best in explanation of math behind linear regression. Is there a way to do multiple non-linear regression?

[@snehashishpaul2740]

Very nicely explained

[@PetaZire]

thank you, you also saved my life :)

[@ashokpalivela311]

Very well explained!! Tq❤

[@sergeyderbeko9954]

This video is so good, it explained several weeks of a course to me in 12 minutes smh

[@CodeEmporium]

Glad this is useful :)

[@deeptigupta518]

How did you get y=xA as A transpose ? As both x A doesnt have the dimensions to get multiplied?

[@deeptigupta518]

Can you give us the reference for the matrix differentiation used here?

[@nilankalord5974]

Great video, thank you!

[@techchit6797]

Does the inverse( X*transpose(X)) always exists in the formula? Why?

[@xeeeshaunali]

Which book you consulted??

[@DiegoAToala]

Great math , thank you

[@dellazhang5785]

Super good

[@tmrmbx5496]

Good job, thanks

[@chaosido19]

I come from Psychology and am following data science courses rn. The completely different way of approaching regression was a mystery to me, but this video helped me a lot. I do feel like I should practise stuff like this myself too, do you have any suggestions for places where to find exercises.

[@CodeEmporium]

Thanks for commenting and watching! Maybe a textbook might be good for establishing a foundation. You can check out the “Introduction to Statistical Learning”. Aside from that I have I lol playlist on linear regression, though I admit it hops around some concepts. It still might be worth your watch.

[@rahuldeora5815]

Hey great video. Can you suggest some mathematical test that explains ML with some debt as in this video?

[@1UniverseGames]

How can we obtain intercept and slope of B0 and B1 after shifting line l to l'

[@troxy1935]

why for calculate b_1, the 1/n, becomes n?

[@BHuman2024]

8.54 min: last line 3rd term, I cannot match, could anybody clear me, please?

[@yunqianggan2906]

Thank you so much, this video helps a lot :)

[@CodeEmporium]

Yunqiang Gan Thanks! Glad you liked it!

[@christophereicher3928]

Thank you!

[@mohannadbarakat5885]

why is sum(sqr(e)) = e^T * e

[@naveengabriel9368]

I did not understand the part where it explains "for n samples number of operations.." Can anyone explain that to me.

[@medalighodhbani5927]

Is there a video that explains how this "min arg" work graphically? like how it actually minimizes the residuals

[@norbertramaj3024]

....don't tell me you also have AMS 210 finals?

[@medalighodhbani5927]

@@norbertramaj3024 nope, im actually from Tunisia, not the states, but there are similar materials between AMS and what i study.

[@sfundomabaso3200]

where's the rest. please make a video on normal linear models (I'm using a book called An Introduction to Generalized Linear Models by Dobson). the book is so confusing please help

[@surajJoshiFilms]

Why y predicted=beta hat X?? Instead Beta not should also be included

[@purvanyatyagi2494]

Pls tell me that RSS is same as mean squared error

[@pulgasbaez6580]

You're the GOAT

[@CodeEmporium]

Thenks!

[@benharrison6182]

Oh boy, am I alone in seeing nothing but a page of squiggles at 4:47 ? My eyes glazed over..

[@Wisam_Saleem]

GREAT!

[@BlackJar72]

Good, but I think I need to review some general math and sit down to work it out -- solving its not hard, but its good to know why it works.

[@CodeEmporium]

For sure. One can always use builtin libraries to code it in a single line. But understanding why it works the way it does will help understand when to use it.

[@allensrampickal1997]

How to find 'ε' ?