7. Eckart-Young: The Closest Rank k Matrix to A

[@adolfocarrillo248]

I love Prof. Gilbert Strang, he is a dedicated man to teach mathematics. Please receive a huge hug on my behalf.

[@yupm1]

What a wonderful lecture! I wish Prof. Gilbert a very long life.

[@kirinkirin9593]

10 years ago I took OCW for the first time and I am still taking it. Thank you professor Gilbert Strang.

[@amirkhan355]

Thank you for being who you are and touching our lives!!! I am VERY VERY grateful.

[@neoblackcyptron]

Really deep lectures, I learn something new every time I watch them again and again. These lectures are gold.

[@tempaccount3933]

Gil 3:30. Eckart & Young in 1936 were both at The University of Chicago. The paper was published in the (relatively new?) journal Psychometrica. Ekhart had already worked in the foundations of QM with some of the founders. And went on to work in Fermi's section on the Manhattan Project. If I recall correctly, Eckart married the widow of von Neumann. And ended up at UCSD. He was very renowned in applied physics including oceanographicy/ geophysics. Mr Gale Young was a grad student at Chicago. He also had a successful career taking his Master's from Chicago to positions in acedemia & US nuclear power industry.

[@mitocw]

Fixed audio sync problem in the first minute of the video.

[@turdferguson3400]

You guys are the best!!

[@darkmythos4457]

Thank you !

[@eljesus788]

Gil has been my math professor for the last 12 years. These online courses are so amazing.

[@JuanVargas-kw4di]

In the least-squares vs. PCA discussion that starts at 37:44, he's comparing minimizing sum of squares of vertical distances to minimizing sum of squares of squares of perpendicular distances. However, each vertical error is related to each perpendicular error by the same multiplicative constant (cosine of the angle made by the estimated line), so in a way, minimizing one is tantamount to minimizing the other. Where the two methods do seem to differ is that least squares allows for an intercept term, while the PCA line goes through the origin. However, when we look at the estimate of the intercept term ( b_0 = mean(y) - b_hat*mean(x) ), least squares appears to be performing a de-meaning similar to the first step in PCA. In summary, I think we would need a more thorough discussion than we see in the video in order to conclude that least squares and the first principal component of PCA are different.

[@xXxBladeStormxXx]

It's funny that this video (lec 7) has vastly fewer views than both lecture 6 and 8. But if the title of this video was PCA instead of Eckart-Young it would easily be the most viewed video in the series. That's why kids, do the entire course instead of just watching 15 minutes of popular concepts.

[@prajwalchoudhary4824]

well said

[@neoblackcyptron]

He has not explained anything about PCA in this lecture. He barely started out in the end and it wrapped up.

[@oscarlu9919]

That's the exact same thing I think about. I just followed up on the sequence of videos, and I surprisingly notice that this video is about PCA, which is closely connected to previous videos. But viewing previous videos makes the understanding of PCA far deeper!

[@KipIngram]

44:40 - No, it's not making the mean zero that creates the need to use N-1 in the denominator. That's done because you are estimating population mean via sample mean, and because of that you will underestimate the population variance. It turns out that N-1 instead of N is an exact correction, but it's not hard to see that you need to do *something* to push your estimate up a bit.

[@obarquero]

Well, indeed I guess both are saying more or less the same thing. This is called Bessel’s correction. I prefer to think that dividing by N-1 yields an unbiased estimator, so that on average the sample cov matrix is the same as the cov matrix from the pdf.

[@dmitriykhvan2035]

you have changed my life Dr. Strang!

[@Aikman94]

His passion, knowledge and unique style. He's such a treasure. An amazing professor and wonderful mathematician.

[@SalarKalantari]

33:54 "Oh, that was a brilliant notation!" LOL!.

[@georgesadler7830]

Professor Strang thank you for great lecture that involves Norms, Ranks and Least Squares. All three topics are very important for solid linear algebra development.

[@xc2530]

27:00 matirx A multiply orthogonal matrix, the norm of A doesn’t change

[@KapilGuptathelearner]

at around 37:15 when the Sir is talking about difference in Least Squares and PCA. I think minimization will lead to same solution as perpendicular length is proportional to other line. Hypotenuse * sin(theta), where theta is the angle between vertical line and the line of least squares which must be fixed for a particular plane(line). I could not understand where I am going wrong.

[@AmanKumar-xl4fd]

Where r u from

[@KapilGuptathelearner]

@@AmanKumar-xl4fd ??

[@AmanKumar-xl4fd]

@@KapilGuptathelearner jst asking

[@shivammalviya1718]

Very nice doubt bro. The catch is in the theta. Let us assume that first you used least squares and found out a line such that error is minimum and it is equals to E. Then as you said the error in case of PCA should be sin(theta) * E , here change in theta will have a direct effect to minimize error of PCA since it is in the product. So minimizing just E will not work, as you should minimize the product, and sin(theta) is also there. I hope you got i want to say.

[@AmanKumar-xl4fd]

@UCjU5LGbSp1UyWxb8w7wPE6Q u know about coding

[@nikre]

a privilege to take part in such a distilled lecture. no confusion at all.

[@tusharganguli]

Protect this man at all cost! Now we know, what an angel looks like!

[@JulieIsMe824]

Most interesting linear algebra lecture ever!! It's very easy to understand even for us chemistry students

[@SphereofTime]

5:56 vector norm matrices

[@GeggaMoia]

Anyone else thinks he talks with the same passion for math, as Walter White does for Chemistry? Love this guy.

[@prajwalchoudhary4824]

lol

[@jayadrathas169]

Where is the follow-up lecture on PCA it seems to be missing from the following lectures?

[@philippe177]

Did you find it where. I am dying to find it.

[@krakenmetzger]

@@philippe177 The best explanation I've found is in a book called "Data Mining: The Textbook" by Charu Aggarwal. The tl;dr. Imagine you have a bunch of data points in Rn, and you just list them as rows in a matrix. First assume the "center of mass" (mean value of rows) is 0. Then PCA = SVD. The biggest eigenvalue/eigenvector points in the direction of largest variance, and so on for the second, third, fourth, etc eigenthings. In the case where center of mass is not zero, SVD gives you the same data as PCA, it just takes into account that the center of mass has moved.

[@vasilijerakcevic861]

Its this lecture

[@justpaulo]

kzbin.info/www/bejne/m6qVgXhrrc5sY6M

[@vivekrai1974]

28:50 Isn't it wrong to say that Square(Qv) = Transpose(Qv) * (Qv)? I think Square(Qv) = (Qv) * (Qv).

[@xc2530]

44:00 covariance matrix

[@mathsmaths3127]

Sir You are wonderful and beautiful mathematician Thank you so much for teaching us for being with us

[@k.christopher]

Thank you Prof Gilbert.

[@johnnyhackett199]

@2:48 Why'd he have the chalk in his pocket?

[@Nestorghh]

He’s the best.

[@haideralishuvo4781]

Can anyone explain whats relation of Eckart Young theorem and PCA ?

[@lavalley9487]

Thank, Pr... Very helpful!

[@xc2530]

31:00 PCA

[@SphereofTime]

0:37 what is a pca(?

[@Zoronoa01]

is it my computer or is the sound level a bit low?

[@dingleberriesify]

I always thought the N-1 was related to the fact that the variance of a single object is undefined (or at least nonsensical), so the N-1 ensures this is reflected in the maths? As well as something related to the unbiasedness of the estimator etc.

[@obarquero]

This is called Bessel’s correction. I prefer to think that dividing by N-1 yields an unbiased estimator, so that on average the sample cov matrix is the same as the cov matrix from the pdf.

[@zkhandwala]

Good lecture, but I feel it only just starts getting into the heart of PCA before it ends. I don't see a continuation of the the discussion in subsequent lectures, so I'm wondering if I'm missing something.

[@rahuldeora5815]

Yes you are right. Do you know any other good source to learn PCA of this quality? Am having a hard time finding

[@ElektrikAkar]

@@rahuldeora5815 This one seems pretty nice for more information on PCA: kzbin.info/www/bejne/gl7Tgqd9opiJiZo

[@joaopedrosa2246]

@@ElektrikAkar thanks for that, I've wasted a huge amount of time looking for a good source

[@DataWiseDiscoveries]

nice lecture loved it @@ElektrikAkar

[@王海如-i1i]

How can I find the proof of Eckart-Young theorem mentioned in the video? Where is the link?

[@mitocw]

The course materials are available at: ocw.mit.edu/18-065S18. Best wishes on your studies!

[@xc2530]

4:26 norm

[@xc2530]

Minimise- use L1

[@xc2530]

18:00 nuclear norm- incomplete matrix with missing data

[@micahdelaurentis6551]

I just have one question not addressed in this lecture...what actual color is the blackboard?

[@Enerdzizer]

Where is continuation? It must have been on Friday as Prof announced. But lecture 8 is not that lecture. Right?

[@joaopedrosa2246]

I'm looking for it too.

[@baswanthoruganti7259]

Desperately waiting for that Friday for MIT to upload.....

[@SphereofTime]

7:09

[@naterojas9272]

Gauss or Euler?

[@sb.sb.sb.]

ancient indian mathematicians knew about Pythagors thrm and euclidean distance

[@Andrew6James]

Does anyone know where the notes are?

[@mitocw]

Most of the material is in the textbook. There are some sample chapters available from the textbook, see the Syllabus for more information at: ocw.mit.edu/18-065S18.

[@yidingyu2739]

It seems that Prof. Gilbert Strang is a fan of Gauss.

[@TheRossspija]

16:55 There was a joke that we didn't get to hear :(

[@pandasstory]

Great lecture! Thank you so much Prof. Gilbert Strang. But can anyone tell me where to find the following part of PCA?

[@mr.soloden1981]

Нихера не понял, но лайк поставил)

[@AndrewLvovsky]

xaxa

[@manoranjansahu7161]

Good. But I wish proof was given

[@kevinchen1820]

20220526 簽