22. Diagonalization and Powers of A

MIT 18.06 Linear Algebra, Spring 2005
Instructor: Gilbert Strang
View the complete course: ocw.mit.edu/18-06S05
KZbin Playlist: • MIT 18.06 Linear Algeb...
22. Diagonalization and Powers of A
License: Creative Commons BY-NC-SA
More information at ocw.mit.edu/terms
More courses at ocw.mit.edu

[@bigfrankgaming2423]

This man single handedly saved my university algebra course, my teacher was just reading notes, he's actually expalining in a very clear manner.

[@jollysan3228]

I agree. > Just one small correction at 32:30: It should have been S * LAMBDA^100 * c instead of LAMBDA^100 * S * c.

[@Slogan6418]

thank you

[@ozzyfromspace]

The sad thing was, a few moments later he was struggling to explain things because even though he hadn't pinned down the error, he someone knew that something wasn't quite right. But he obviously had the core idea nailed

[@alexandresoaresdasilva1966]

thank you so much, was about to post asking about this.

[@user-px8iy5jj8q]

I stuck on this for like 10 mins, until I saw the comments here...

[@maitreyverma2996]

Perfect. I was about to write the same.

[@apocalypse2004]

I think Strang leaves out a key point in the difference equation example, which is that the n unique eigenvectors form a basis for R^n, which is why u0 can be expressed as a linear combination of the eigenvectors.

[@alessapiolin]

thanks!

[@wontbenice]

I was totally confused until you chimed in. Thx!

[@seanmcqueen8498]

Thank you for this comment!

[@arsenron]

in my opinion it is so obvious that it is not worth stopping on it

[@dexterod]

I think Strang assumed that A has n independent eigenvectors since most matrices do not have repeated eigenvalues.

[@charmenk]

Good professor with good old blackboard and white chalk teaching method. This is way better than all the fancy powerpoints that many teachers use now a days.

[@9888565407]

hey did ya benefit from these lectures

[@yusufkavcakar8744]

@@9888565407 yeah ı did

[@jiayuanwang1498]

I can't agree more!!!

[@kayfouroneseven]

this is a bazillion times more straightforward and clear than the lectures i pay for at my university. :( I appreciate this being online

[@bsmichael9570]

He tells it like a story. It’s like he’s taking us all on a journey. You can’t wait to see the next episode.

[@Tutkumsdream]

Thanks to him! I passed Linear Algebra.. I watched his videos for 4 days before final exam and I got 74 from final.. If I couldnt watch Dr.Strang's lectures, I would probably fail...

[@snoefnone9647]

For some reason i thought you were saying Dr. Strange's lecture!

[@eye2eyeerigavo777]

Math surprises you everytime...🤔 Never thought that connections between rate of growth in system dynamics, fibonacci Series and diagnalization of an INDEPENDENT vectors will finally boil down INTO GOLDEN RATIO OF EIGENVALUES at END! 😳

[@albertacristie99]

This is magnificiant!! I have no words to express how thankful I am towards the exposure of this video

[@eroicawu]

It's getting more and more interesting when differential equations are involved!

[@georgesadler7830]

From this latest lecture , I am learning more about eigenvalues and eigenvectors in relation to diagonalization of a matrix. DR. Strang continues to increase my knowledge of linear algebra with these amazing lectures.

[@cozmo4825]

Thank you a lot Prof. Strang you really made this course clear to understand, the way you teach these topics are superior, what really special about it is you put yourself in the position of your students answering their questions without them even asking, a true skill acquired not only by decades of teaching but actually having the mindset of a true teacher. Mr. Strang, words can't describe how good these lectures are its a true form of art. And thanks a lot for MIT OpenCourseWare for providing these lectures in good quality and free of charge, hopefully one day to I will pursue a master in my degree in electrical engineering in MIT.

[@christoskettenis880]

The explanations of this professor of all those abstract theorems and blind methodologies are simply briliant

[@BirnieMac1]

You know you’re in for some shenanigans when they pull out the “little trick” Professor Gilbert is an incredible teacher; I struggled with Eigenvalues and vectors in a previous course and this series of lectures has really helped understand it better Love your work Professor Gilbert

[@florianwicher]

Really happy this is online! Thank you Professor :)

[@kanikabagree1084]

This teacher made fall in love with linear algebra thankyou ❤️

[@meetghelani5222]

Thank you for existing MITOCW and Prof. Gilbert Strang.

[@sathviktummala5480]

44:00 well that's an outstanding move

[@RolfBazuin]

Who would have guessed, when this guy explains it, it almost sounds easy! You, dear dr. Strang, are a master at what you do...

[@Zumerjud]

This is so beautiful!

[@ccamii__]

Absolutely amazing! This lecture really helped me to understand better the ideas about Linear Algebra I've already had.

[@cuinuc]

I love professor Strang's great lectures. Just one small correction at 32:30: It should have been S * LAMBDA^100 * c instead of LAMBDA^100 * S * c.

[@starriet]

Nice catch!

[@jeffery777]

haha I think so

[@eyuptarkengin816]

yeah, i though of the same thing and scrolled down the comments for a approval. Thanks mate :D

[@SimmySimmy]

through single matrix transformation, the whole subspace will expand or shrink with the rate of eigenvalues in the direction of its eigenvectors, suppose you can decompose a vector in this subspace into the linear combination of its eigenvectors, so after many times of the same transformation, the random vector will ultimately land on one of its eigenvectors with the largest eigenvalue.

[@MsAlarman]

Mindbending

[@tomodren]

Thank you for posting this. These videos will allow me to pass my class!

[@dennisyangji]

A great lecture showing us the wonderful secret behind linear algebra

[@ozzyfromspace]

For the curious: F_100 = (a^99 - b^99) * b/sqrt(5) + a^99 , where a = (1 + sqrt(5))/2 and b = (1 - sqrt(5))/2 are the two eigenvalues of our system of difference equations. Numerically, F_100 = ~3.542248482 * 10^20 ... it's a very large number that grows like ~1.618^k 😲 Overall, great lecture Professor Strang! Thank you for posting, MIT OCW ☺️

[@Mohamed1992able]

a big thanks tothis prof for his efforts to give us cours about linear algebra

[@alexspiers6229]

This is one of the best in the series

[@rolandheinze7182]

Hard lecture to get through personally but does illustrate some of the cool machinery for applying eigenvectors

[@kunleolutomilayo4018]

Thank you, Prof. Thank you, MIT.

[@syedsheheryarbokhari2780]

There is a small writing mistake at 32:30 by Prof Strang. He writes (eigenvalue matrix)^100 multiplying (eigenvector matrix) multiplying c's (constants). It ought to be (eigenvector matrix) multiplying (eigenvalue matrix)^100 multiplying c's. At the end of the lecture Professor Strang does narrate the correct formula but it is easier to miss.

[@clutterbrainx]

Yeah I was confused for a very long time there

[@aattoommmmable]

the lecture and the teacher of my life!

[@Hindusandaczech]

Bravo!!! Very much the best and premium stuff.

[@uzferry5524]

bruh the fibonacci example just blew my mind. crazy how linear algebra just works like that!!

[@user-ld6fq4wi5e]

I have learned about the Fibonacci sequence in my high school, and it is so good to have a new perspective on the magical sequence.I think the significane of learning lies in the collection of new perspectives.😀

[@amyzeng7130]

What a brilliant lecture !!!

[@go_all_in_777]

At 28:07, uk = (A^k)uo, can also be written as uk = S*(Lambda)^k*(S^-1)*uo. Also, we can write uo = S*c as explained at 30:00. therefore, uk = S*(Lambda)^k*(S^-1)*S*c=S*(Lambda)^k*c

[@jeanpierre-st7rl]

Hi @ 29:46 Uo = C1X1 + C2X2 +C3X3... Is U0 a vector? If so, How can split this U0 in to a combination of eigen vectors? What is Ci ? If you have any info pleases let me know. Thanks.

[@coreconceptclasses7494]

I got 70 out of 75 in my final linear algebra exam thanks MIT...

[@cooperxie]

agree! that's what I plan to use in my teacing

[@abdulghanialmasri5550]

The best math teacher ever.

[@cecilimiao]

@cuinuc I think they are actually the same, because LAMBDA is a diagonal matrix, you can have a try.

[@veronicaecheverria594]

What a great professor!!!

[@ricardocesargomes7274]

Thanks for uploading.!

[@mospehraict]

@PhilOrzechowski he does it to make first order difference equations system out of second order

[@SamSarwat90]

I love you professor !!!

[@nguyenbaodung1603]

I read something on SVD without even knowing about eigenvalues and eigenvectors, then watch a youtube video, explaining that V is actually the eigenvector decomposition of A^TA. Which is extremely insane when I got to see this video oh my godness. Now even haven't watched your SVD lecture, I can even tell the precise concept of it. Oh my godness Math is so perfect!!

[@Afnimation]

well i got impressed at the begining, but when he stated the second eigenvalue i realized it is just the golden ratio... That does not demerits him, he's great!

[@benzhang7261]

Master Yoda passed on what he has learnt by fibonacci and 1.618.

[@ax2kool]

That was amazing and awe-inspiring. :)

[@dwijdixit7810]

33:40 Correction: Eigenvalue matrix be multiplied to S from the right. That has been made in the book. Probably, it slipped off Prof. Strang in the flow.

[@blondii0072]

Beautiful lecture. Thanks

[@praduk]

Fibonacci numbers being solved for as an algebraic equation with linear algebra was pretty cool.

[@kebabsallad]

@PhilOrzechowski , he says that he just adds it to create a system of equations.

[@user-pd1sx9tx4q]

MIT, thanks you!

[@jamesmcpherson3924]

I had to pause to figure out how he got the eigenvectors at the end. Plugging in Phi works but it wasn’t until I watched again that I noticed he was pointing to the lambda^2-lambda-1=0 relationship to reveal the vector.

[@bastudil94]

There is a MISTAKE on the formula of the minute 32:31. It must be S(Λ^100)c in order to work as it is supposed. However it is an excellent lecture, thanks a lot. :)

[@YaguangLi]

Yes, I am also confused by this mistake.

[@sammao8478]

Yaguang Li agree with you.

[@AdrianVrabie]

+Bryan Astudillo Carpio why not S(Λ^100)S^{-1}c ???

[@apocalypse2004]

u0 is Sc, so S inverse cancels out with the S

[@daiz9109]

You're right... it confused me too...

[@wendywang4232]

something wrong with this lecture, 32:39, A^{100}u_0=SM^100c. Here I use M to substitute the eigenvalue diagonal matrix. The professor said A^{100}u_0=M^100Sc which is not correct.

[@ozzyfromspace]

Did we ever prove that if the set of eigenvalues are distinct, the set of eigenvectors are linearly independent? I ask because at ~ 32:00 taking u_o = c1*x1 + c2*x2 + ... + cn*xn requires the eigenvectors to form a basis for an n-dimensional vector space (i.e. span the column space of an invertible matrix). It feels right but I have no solid background for how to think about it

[@roshinis9986]

The idea is easy for 2d. If you have two distinct eigenvalues and their corresponding eigenvectors, you don't just have one eigenvector per eigenvalue, the whole span of that vector (its multiples forming a line) are also the eigenvectors associated with that eigenvalue. If the original eigenvectors were to be dependent, they would lie in the same line making it impossible for them to scale by a factor of two distinct eigenvalues simultaneously. I haven't yet been able to extend this intuition to 3 or higher dimensions though as now dependence need not mean lying in the same line.

[@jeanpierre-st7rl]

@@roshinis9986 Hi @ 29:46 Uo = C1X1 + C2X2 +C3X3... Is U0 a vector? If so, How can split this U0 in to a combination of eigen vectors? What is Ci ? If you have any info pleases let me know. Thanks.

[@dalisabe62]

The golden ratio arose from the Fibonacci sequence and has nothing to do with eigenvectors or eigenvalues. The beauty of using the eigenvectors and eigenvalue of a matrix though is limiting the effect of the transformation to the change in magnitude only, which reduces dynamics systems such as population growth that is a function of several variables to be encoded in a matrix computation without worrying about the effect of direction or rotation typically associated with matrix transformation. Since eigenvectors and eigenvalues change the magnitude of the parameter vector only, the idea of employing the Eigen transformation concept is quite genius. The same technique could be used in any dynamic system that could be modeled as a matrix transformation but one that produces a change in magnitude only.

[@Arycke]

Hence the title of his *example* as "Fibonacci Example." Nowhere was it stated explicitly sthat the golden ratio didn't arise from the Fibonacci sequence, so I don't see where you got that from. The example has a lot to do with eigenvalues and eigenvectors by design, and is using a simple recurrence relation to show a use case. The Fibonacci sequence isn't unique anyway.

[@neoneo1503]

A*S=S*Lambda (Using the linear combination view (Ax1=b1 column part) of Matrix Multiplication), That is Brilliant and Clear! Thanks!

[@neoneo1503]

Also expressing the state u_0 to u_k as linear combination of eigenvectors (at 30:00 and 50:00)

[@fragileomniscience7647]

Well, it's - if you interpret it that way - just a basis transformation from the standard base (up to isomorphism, then just additionally multiply the transforms of the alternate basis) onto the eigenvector basis. Provided of course, that either the characteristic polynomial factors distinctly, or that geometric and algebraic multiplicity match (because then the eigenspaces distinctly span the vector space up to isomorphism; if they weren't, you'd just have a subspace as a generating system). For anyone who wanted one more run-through.

[@neoneo1503]

@@fragileomniscience7647 Thanks! =)

[@zionen01]

Great stuff. I was able to do my homework with this lecture. I will definitely be getting Strang's book.

[@davidsfc9]

Great lecture !

[@eugenek951]

He is my linear algebra super hero!🙂

[@noorceen]

thank you :)) you are amazing

[@shadownik2327]

Now I get it, so its like breaking the thing ( vector or matrix or system really) we want to transform into little parts and then transforming them individually cz thats easier as the parts get transformed in the same direction and then adding up all those pieces. E vectors tell us how to make the pieces and e values how to make the transformation with the given matrix or system. Wow thanks ! It’s like something fit in in my mind and became very simple. Basically this is like finding the easiest way to transform. Thanks to @MIT and Professor Strang for making this available online for free.

[@shamsularefinsajib7778]

Gilbert strang a great math teacher............

[@iDiAnZhu]

At around 32:45, Prof. Strang writes Lambda^100*S*c. Notation wise, shouldn't this be S*Lambda^100*c?

[@alijoueizadeh8477]

Thank you.

[@Stoikpilled]

awesome!! Greetings from Peru

[@hektor6766]

You can hear the students chuckling as they recognized the Golden Ratio. Didn't quite recognize it as (1 + root 5)/2.

[@healthfreak5438]

why are we representing mu(k) vector as the combination of eigen vectors in the problem of Fibonacci sequence?

[@jingwufang1796]

Great professor

[@starriet]

Notes for future ref.) (7:16) there are _some_ matrices that do _NOT_ have n-independent eigenvectors, but _most_ of the matrices we deal with do have n-independent eigenvectors. (17:14) If all evalues are different, there _must_ be n-indep evectors. But if there are same evalues, it's possible _no_ n-indep evectors. (Identity matrix is an example of having the same evalues but still having n-indep evectors) * Also, the position of Lambda and S should be changed(32:36). You'll see why by just thinking matrix multiplication, and it can also be viewed by knowing A^100=S*Lambda^100*S^-1 and u_0=S*c. Thus, it should be S*Lambda^100*c, and this also can be thought of as 'transformation' between the two different bases - one of the two is the set of the egenvectors of A. * Also, (43:34) How prof. Strang could calculate that?? Actually that number _1.618033988749894..._ is called the 'golden ratio'. * (8:15) Note that A and Lambda are 'similar'. (And, S and S_-1(S inverse) transforms the coordinates.. you know what I mean.. both A and Lambda can be though of as some "transformation" based on different basis.. and S(or S_-1) transforms the coord between those two world.)

[@deveshvarma8531]

I spent a few hours on the second point before figuring it out :(

[@eren96lmn]

43:36 that moment when your professor's computational abilities goes far beyond standart human capabilities

[@BalerionFyre]

Yeah wtf? How did he do that in his head?? lol

[@BalerionFyre]

Wait a minute! He didn't do anything special. 1.618... is the golden ratio! He just knew the first 4 digits. Damn that's a little anticlimactic. Bummer.

[@AdrianVrabie]

+Stephen Lovejoy Damn! :D Wow! AWESOME! I have no words! Nice spot! I actually checked it in Octave and I was amazed the prof could do it in his head. But I guess he knew the Fibonacci is related to the golden ratio.

[@IJOHN84]

All students should know the solution to that golden quadratic by heart.

[@ozzyfromspace]

Fun fact since we're all talking about the golden ratio. The Fibonacci sequence isn't that special. Any sequence F_(k+2) = F_(k+1) + F_k for any seeds F_0 = a and F_1 = b != -a generate a sequence that grows at the rate (1+sqrt(5))/2 .. your golden ratio. Another fun way to check this: take the limit of the ratio of numbers in your arbitrary sequence with your preferred software :) edit: that's a great excuse to write a bit of code lol

[@khanhdovanit]

15:02 interested information inside matrix - eigenvalues

[@leothegreat3]

thank you

[@chiaochao9550]

46:08 It should be F_100 is similar to c_1 * lambda1 * x_1. The professor missed x_1 here. But if you assume x_1 is 1 (which is the case here), then this is correct.

[@starriet]

Not actually. F_100 is a number and x_1 is a vector.

[@jasonhe6947]

absolutely a brilliant example for how to apply eigenvalues to real world problem

[@gianlucacococcia2384]

Can I ask you why A1 x x1 is lambda x1?

[@gianlucacococcia2384]

Zhixun He

[@dexterod]

I'd say if you play this video at speed 1.5, it's even more awesome!

[@abdulbasithashraf5480]

Trueee

[@ozzyfromspace]

1x all the way. I savor the learning ☺️

[@phononify]

very nice discussion about Fibonacci ... great !

[@technoshrink]

U0 == "you know it" First time I've heard his boston accent c:

[@zyctc000]

If any one ever asks you about why the Fibonacci and the golden ratio phi is connected , point him/her to this video. Thank you Dr. Strang

[@gomasaanjanna2897]

Iam from india I love your teaching

[@jojowasamanwho]

19:21 I would sure like to see the proof that if there are no repeated eigenvalues, then there are certain to be n linearly independent eigenvectors

[@joe130l]

so it seems like the professor emphasized the importance of the eigenvalue here, that's nice. but is the eigenvector of any importance? what's a good example of eigenvectors?

[@pelemanov]

@LAnonHubbard You don't really need to know about differential equations to understand this lecture. Just watch lessons 1 to 20 as well ;-). Takes you only 15h :-D.

[@hypnoticpoisons]

@ycao19 yes indeed!

[@stumbling]

7:33 Surprise horn?

[@mayurkulkarni755]

wtf was that xD

[@canmumcu9804]

probably grinding of a chair xd

[@dadadada2367]

the best of the best

[@richarddow8967]

beautifully simple how that Fibonacci worked out

[@codeo6246]

Can't you also use the determinant to figure out that Aᵏ → 0 as K → ∞ ? i.e. if det A < 1 then Aᵏ → 0

[@iebalazs]

At 32:32 the expression is actually S*Lamdba^100*c, and not Lambda^100*S*c .

[@cooperxie]

Agree!

[@LAnonHubbard]

I've only just learnt about eigenvalues and eigenvectors from KhanAcademy and Strang's Lecture 21 so a lot of this went whoooosh over my head, but managed to find the first 20 minutes useful. Hope to come back to this when I've looked at differential equations (which AFAIK are very daunting), etc and understand more of it.

[@rolandheinze7182]

Don't think you need diff EQ at all to understand the algebra. Maybe the applications

[@PaulHobbs23]

@lolololort 1/2(1 + sqrt(5)) is also the golden ratio! Math is amazing =] I'm sure the professor knew the answer and didn't calculate it in his head on the spot.

[@xl000]

Is there a practical use to matrix diagonalization, or is it just a cool trick like 3 balls Mill's mess or sword swallowing ?

[@gianlucacococcia2384]

Can I ask you why A1 x x1 is lambda x1?

[@mpbgd]

tnx for sharing this

[@drhf1214]

i'm still a little confused, what is u naught supposed to represent? Are those just the linear combination of the eigenvectors?

[@lucasm4299]

Nicole O Yes. They form a basis

[@lastchance8142]

Clearly Prof.Strang is a master, and his lectures are brilliant. But how do the students learn without Q&A? Is this standard procedure at MIT?

[@Zoro3120]

In the computation of the Eigen values for A², he used A = SʌSˉ¹ to derive that ʌ² represents its Eigen value matrix. However this can be true only if S is invertible for A², which need not be always true. For example, for the matrix below (say A), the Eigen values are 1, -1(refer previous lecture). This would imply that A² has only one Eigen value of 1. This would imply that S has 2 columns which are same (if it has only one column then it is no longer square and hence inverse doesn't apply) and hence non invertible. This implies that this proof cannot be used for all the cases of the matrix A. _ _ │ 0 1 │ │ 1 0 │ ¯ ¯ Is there something I'm missing here?

[@hinmatth]

Please check 17:32

[@utxeee]

So, the second component of u(k+1) is useless, right? The actual value is given by the first component.

[@thovinh5386]

Yep, you can use u(k) = u(k) and it still works.