10. The Four Fundamental Subspaces

MIT 18.06 Linear Algebra, Spring 2005
Instructor: Gilbert Strang
View the complete course: ocw.mit.edu/18-06S05
KZbin Playlist: • MIT 18.06 Linear Algeb...
10. The Four Fundamental Subspaces
License: Creative Commons BY-NC-SA
More information at ocw.mit.edu/terms
More courses at ocw.mit.edu

[@corey333p]

"No mathematics went on there; we just got some vectors that were lying down to stand up."

[@corey333p]

Gotta know the bases for the spaces.

[@why6447]

AHAHHAHHAHAHHAHAH

[@delta_sleepy]

😂

[@GavinoFelix]

"But, after class - TO MY SORROW - a student tells me, 'Wait a minute that [third vector] is not independent...'" I love it. What other professor brings this kind of passion to linear algebra? This is what makes real in the flesh lectures worthwhile.

[@xoppa09]

Give that brave student a medal.

[@fanzhang3746]

xoppa09 I think here it is the Professor that's honorable . He elaborated on his mistake, which is reasonably embarrassing for him, and made clear important concepts. I think most others would just correct it, apologize, and move on. You can see his embarrassment when he used words like 'bury', and the reaction when he accidentally uncovered the board again later.

[@andersony4970]

@@fanzhang3746 I don't think he is much embarrassed. He talked about doing math in class in the first vedio of this series, if you've watched that. He said that it might be inevitable to make mistakes, and it's great to go through all the processes with the students including making errors and correcting those.

[@NazriB]

Lies again? FAS FUS Sheng Siong

[@sahil0094]

whats so passionate about accepting & correcting own mistake?

Thank you MIT, thank you Prof Strang.

[@PhucLe-qs7nx]

00:00 Error from last lecture, row dependent. 04:28 4 Fundamental subspaces. 08:30 Where are those spaces? 11:45 Dimension of those spaces. 21:20 Basis for those space. 30:00 N(A^T) "Left nullspace"? 42:10 New "matrix" space?

[@lokahit6940]

i am asking you because your's is the most recent comment? 1)at 9:15 how the column space is R^m? for mxn(m rows x n columns)matrix there are n colums so there are n column vectors so it supposed to be R^n right?

[@aarongreenberg159]

@@lokahit6940 Because each vector in the column space has m components. Yes, there are n vectors, but the number of components of a vector describes the dimensions of space its in. This is different once you get to a basis, where the number of vectors describe its dimension, but even that is a subspace of R^(# of components). So a two-vector basis where each vector has 5 components is a 2d subspace in R^5.

[@juansepardo2020]

I am a 4th year, double engineering student re-learning linear algebra so I can have a stronger basis for ML, DL and AI. Never in my college classes, or independent studying, have I been so amazed in the way a concept is introduced as I was when prof. Strang got to the computing of the left null space. The way this man teaches is just astonishing, thank you very much.

[@reganmian]

Have you checked out his newest book "Linear Algebra and Learning from Data"?. That plus "Introduction to Statistical Learning" given a foundation in programming, probability, and statistical inference is a killer combo. I'm a statistics graduate student wanting to specialize in ML. I've been watching these on 2x speed as a review

[@DanielCoutoF]

I am so fascinated by the way that professor G. Strang gives his lectures, he does it in such a great way that even a 5 years old boy could understand , on the side , teachers from my university make the subject so complicated, that even highly above the avarege students struggle to understand the concepts poperly.

[@andydidyouhear]

Daniel Couto Fonseca A 5 years old is a bit extreme:)

[@JadedForAlways]

+Daniel Couto Fonseca What about a 5 year old girl?

[@DanielCoutoF]

Only 5 years old WHITE BOYS I would say

[@JadedForAlways]

Are you joking? I can't tell

[@DanielCoutoF]

I guess it's more funny if you dont

[@serg303]

I want to write on that chalkboard with that chalk.

[@vabez00]

It seems quite satisfying indeed

[@Lets_MakeItSimple]

the chalk looked like a big stone

[@KaveriChatra]

"I see that this fourth space is getting second class citizen treatment..it doesn't deserve it"

[@NG-we8uu]

Kaveri Chatra by coincidence I read this exactly when he said it

[@alenjose3903]

@@NG-we8uu me too, i just read this while i was listening to it 😂

[@MrGameWWE]

Me too 😂😂

[@maoqiutong]

The second time to see nobody in the classroom. The camera man is really happy to be a VIP student I believe.

[@phil97n]

How can you tell? He seemed to be talking to audience

[@easterPole]

I'm into the fifth minute and wondering whether he made that mistake in last lecture knowingly

[@sachidanandprajapati9446]

man, exactly. Due to this error, i came to know if a matrix in non invertible, the columns would be linearly dependent

[@eduardoschiavon5652]

40:54 There's no one in the class...

[@ManishKumar-xx7ny]

Same thought and maybe he did. Great chance

[@matthieugrosrenaud1777]

@@eduardoschiavon5652 nah it's because they reduced the rows of the class, whtat we see are the rows of zeros.

[@GiovannaIwishyou]

I'm actually pretty sure he did this on purpose to trick the audience. Since first two rows are identical, it's too obvious when you learn that matrix must have the same number of linearly independent columns and rows (and it's a GREAT introduction to the lecture).

[@antoniolewis1016]

This man has dedication! Also, that girl in the beginning must have been a sharp genius.

[@ispeakforthebeans]

Bruh its MIT they got Gods in there you talk about sharp

[@akmalsultanov9801]

well, when you have an intuition of just row space and column space and connection between them, it's quite obvious and you don't have to be a genius to recognize the dependency of those row vectors. In fact, the first half of the linear algebra is relatively simple.

[@sreenjaysen927]

I think professor just made that up and he intentionally did wrong in the previous lecture just to introduce the row space. Professor just planned it like in "Money Heist"

[@leophysics]

@@sreenjaysen927 I agree

[@matthewsarsam8920]

Cant lie being able to pause the video and ponder about the ideas is so nice to have. Goes to show how much work those students had to put in

[@xiaohanwang3885]

For the first time I envy students in MIT. Because they have such genius lectures to attend.

[@NostraDavid2]

I don't. I've got it better. No time pressure to watch the lectures, I don't NEED to make the exercises, nor the exams. It's great! 😁

[@swatejreddy216]

@@NostraDavid2 and nor the hefty money too. So yeah.

[@jonathanoneill3464]

These lectures are saving my bachelors in Engineering. Thanks MIT!

[@rohanmalik895]

woah your icon image tells that very precisely that you survived engineering after all.....wish me luck

[@yanshudu9370]

Conclusion: Four fundamental subspaces of A(m*n), including 1. The column space means spanning the column vectors, which is in R to m, notation as C(A) 2. The nullspace of A means the free variables corresponding vector span the null space, which is in R to n, notation as N(A) 3. The row space means spanning the row vectors, which is in R to n, notation as C(A') equal to n-r 4. The left nullspace of A means the A' free variables corresponding vector span the null space, which is in R to m, notation as N(A') equal to m-r. other conclusions: The sum of dim(C(A')) and N(A) is equal to n, the sum of dim(C(A)) and N(A') is equal to m.

[@PyMoondra]

The end portion really educated how matrix algebra theory can be applied to computer vision; really glad he added that in.

[@davidwilliam152]

How a perfect thing that being able to be a great mathematician and a great teacher at the same time! Especially, being a great teacher is priceless!

[@Q.Mechanic]

It's my honor to have met you even virtually, sir!

[@duqueng]

The best teacher ever. I really admire the act of MIT. Like in a phrase in its website: "Unlocking Knowledge, Empowering Minds."

[@lokeshkumar-ub9bb]

at 3:15 - 3:20 Instead of looking at the row picture to realize the dependence we may also see that 2*(column 2) - (column 1) gives (column-3) :)

[@jacobm7026]

This is correct, but his mistake actually illuminates the importance of understanding independence from both the row space and column space. Most matrices wont be this easy to find column space independence so conceptualizing both of those spaces will give you a deeper, richer understanding of vector spaces in general

[@dhruvg550]

He explains in the first three minutes why you didn't even have to look at the columns. The girl who pointed this out was quick!

[@user-qq2gl9ep5d]

@@dhruvg550 I think the girl was Gilbert Strang himself

[@trevandrea8909]

Thank you so much!! Your explanation is soo amazing! Now I finally get why the column space of A and R are different, and why the row space of A and R is the same!! Btw, I'm saving 24:00 for the explanation of the subspaces of A and R

[@Cyraxsify]

At t = 38:00, Strang shows a way that expedites finding L: find E, then solve [E| I | to get E inverse which = L. Now we can quickly decompose A into LU if we do Gaussian elimination only--not Gauss-Jordan elimination--from the beginning. At t = 43:00, he defines a vector space out of 3x3 matrices, call it M_33. At t = 47:00, he covers the dimensions of subspaces of M.

[@bobmike828]

Correct me if I'm wrong but Strang was introducing abstract algebra at the end. Once you have all of these linear transformation transforming more linear transformations, you have an even greater transformation of space. Absolutely love this man

[@usozertr]

Bob Mike yes, and in an earlier lecture he was talking about how n x n permutation matrices form a group

[@pubgplayer1720]

Yes, abstract vector spaces are quite important in linear algebra

[@bfl9075]

I was totally astonished by the idea of computing left nullspace! Thank you Dr. Gilbert.

[@navs8603]

Thank you MIT for enabling us enjoy these treats.. And Prof. Strang is just pure genius

[@MAGonzzManifesto]

Thank you Dr. Strang and MIT. These videos are amazing and keeping me afloat in my class.

[@jingyiwang5113]

I am really grateful for your wonderful explanation about the four fundamental subspaces. My mathematics exam is tomorrow. It is a wonderful source for me to learn and refresh my memory. Thank you so much!

[@maximliankremsner633]

Thank you so much for this lecture series. This helps a lot! Great professor with great and easy to understand explanations.

[@LAnonHubbard]

Loved the bit at the end where he showed that upper triangular or symmetric or diagonal matrices form a subspace.

[@pianosdeaf]

16:35 how I want to feel after the exam when I screw up

[@ispeakforthebeans]

"Poor misbegotten fourth subspace" -Gilbert Strang, 1999 Remember when Elizabeth Sobeck decided to give GAIA feelings? These guys gave math feelings. And I love him for that. I didn't even know that was possible.

[@archilzhvania6242]

He makes everything look so clear.

[@DeLuini985]

Thank God for dr.Strang. I am understanding concepts that have eluded me for over a decade.

[@georgesadler7830]

Incorporating MATLAB commands in the lecture is a great way for students to learn about matrices and linear algebra in context. The overall lecture is another classic by DR. Gilbert Strang.

[@georgipopov2754]

Brilliant. This lectures connects the complex puzzle

[@shivamkasat6334]

A mathematician with Great sense of Humour. Mr. Strang !

[@yufanzhou9948]

The mistake professor Strang made turned into a great connection to the new topic. That's why he is a genius

[@All_Kraft]

Thank was great performance! Thank you MIT.

[@Afnimation]

It's interesting that he constantly regards on the fact that he exposes things without proving them, but in fact I think he explains the things so clearly an understandable that he does'nt need to prove them, because we can realize about them almost in an axiomatic way.

[@robertcarhart4168]

Strang proves things without you even realizing that you've just experienced a 'proof.' He makes it very conversational and intuitive.

[@stefanfarier7384]

I really like how he talks. He sounds so friendly in his explanations.

[@gavinresch1144]

It is amazing how he can do these lectures in front of no students and still be so engaging. In a way he is a great actor.

[@Lets_MakeItSimple]

There are students in back rows

[@chuckhei]

I really don't know what to say..... Satisfying? Grateful? OMG I just love it!!!!

[@Mike-mu3og]

45:26 transform an exclamation mark into an M. Brilliant!

[@serenakillion7008]

Thank you MIT and Professor Strang!

[@gavilanch]

So? This can mean a lot of things, and one of them is that they couldn´t tape this class and Strang had to repeat it in front of the cameras and they didn´t pay to some people to just sit right there so people like you would stop commenting that fact. Great classes, I do not speak english as native language, but certainly this is awesome, I really appreciate it So much Thanks to MIT and Professor Strang!!

[@ozzyfromspace]

*Question:* what is the relationship between rank(A) and rank(A^T)? Does rank(A) = rank(A^T) in general? The professor seems to be hinting at this, but rref(A) only preserves the column space, so it doesn’t seem so trivial to me. Any insight is highly appreciated. Edit: I found the answer. rank(A) = rank(A^T) by virtue of the fact that linear independence of the columns implies linear independence of the rows, even for non-square matrices. I proved this for myself this evening. The main idea for the proof (at least how I did it) is that if you have two linearly dependent rows, one above the other say, row reduction kills the lower one (reduces number of possibly independent rows). Killing off the row (making the row all zeros) also makes it so that the given row can’t have a pivot. Thus, we’ve reduced the number of potential pivot columns by one. That’s the relationship in a nutshell. The math is only slightly more involved

[@ostrodmit]

rref(A) does not preserve the column space, only the null and row spaces. It does preserve the dim(Row(A)) however, which suffices to prove that the row and column ranks are equal.

[@anikislamdu]

great lecture .i am so grateful to prof.gilbert

[@fuahuahuatime5196]

25:06 So performing row eliminations doesn't change the row space but changes the column space? So to get the basis for the column space, would you have to do column elimination for matrix [A]? Or could you take the transpose, do row elimination, and just use that row basis for [A] transpose as the column basis for [A]?

[@readap427]

+Pablo P That's what I was thinking as I watched that part of the video. It seems that approach would work. Before this lecture, it's the approach I probably would have used, but now that I see the tie-in to pseudo-Gauss-Jordan, I think I prefer pseudo-Gauss-Jordan.

[@aymensekhri2133]

Thank you Prof. Strang

[@DerekWoolverton]

I was nodding my head, keeping up just swimmingly, it all made perfect sense. He wrapped up the diagram and it seemed like we were done. Then he stepped over to the far board and replaced vectors with matrices and just turned everything upside down. Didn't see that coming.

[@shavuklia7731]

Oh cool. I've never computer the nullspace of the row space before. Initially, I thought of computer the nullspace of the columnspace of the transpose, but the method he provides - calculating E - is so easy, once you've already done all the work computing the other subspaces.

[@magdaamiridi7090]

Hello! Does anybody know any other lecturers like Dr. Strang with such passion in fields like convex optimization, detection estimation or probability theory?

[@q44444q]

Look up lectures by Steven Boyd. "Stanford Engineering Everywhere" is like Stanford's version of OCW and has some great courses in convex optimization: EE263 and EE364A. They aren't quite as good as Strang's lectures, but he's hard to beat!

[@nonconsensualopinion]

John N. Tsitsiklis has great probability lectures on MIT open courseware here on KZbin. Highly recommended.

[@LAnonHubbard]

Great video. Thanks Prof. Strang.

[@markymark443]

lol funny I'm just first watching this today and it was posted exactly 7 years ago xD thanks for the video, really helpful! I was struggling with this concept for my current linear algebra 2 course since I took the non-specialist version of linear algebra 1 which didn't really test us on proofs at all. I think I have a better understanding of the four fundamental subspaces now! :)

[@Ritam_404]

it's 7 years now !!

[@kaiding3322]

I believe Prof. Strang deliberately made the mistake at the end of Lec 9, in order to transition the focus from column space to row space. The transition was too smooth for this to be an accident. This is also a great show of humility that he didn't mind being perceived making a mistake!

[@xiemins]

May I say that the vectors in R span the same space as vectors in A after row operation because you can do a reverse ROW operation and construct the same vectors in A from R? It can't be true for column space because after row operations you most likely can't reverse and reconstruct the original column vectors from R through COLUMN combinations.

[@jenniferlai8752]

Great lectures on linear algebra!

[@ucrclxl]

What fascinates me are some stats you can find below this video. Maybe it's some bug but youtube tells us that this video is most popular among: 1) men 45-54 yo 2) men 35-44 3) men 25-34 Which I find really strange cause I've thought that most of viewers would be actual students. Also, popularity by region is interesting stat.

[@agarwaengrc]

where exactly can you find these stats? When I click on statistics I just get a viewcount graph

[@yourroyalhighness7662]

My, I feel so….dense. What a sense of humor this brilliant man must have to have penned a book entitled “Linear Algebra for Everyone”. Sir, I can’t even subract!

[@ozzyfromspace]

Worth mentioning: if row-reduction of the matrix generates the most natural row space basis without much effort, we can also generate the most natural basis of the column space of said matrix by doing row-reduction on the transpose of the matrix. This is all so incredibly fascinating!

[@ChandanKumar-ct7du]

Thank You Frof. Strang...

[@AkshayGundeti]

Thanx a lot Mr.Strang and MIT

[@guptaji_uvach]

Thanks Dr. Strang

[@marverickbin]

vector spaces of matrices! mindblow!

[@miladaghajohari2308]

I love these lectures

[@brogcooper25]

He is not only a master lecturer, he is a master of writing on a chalkboard. I swear, it looks like he is using a paint pen.

[@dariopl8664]

min 18:50 If it's helpful for anybody: the dimension of the null space is the same as the number of basis vectors that form the null space. Just like the dimension of a column space (or rank) is the number of linearly independent columns (i.e. vectors within the matrix), in the case of the null space, its dimension is the number of linearly independent columns, i.e. the number of basis vectors that form the null space.

[@thejasonchu]

thanks Prof and MIT

[@RomiiLeeh]

Thank you for sharing this video prof Strang!!! Very helpful! :D

[@Saket-op2xp]

26:15 here can we take basis as first r rows of A also , iff our elimination doesn't involve any row exchanges?

[@marcuschiu8615]

this is mind-blowing i don't fully understand it but i know it's mind-blowing

[@durgeshmishra9449]

@ 29:32 the prof said that the basis are the same, but that is not correct, right? Row space are same but with different set of basis for A and R?

[@bitstsunami9520]

suppose I'm in 3D if nullspace is a plane can we not simply write nullspace as an equation of that plane and every x,y,z be the possible values that give b of zeroes similarly if column space is plane and vice versa for row space and null space of A^t? p.s I do understand we can't write any of four subspace as a line in 3D because there is no equation of a line in 3D it's just the equation of the plane

[@ramkrishna3256]

For finding basis for N(A), Why can't we use similar approach of finding basis for left nullspace. 1) trans(A) - - - -> RREF 2) E' × trans(A) = RREF 3) finding basis from E'

[@MultiRNR]

Yes I have same question and this way sounds more mechanical (programmable) than earlier way

[@himanchalsingh1135]

Can anyone explain how "length of the linearly independent list ≤ length of spanning list"? TY in advance.

[@christophercrawford2883]

Nice lecture. Would like to have seen that N(A) and C(A^T) are independent (or even orthogonal!)

[@m1994m1]

Thank you so much Prof. Greetings from Jordan ^_^

[@ghsjgsjg53chjdkhjydhdkhfmh74]

😖😖 He's the best professor I know and yet my brain doesn't get it at once😂

[@nonconsensualopinion]

That's fine. All at once doesn't matter. What matters is "forever and always". Do what you must to understand it deeply so that you will know it the rest of your life. It may take watching the video many times and will probably require writing down some matrices and doing them yourself. Math is a subject which is hard to learn by observation; it really depends on participation. Remember, the students in the audience were MIT students, so they had proven they were quite talented. Those students saw what you saw in the video. Those students had the ability to talk to this professor after class. Those students had homework practice. Still, when the quiz was administered, I guarantee the average score was below 100%. Even after all that help, some students didn't quite get it all. They didn't get it "all at once". How can you expect yourself to do better than that, especially if you demand it happen "all at once"?

[@yiyu9519]

love this lecture

[@cathyyuqingyang2030]

Is there any proof on the column vectors where the free variables lie are independent to each other so that one can say the dimension of the null pace of a m by n matrix A is n-r ¿

[@cartmansuperstar]

the nullspace is not constructed from the non-pivot-columns.

[@fanggladys9986]

He is lecturing to an empty classroom if you look at time 40'53'' !! Even more wonders!

[@kevintoner6068]

How many blackboards do you want? Dr Strang: Yes

[@slowpoke7785]

Prof Strang said that C(A) != C(R). I'm wondering if this true, because the basis for C(A) are the pivot columns which we got from row operations...

[@onatgirit4798]

If all youtube content would be deleted today, the most upsetting thing for me would probably be losing this series of lessons.

[@AlexanderList]

Class is crowded these days, no worries. Don't know why no one is attending back in 2005!

[@omega7377]

It was actually in 2000. But it was uploaded to web in Spring 2005. The dates written in video titles are dates of upload not dates of record.

[@scoringwolf]

35:55 Size of identity matrix should be be nxn so that its conformable, shouldn't it?

[@BVaibhav-mt8jx]

he is so dam good at explaning! I love him!!!!!!!!!!!

[@abdelaziz2788]

40:50 is the best plot twist awesomee

[@gustav87]

This is so helpful, thanks alot!

[@Afnimation]

I don't know if if made my point, but if you see the other lectures you will understand better... in fact i just realize i made a mistake in the grammar at the end... next time i'll review before posting anything.

[@saadsaad77869]

How intersection of upper triangular matrix and symmetric matrix is equal to diagonal matrix

[@middlevoids]

Just beautiful

[@JohnPaul-di3ph]

My mind got blown when I realized you could get the basis for the left null space from row transformation. I mean, it seems completely obvious after he points it out but I never thought much of it until then.

[@alsah-him1571]

9:45 Professor Strang subtly integrates class consciousness into his lecture of the Four Fundamental Subspaces. Truly a genius.

[@bokumo7063]

Last hired First fired?

[@flowewritharoma]

great lecture

[@phil97n]

Great lecture thank you.

[@imegatrone]

I Really Like The Video The Four Fundamental Subspaces From Your

[@monazhang2377]

In the 3X4 matrix example, shouldn't the dim Null(A transpose) be 4-2= 2 instead of 3-2=1?

[@cartmansuperstar]

if A is 3x4, then A-tr(anspose) is 4X3 and dim (Null(A-tr)) = 3 - rank(A-tr) = 3 - dim (column-space(A-tr)) = 3 - row-space(A) = 3-2 = 1

[@karthik3685]

There is a problem with Dr. Strang's lectures. The problem is, he makes it so intuitive that I'm literally nodding in agreement the entire lecture. I've now watched the lectures once, read the book chapters, and watched the lectures a second time. And while I have a good grasp of everything discussed so far, they all sort of blend in. I couldn't list the things I learnt one by one for these 10 lectures. :D (well, I sort of can.)

[@johnk8174]

ya gotta do the problems. That pulls it together in your head.

[@encheng1136]

There are no students sitting there, but the lecture is still so good.