Underdetermined systems and compressed sensing [Python]

[@fazelpour]

Dear Professor, We cannot thank you enough for sharing your knowledge with us, especially in a way that we, as non-experts, can get it. Your Python codes makes the lessons immediately usable for our daily problems. I wish you would one day, extend it to the numerical solutions for the Maxwell equations to simulate S-parameters for an interconnect structure consisting of power ports.

[@jairjuliocc]

It's awesome what you can do with linear algebra. Thank you for the explanation

[@Eigensteve]

Glad you think so!

[@cz5672]

Why is s=(Theta+)y called L2 solution? I thought there should be an extra L2 norm (||s||_2) to penalize before it can be called L2 solution. Am I missing something?

[@ivankwok9104]

the python code just type the function "minimize(xxxxxxxx)" and return the result. What is the principle of this function? Why do we calculate the answer of this kind of underdetermined problem?

[@arunrajiitbaero]

I wish i have such a lightboard with good acoustics. Simply the best way for teaching theory and math.

[@jorgevargas8458]

Thank for the lecture, one question: the algorthims that you implemented in python with the L1 norm is called basic pursuit rigth?

[@RussStukel]

Very cool stuff from a TAMS graduate! ha - all the best to you Dr. B.

[@Eigensteve]

Awesome, thanks Russ!

[@leif1075]

@@Eigensteve Thanks for your lecture series. Hope you can respond to my other message when you can.

[@germanvillalobos3728]

I'm not sure if this question make sense, but if s corresponds to the Fourier transform of x, would that mean that Psi technically corresponds to the inverse of the Fourier basis? If Psi corresponds to the Fourier basis, then applying Psi to a vector would technically correspond to taking its Fourier transform.

[@bulutosman]

Awesome videos. Thank you so much for your effort. But the python and Matlab links on the website direct me to this video again. How can I reach the codes?

[@germanyafricansoul8269]

deal steve, love your book, but in your book page 22, formel 1.26 , you may means B=X- avg(X ) ?

[@CsatiZoli272]

According to the histogram, there are many very small, but still not zero solution components for vector s after the L1 minimization. Are those tiny entries rounded to 0 to obtain a truly sparse solution vector (denote it by ss)? If done so, is it always guaranteed that the Theta*ss - y is small? I ask it because it reminds me of the process when a mixed integer linear programming is replaced by a mixed linear programming problem and the corresponding optimum can largely differ.

[@parskatt2971]

I guess the error is gonna be bounded by something like ||theta||*||s-ss|| (where ||theta|| is some matrix norm, either 1 norm or 2 norm). For fourier stuff with sampling you have a unitary matrix followed by discrete sampling which means that you would have ||theta|| < 1with some handwaving, so hence the error should be decently small I guess.

[@systemx6603]

Most algorithms out there for solving these type of systems employ a shrinkage operator to ensure that those entries do in fact become zero.

[@CsatiZoli272]

@@systemx6603 Thank you. Now that I know what keyword to look for, I can read more about it.

[@MikhailBarabanovA]

Brilliant as always! Thanks!

[@Eigensteve]

Glad you enjoyed it!

[@billybest5276]

these videos are so good

[@Eigensteve]

Thanks!

[@JessicaMcKenna-e8y]

Love these lectures, thank you!

[@marllonmoraes658]

Teacher, can you make the notebooks of this chapter available? This topic is very interesting

[@nischalsehrawat2130]

Hi Steve can you please do a series on time series analysis?

[@Eigensteve]

Absolutely. I have bits and pieces of this floating around, but maybe something more consolidated would be good.

[@nischalsehrawat2130]

@@Eigensteve Thanks. I started with your control series and I made my own Segway (studio.kzbin.infoLzBVJ7Rq4XY/edit). I can't thank you enough. Looking forward to time series analysis.

[@rito_ghosh]

How exactly does the L1 and the L2 norm make the difference in the values and nature of X? I would really like to know that. Exactly what happens and how- that is my question.

[@ivankwok9104]

same doubt with you

[@totifroti4515]

Awesome, brilliant 👏

[@amuprakash25]

could you please do a lecture on curvelets. Thanks!

[@germanyafricansoul8269]

and on page 23. Code 1.10 added the code für PCA coodinate would be helpful, otherwise we love you, greeting from germany :)

[@prashantsharmastunning]

its all coming together..

[@Eigensteve]

I love it when that happens

[@parskatt2971]

I think it would be more instructive if you actually generated y from some sparse vector

[@parskatt2971]

@@var67 yes i know, thats why i mean y should be generated through some sparse s times theta

[@CsatiZoli272]

How would that be more instructive?

[@parskatt2971]

@@CsatiZoli272 you could for example actually compare the true vector with the estimated one for both the l2 and l1 case

[@parskatt2971]

@@CsatiZoli272 Here I made a plot of how the solution actually compares (using n=500, p=200): i.imgur.com/q2aj4IR.png

[@CsatiZoli272]

@@parskatt2971 What is gt? It seems to be a manufactured solution for vector s with the first few entries being non-zero (i.e. 1). Did you use the L2 minimizer as an initial vector?

[@sdal4926]

Very good lectures but I have one critisim. there are many repetition during the class so makes difficult to follow. You have already explained y, theta and x. So you can directly go on.

[@fullmetalschizoid]

Why didn't you just use Lasso regression? Much faster