The Lp Norm for Vectors and Functions

[@colinpitrat8639]

One point missing (or not stressed out enough) is that the higher p is, the more important is the contribution of large errors (e.g points far from the values to evaluate). On the contrary, the lower p is the higher the contribution of the small errors. So a large p will favour estimations that have small maximal errors whereas small p will favour estimations that stay close to the function overall allowing large spikes in places. This is particularly visible comparing L_1 and L_inf. L_inf will be high for a perfect match except for a single point being far from the function to estimate, when L_0 will be 0. On the opposite, L_inf will be small (=epsilon) for an estimate e(x) = f(x) + epsilon whereas L_1 would be epsilon*(b-a) where [a,b] is the integration domain (so large error).

[@DrWillWood]

Absolutely, spot on. thanks for taking the time to write this!

[@hudasedaki5529]

Thank you. That what I was looking for exactly.

[@joaofrancisco8864]

One of the clearest explanations I've seen for this topic. Thank you!

[@shafihaidery848]

1 year of confuse in my head solved in 7 minutes. god bless you well done

[@DrWillWood]

That's great, glad it was useful :-) Thank you so much!

[@madcapprof]

The Lp norm,as defined in the video is valid only if p>=1 (not merely p>0). The triangular inequality is not satisfied for 0

[@enterrr]

So for p

[@angeldude101]

p < 1 can still yield interesting results such as when plotting the unit "circle," though it technically doesn't give a "norm." Of course once you've started abandoning axioms like that, there's nothing stopping you for using p < 0 either. (p = 0 actually does give a valid _distance,_ but not a valid _norm,_ and only if 0^0 = 0, in which case it's the Hamming Distance.)

[@bdennyw1]

This has been invaluable! I’m understanding the Norms in machine learning now.

[@DrWillWood]

That's awesome!

[@ArshamMikaeili-xy1td]

OMG! This is the best video I've seen on this topic. Thank you so much for making math easy and enjoyable!

[@steveneiselen7993]

*Outstanding Video!* Informative with both mathematical and behavioral explanations alongside graphical examples; while being succinct and 'to-the-point' without overdoing nor underdoing the level of detail.

[@carlosunsolayag2728]

I was struggling to figure this concept out for a while. Thanks a lot!!

[@DrWillWood]

Thank you so much! Glad this helped :-)

[@amriteshsinha437]

Using Lp norms in machine learning now. I only ever understood it vaguely. This is the only time I understand this perfectly. Thanks for the video.

[@itscristianodasilva]

THANK YOU!! my linear algebra teacher is so confusing and absolutely abuses notation without visuals, this makes so much more sense

[@mmko7374]

heading towards my first exam block of mechanical engineering and this is really making things clearer. If I wasn't living off a scholarship I would ask if I could donate somewhere, I guess a subscription and likes will have to do for now

[@DrWillWood]

Thanks so much. It's great to know this has been helpful! All the best for your exams! :-D

[@emanualmarques5757]

Very nice and clear with visualizations. Thank you.

[@roshanshanmughan7218]

Amazing! Clearly explained! Thank you for the wonderful content!

[@carlosjesuscaro8274]

Excellent explanation, very clear. Thank you!

[@DrWillWood]

Thank you!

[@mertyazan]

Very clear explanation, thank you! I only wish I have seen your video before, that could have saved me a lot of time!

[@themandlaziman]

thanks for the visualisation here. my lecturers just throw theory at me oof

[@yashrocks31]

Very well explained. Thank you so much. God bless you!

[@NguyenTamDanK18HCM]

How are you so underrated? Your videos are clear with thorough explanation and nice visual!! Probably the maths on your channel are too niche for the general public haha

[@yidaweng7647]

Thanks for your explanations, very helpful!

[@DrWillWood]

Thanks, glad it was helpful!

[@fathimahida8878]

It was really easy to understand with visualisation ... Thankyou so much

[@menkiguo7805]

You expanded so well for me

[@vatandoust]

Thank you! Simple and effective.

[@DrWillWood]

Thank you for your kind words!

[@liamhoward2208]

Very clear explanation. You have one more subscriber! I also really enjoyed learning what a squwerkal is too!

[@glichjthebicycle384]

Very very helpful. Feeling a bit more prepped now for my analysis 2 exam c:

[@Via.Dolorosa]

great one, so super easy to understand, I would like to see a video about norms of a transfer function G(s)

[@limitstates]

Wow, such a great explanation.

[@HarshaJK]

Thanks

[@Sckratchify]

great video, it really help me out to understood better the concepts. new subscriber :)

[@DrWillWood]

Amazing, thank you! glad it was helpful :-)

[@r.f2173]

thanks this is great

[@basantmounir]

Fascinating! 🤩

[@giostechnologygiovannyv.ri489]

1:36 p = 0 is called pseudo-norm ;)) and measures the sparsity of the vector :D with that your video is complete :D

[@goddylincoln8590]

You're the best, thank you very much. Great work man. Just subscribed

[@DrWillWood]

Awesome, thanks so much! :-)

[@theodoremercutio1600]

A lovely video! I can't help but ask, where is your accent from?

[@SiriusFuenmayor]

very nice visual representation

[@DrWillWood]

thank you!

[@leticiaaires9123]

amazing video. thank you so much.

[@canyadigit6274]

8:28 why are we integrating? From my definition of the Lp metric we have a sum not an integral Nvm figured it out

[@isxp]

My text refers to these concepts as Matrix Norms. In the text, they give a 5th qualification, ||AB||

[@DrWillWood]

I think that matrix norms are a special case of the more general vector norms. In my opinion that inequality would be something that is true for matrices but not a requirement (i.e. something could be a norm without that being true but just so happens that it is true for all the norms we care about :-) ). I of course could be wrong with this though. The reason this inequality is more specific to matrix norms and not normed linear spaces in general is because the ability to multiply two vectors together is not always given for a normed linear space (you can add them together and/or multiply by constants). I hope that makes sense!

[@austinbristow5716]

How do you know what Lp norm you would want to use when comparing 2 functions? Would the desired Lp norm change for functions of 3 variables? How does the L3 norm differ in 3_D vs 2_D. Thank you!

[@canyadigit6274]

Another question. Why are functions included in Lp space? A member of the Lp space is a set of numbers (x,y,z,…) that satisfies some axioms, however functions aren’t a single set of numbers. They are multiple sets of numbers since each point of a function has a set of numbers (x,y,z…) describing it.

[@DrWillWood]

Hi! great question! I like to think of the Lp norm for functions as a kind of "limiting case" of vectors tending to infinity. This also relates to your earlier question actually. so imagining you have a set of n data points of a function over an interval eg (1,1), (2,2), (3,3), and we increase the number of points (1,1), (1.1,1.1), (1.2,1.2)... (3,3), not increasing the interval width beyond 1-3 in the x coordinate but the number of points between 1 and 3. Essentially, the limit as we reach an infinite number of data points is what a function is (at least that's how I like to think of it). At which point the sum becomes an integral. well, kind of, it works and that's the most important thing I guess! Now, regarding functions and Lp spaces, I think the crucial thing is the function being bounded in an interval otherwise lots of functions would have a norm of infinity which isn't useful. It might be a useful exercise to consider a continuous function over an interval with a given Lp norm (integral version) and ask if it satisfies the criteria of a norm (the 4 criteria at 0:58). Also to give context, I would say Fourier series are the most widely used application of norms involving functions over a given interval. Theoretically Fourier series finds the trig series (q lets say) which minimises the L2 norm of the function you want to approximate (f lets say) minus q, which is the minimum of sqrt(integral from a to b |f(x)-q(x)|^2 ). Hope this was helpful!

[@the_informative_edge]

Really a great video

[@DrWillWood]

Thank you!

[@pianochannel100]

1:04, why is this the case ? ||x-z|| leq ||x-y|| + ||y-z|| ? What does this mean?

[@tomkerruish2982]

Essentially, it means that there are no shortcuts; you can't get from one point to another in less distance by deviating to a third point rather than going directly. It's a holdover from defining a metric space.

[@pianochannel100]

@@tomkerruish2982 Ah! Thanks!

[@pianochannel100]

@@tomkerruish2982 So if im understanding correctly it means that a straight line is always the shortest distance between two points in the space?

[@tomkerruish2982]

@@pianochannel100 Yes. It's the triangle inequality.

[@pianochannel100]

@@tomkerruish2982 Right, well thanks Tom!

[@dansheppard2965]

Why is the L2 norm so ubiquitous in applied mathematics, even when there's no direct spacial aspect to the setup, for example in statistics and coding theory? I understand it's nicer to apply than L1 (especially near zero, which is an important point) but is there a good reason other than ease of use?

[@syedfaizan5841]

Need more and more from u

[@ishakakyuz8586]

Thank you!

[@pengfenglin6851]

Super good

[@zoheirtir]

Hi Dr Will, at 8.33 L2 is defined as a circle not square

[@sharonarandia3630]

Thank you so much

[@SIMITechIndia]

won't the l2 norm be a circle instead of a square at each point. i.e. similar to a making a 3D shape by rotating each point of g(x) about f(x) as center and then taking the volume of that shape. Of course we are then applying a square root after this.

[@dsagman]

excellent!

[@manishmiracle4317]

Love from 🇮🇳india

[@osamansr5281]

AMAZING THANK U

[@DrWillWood]

Thank you so much!

[@tomkerruish2982]

Does anyone ever use L_p norms for anything other than p=1, 2, or infinity? For example, the L_3 norm is well-defined, but does it ever arise naturally?

[@colinpitrat8639]

L_0 norm can be used sometimes in machine learning (in some particular places) to favour models that have many 0 components. It's simply the number of non zero components of your vector.

[@enterrr]

Norms for p

[@syedfaizan5841]

Thanks a lot

[@guilhemescudero9114]

Thanks!

[@brunomartel4639]

perfect

[@bugrayalcn6921]

Good old fashioned subscribe button

[@davidansi1683]

Woow...thanks

[@manishmiracle4317]

What a beautiful mind u have!

[@yodafluffy5035]

Thx 🙏😇

[@VaradMahashabde]

When I found out all my favorite metrics are actually siblings

[@arberithaqi]

You sound like the guy from Head Space

[@DrWillWood]

Love it! I can definitely see that :-)

[@T015-f1y]

U saved my ass! Thx

[@elnotacom]

Why Norm is called by "L"?

[@DrWillWood]

To be honest I've never even thought of this! I've no idea but I definitely feel like I need to find out now! thanks for the question.

[@peng.montreal]

how about l0 norm ?

[@DrWillWood]

I think Gilbert Strang discussed this in one of his lectures on matrix methods for data science. He argued that logically this should be the number of none zero components eg || (2,5,0) ||_0 = 2. unfortunately, it fails the triangle inequality so isn't technically a norm! the same is true for any Lp norm where 0

[@user-gh6wd5cv1h]

this is actually very confusing and misleading, why would |x| has to be 1??

[@aidandavis1780]

It doesn’t have to be- we are just exploring what shape it makes when |x|=1. This is a diamond under the L1 norm, a circle under the L2 norm, and a square under the L infinity norm. Try showing that if we have |x|=R for some R>0 we get the same shape, just scaled by R.

[@eldoprano]

Ah fk, squircles really exist!

[@user-sb6os]

will wood real

[@derblaue]

What about semi-p norms for p ∈ Q, 0 < p < 1? They also have a very interesting unit circle (for x ∈ R^n, (x)ᵢ ≥ 0)