Understanding Lagrange Multipliers Visually

When you first learn about Lagrange Multipliers, it may feel like magic: how does setting two gradients equal to each other with a constant multiple have anything to do with finding maxima and minima? Here's a visual explanation.
~~~
This video was funded by Texas A&M University as part of the Enhancing Online Courses grant.
~~~
The animations in this video were mostly made with a homemade Python library called "Morpho". You can find the project here:
github.com/mor...

[@scalex1882]

This is one of these things where you are sitting in university, getting fed the final formula with an absolutely insane proof of the formula that makes you question reality and when you see this video it takes no more than 10 minutes to understand the entire concept. Absolutely incredible, thank you so much!

[@lehninger2691]

Wait, you guys are getting an absolutely insane proof???

[@ico-theredstonesurgeon4380]

Why the heck dont they teach these things visually in university?? This video is literally higher quality education for free. It makes no sense at all

[@pyropulseIXXI]

You should start reading the textbook and doing the proof yourself. This stuff in the video is basically just straight from the textbook. As for visualizations, you should be visualizing this stuff in your head. If your 'learning method' is to just sit in lecture and let a professor program you, you won't ever learn anything, which is why you'll be confused all the time until someone basically does the learning for you (like this video).

[@ico-theredstonesurgeon4380]

@@pyropulseIXXI that's true but I would argue that sometimes visualisations really speed up the learning process, and teachers are often not the best at drawing.

[@ahmedbenmbarek9938]

​@@ico-theredstonesurgeon4380it is not free it is sponsored by a university. The main issue with understanding math is to have a teacher who really understands maths to begin with. Most math teachers are simple folks looking for a fat salary. Maybe themselves do not understand the concept so they simply regurgitate what another teacher did to them. Anyway all thanks to KZbin that allowed brilliant teacher to explain mathematics from simplest concepts to the most complicated ones.

[@GiulioDean]

I'm doing a PhD in aerospace engineering and never have I seen a video so clear on this topic. chapeau!

[@paganaye]

Chapeau = "hats off."

[@rintepis9290]

I am so impressed by how clear this video manages to explain the intuition behind the Lagrange Multipliers. The only part I had to pause and ponder is to show the gradient of f must be perpendicular to the level curve when the point is a local maximum on the boundary curve.

[@shouligatv]

Same, if anyone has an intuitive explanation, please do share it !

[@jozsefnemeth935]

@@shouligatv it was explained by the ball on the slope: a perpendicular barrier to the ball trajectory will stop the ball, hence the barrier is in the horizontal plane.

[@gdvirusrf1772]

@@shouligatv If you imagine the parametrized curve of the boundary of f(x,y), you'll know that the maxima/minima occur at points where the derivative of the parametrized curve is equal to 0 (the single variable calculus way of solving the problem). The thing is, if the derivative is nonzero, then it must either point to the right (positive derivative) or to the left (negative derivative) on the parametrized curve. But this must also mean the gradient vector on the actual function f(x,y) itself must _also_ point to the right or left! Another way to say this is that for a point on the boundary of f(x,y), any deviation in the gradient vector away from perpendicular _must_ imply that the derivative of the parametrized curve of the boundary is nonzero at that point, and hence it _cannot_ be a max/min. So only the points where the derivative of f(x,y) is perpendicular could possibly be a max/min.

[@sender1496]

It follows from the definition of the gradient. At a local min/max, the slope of f is zero along the boundary curve, meaning that f doesn't change in that direction. The gradient gives you the direction and magnitude in which a function changes the most and is thus perpendicular to this. In other words, if the gradient were to have a component in the "boundary curve"-direction (ie not perpendicular), then surely it couldn't have slope zero since f would be increasing/decreasing when wandering on the boundary.

[@jozsefnemeth935]

@@shouligatv another way to look at the problem: we search for points where a level curve of the f-surface is tangent to the constraint curve. The perpendicular to these curves belonging to the X,y plane will be the same. By definition, the gradient on the respective surfaces provides this perpendicular.

[@hatelovebowel4571]

this is fking amazing. The best explanation and Calculus should be taught with geometry, it is so clear.

[@richardvondracek496]

I have been waiting for this video my whole life. Although I did many calculations with Lagrange multipliers in my life It never clicked in my brain the way other things did. Close to half century old and you have just completed my brain. ♥♥ Thank you so much for this. ♥♥ Damn.. this feel good. You are my new hero!!

[@firstkaransingh]

I salute you for taking a complex concept and breaking it down to understand at a very basic level. More power to you.

[@leonvonmoltke7923]

I would like to say that it is not often that people explain things better than khan academy. Well done sir.

[@NemoTheGlover]

once you go past Cal I, khan academy content isint that great in my opinion

[@agrajyadav2951]

@@NemoTheGlover what

[@golchha_J]

KA video on this topic was crap

[@odysseus9672]

From the point of view of finding the minimization, lambda tells you nothing. If you're working with a Lagrangian, though, then the Lagrange multiplier tells you the force needed to maintain the constraint.

[@gaboqv]

It actually also tells you how a little change in the constraint could make this max much higher or lower, in economics this is important as optima with very high sensititivity could mean that having the correct measurements of constraints is paramount.

[@PacoCotero1221]

Its also, in microeconomics, the marginal effect of budget variations in utility + budget constraint problems in some instances

[@AnimeLover-su7jh]

It's extremely sad, how this and so many other (books, lectures, videos etc.) don't actually discuss Lagrange multiplier correctly as it was discussed by Lagrange himself.

[@spitalhelles3380]

You mean in french?

[@AnimeLover-su7jh]

@@spitalhelles3380 I meant the way the idea is discussed.

[@omargaber3122]

I can't believe I managed to understand Lagrange Multipliers after all these years!!!!!!! , how magical math is when it's understood, thank you so much

[@harshraj2575]

Can someone please explain how the gradient of F became parallel to the gradient of G? I know they share the same tangent plane so the gradients must be parallel but i was not able to understand the logic that was said in the video.

[@carultch]

In case you are curious for where the local maxima along the trail are, I'll tell you the answers. There are 8 total solutions, 4 of which are trivial to find. The 4 remaining solutions that require Lagrange multipliers to find are at: x = +/-1.052, y=+/- 0.7125 See if this is consistent on my contour plot.

[@ebenenspinne4713]

Awesome video. There is only one thing I find misleading here: at 7:10 you show the gradient vectors as vectors in the literal direction of steepest ascend, implying them to be 3-dimensional vectors. In my opinion this is misleading and gives a wrong intuition for the gradient that I myself had for a long time. Remember how the gradient is defined. Then it follows clearly that the gradient vectors are 2-dimensional vectors for a function like f or g which only has two inputs. It helps to visualize the graph of f or g in one's head as a plane with colours indicating the magnitude of the output and the gradient vectors pointing in the direction of "steepest ascend" of the temperature/colour. Then it follows clearly that the gradient vectors are 2-dimensional, perpendicular to the level curve and all in one plane passing through the level curve (as shown correctly at 9:55). *This should not change* just because we change how we visualize the graph/magnitude of the output of the function. 10:43 has the same issue. With this small correction/improvement, this video is very good!

[@피타코라스]

yes you're right i also think that is wrong! in 3 variable, gradient vector should be normal to the tangent plain.

[@krittaprottangkittikun7740]

This video is way underrated, it is very clear and nice!

[@SerpentineIntegral]

@joseph ramos Hey, hello! I still make new videos, but not on this channel anymore. I put all my new stuff on a new channel called Morphocular. You can find it here: kzbin.info/door/u7Zwf4X_OQ-TEnou0zdyRA

[@taravanova]

Does this only work for points of the surface of f(x,y) constrained by the equation g(x,k)=k? In other words, if you were looking for a maximum using this method, would it give you: 1) a point on the boundary or 2) the point in the center of f(x,y)? I suspect 1), and you would get 2) if the constraint was g(x,k)

[@merajhossainpromit6152]

imagine studying in this university...

[@clarenceauerbach7934]

isn't it great ?

[@프로틴요플레]

The first thing I come up with when considering Lagrange Multipliers is that it is a pure hella substitutions if the number of constraints are less than the number of dimensions..

[@hereigoagain5050]

Amazing graphics really help to understand Lagrange Multipliers. My middle name must be "Lambda" because I don't contribute to the solution :)

[@Ganerrr]

would solving like (lim m->k {d/dz (f/(g-m))}) = 0 work?

[@derrick20]

A neat way to conceptualize this idea is to think of the constraint function as a filter of sorts, since we know every point along the constraint curve has a gradient perpendicular to the curve (this can also be understood in the sense that everything is a local extremum, since they are all equal, so the direction of max increase shouldn’t be biased to either side similar to the ball analogy in the video). So, when setting the gradients of the two functions equal, we just filter only the extreme in the objective function

[@user-dz9eb7fu2f]

Very clearly explained, this clarified a lot for me thank you so much

[@yendrian44]

Holy shit when you said that lamda in this case is called the Lagrange multiplier I could literally feel the creation of new neuron connections in my brain. This video is a masterpiece

[@mehdiardavan]

Fantastic video. Well visualized and explained. I was just wondering what you used to make the graphical effects while showing LaTeX formula rotate in 3D?

[@gohanmineiro]

Simple, clear, and concise explanation. Kudos.

[@qwerasdliop2810]

Absolutely incredible! Can't believe something so simple yet incredible was fit into such a simple set of equations, just under the surface!

[@breitbandfunker4332]

best video for understanding lagangian multipliers - now i understood it :-)

[@gergerger53]

That whole framing in terms of terrain, seas and what counts as the shoreline are fantastic metaphors to aid the conceptual understanding of this method. Very, very well represented, here.

[@StarContract]

In my opinion, good mathematical education should strive to develop your mathematical intuition, which in turn you would be able to turn into formality. This video is literally perfect.

[@NoNTr1v1aL]

Absolutely amazing video! Subscribed.

[@vkessel]

At 10:39 I could imagine a counterexample by deforming the surface. Realized the deformation would result in partial derivatives that don't exist because they depend on the direction of the limit. Mentioning in case someone else runs into that line of thought.

[@nagarjun385]

I'm still suprised how people from 16th and 17th century managed to figure this out.

[@omaraissani6255]

Those ppl were 10 times smarter than we are. I wonder what would have been achieved if great mathematicians such as Euler, Gauss and Lagrange, etc were still alive. Flying cars may be ?

[@raulsimon2218]

Very good. However, as I see it, del g is the orthogonal projection of del f onto the xy plane.

[@SCALER]

Hey, nice video, could you tell what animation tool you use for the animations here?

[@SerpentineIntegral]

Thanks! The animations were made using a homemade Python library called "Morpho". You can find the project here: github.com/morpho-matters/morpholib

[@dufrain79]

A very good informative video for beginners in optimisation. Very good entry level for understanding Lagrange Multipliers. Such a beautiful use of the Morpho library under Python.

[@zhuleung2938]

excellent work. you've just made me understand what confuse me throughout my whole collage life.

[@CG119Animator]

That explanation was stellar! You broke down a tough concept without frying anyone's brain cells.

[@klevisimeri607]

This video is more valuable than gold!

[@anthonytafoya3451]

Wow! Thank you for this video. Visuals GO A LONG WAY my brother. Cheers and you have a new subscriber :)

[@JulianHarris]

Outstanding. Just spent a whole morning trying to understand these things and the visualisations really really crystallise the relationships. Obviously this is an advanced topic and the prerequisites involve simultaneous equations, a little bit of linear algebra and partial derivatives. But once you’re in that position, I think this is possibly the best way to understand Lagrange multipliers.

[@pyropulseIXXI]

The concept is quite simple

[@GeoffryGifari]

i almost feel guilty watching this without paying tuition

[@JanPBtest]

10:27 Wouldn't a better way to say it be that the local max point (on the red curve) is tangent to the level curve of _f(x,y)_ through that point? This way the perpendicularity of the gradient of _f_ would be obvious, given the previous explanation of the perpendicularity of the gradient of _g._

[@gusmoraless]

I did not understand why the gradient vectors are depicted for moments as 3D vectors. They must be represented as 2D vectors, completely contained in the XY plane, because this region contains the domain of z = f(x,y).

[@gusmoraless]

I understand the physical interpretation of grad(f) over a solid surface, a "mountain"... but this vector must point to the direction *in the domain* of f for which f increase or decrease at maximum rate. Great work, great explanation except for this detail which could cause noise in students.

[@chamnil8666]

very very useful and amazing explanation.Thank you so very much.

[@meirgold]

Excellent and clear explanation. Thanks very much!

[@alexanderjonsson5749]

Damn this was a good video

[@harrymorris5319]

4:07 for Lagrange multipliers to work - need to have the constraint expressed as some expression involving x and y set equal to a constant x^2 + y^2 = 4 6:57 8:33 10:30 11:20 The max or min of a function f(x,y) which has a constraint g(x,y) = k must occur where ∆f (gradient of f) is parallel to ∆g (gradient of g) . If two vectors are parallel one is a scalar multiple of another. So ∆f = λ ∆g and λ the scalar multiple is called the Lagrange multiplier How to solve 12:13

[@sepehr__byt]

The video was phenomenal and truly amazing; thank you for providing such valuable content!

[@federicoferraro7080]

Even yhough I knew the answer, this helped to visualise the concepts and even helped me make links with other concepts (fluid mechanics). So thanks a lot !

[@ClearMath1]

does anyone know the equation of the curve in 06:09?

[@Mathematics_and_physics]

It is worth noting that g(x,y)=k defines some differentiable manifold , and the gradient vector is expanded in terms of the basis of the orthogonal complement to the tangent space of the manifold.

[@randvar2952]

What does it mean that the gradient of g at a point (a,b) is perpendicular to the constraint (level) set (curve) g^{-1}(k)? It means that the g-surface gradient is perpendicular to the level curve tangent vector, meaning (grad g)(a,b)•(x’(t),y’(t)) = 0, where t is such that (a,b)=(x(t),y(t)). That is, the directional derivative (rate of change of g) in the direction of the tangent vector of the level curve is 0. That is intuitive, as the g-surface gradient points in the direction the value of g grows fastest, while moving on the level curve in its tangent vector direction the value of g does NOT change (‘cause the curve is, well, level!)!

[@BarryKort]

In order to actually find the extremum of a function subject to constraints, it's typically necessary to determine the actual values of the Lagrange multipliers. One of the better behaved algorithms is to replace the scalar Lagrange multiplier by a convex curve which can be adjusted by means of an iterative solution process. This method, known as the Generalized Lagrange Multiplier Method is mathematically related to another important branch of mathematics called Duality Theory. Such Primal-Dual Methods were explored by myself and Professor Dimitri Bertsekas in the early 1970s, when we were both at Stanford University. The resultant algorithm is spelled out in one of Dimitri's textbooks on the subject of Optimization Methods.

[@laodrofotic7713]

This is a good video, congratulations on helping millions around the globe with this.

[@charlesspringer4709]

Worse explanation I have seen in ages :-) All the rhetorical questions that distract the concentration and then, "guess what? This isn't the surface I have been talking about, it is this different one. Surprise!" and more than once. I'm bookmarking it as a very good example of pedagogy gone wild.

[@HiepNguyen-ud8qe]

love it

[@valentinlishkov9540]

Issue: What is a differential of an irrational argument? Let a= some rational approximation, and A be the irrational number itself (if that makes sense). Then A - a > dA and there is no way a + dA > A can there be a length commensurate with all lengths (differential of length)? then all numbers would be rational

[@jackeown]

Is the end wrong? It says "The maximum or minimum of a function f(x,y) subject to g(x,y)=k must occur where df is parallel to dg. Couldn't it also occur where df = [0,0] and dg != [0,0] ? (the normal local mins and maxes that are well within the constraint)

[@cameleon5724]

Endless enigmatic book in all languages. You can write a book with mirrors in all languages of the world. You can speak two languages at once, you just need to find the perfect reflection, same content, different translation. Infinite Mirrors. Pi 3.14 XBooks. Hybrid language!!!!! One content, two languages. What I have now written may have a perfect mirror in another language.!!!!!!!!!!!!lllll

[@bennettmp]

This didn't really solve how to find the min and max fully. Good video, but it becomes hard to understand when you impose delF flat on the same plane as delG when really delF is tangent to the 3d surface on another plane, although their directions on xy are the same. You gave us the f'(x) = 0 example to solve where min and max are on xy, but didn't provide the 3d version of this. Please show how these two work. Do a workout example.

[@pedrocasique7352]

¡Chingón carnal! ¡Hoy soy un poco menos pendejo que ayer, chiiirs!

[@Witwas-m5k]

Good video. Lambda in economics context have meaning call "SHADOW PRICE"

[@morgangraley1049]

What does this mean regarding a 4- or higher-dimension surface? If a 3-dimensional derivative already requires us to find partial derivatives (question, is it always a “D - 1” number of partial derivatives, no matter the number of dimensions, D)?

[@KrasBadan]

I don't understand why ∆f have to be parallel to ∆g. It is absolutely possible for 2 lines to be perpendicular to a 3rd while not being parallel to each other. Like, what if g function is way steeper at that point? Both of the vectors would be perpendicular to boundary, but they'll have different steepnesses. Multiplying one of them by a number would make it change length, not direction, so you shouldn't be able to express ∆f vector as lambda times ∆g vector.

[@sajandaheriya5339]

Yes, del(g) and del(f) are perpendicular to level curve at max but there can be infinite possible vectors perpendicular to a tangent line at a point on level curve, this does not imply that del(g) and del(f) are parallel? Now how to proceed further? Edit: Nevermind I got it. Basically gradient vector is in 2D so there are only one possibility which is that del(g) and del(f) are in same direction. Thanks!!!

[@dorol6375]

Idea for finding the extrema on a boundary: use that boundary's parametric equation and plug it into the function which will result in a 1d function. From that it's as trivial to fund the extrema as it would be on a 1d function!

[@Sanelicv]

just brilliant. If I may ask, what software did you use to make these animations?

[@SerpentineIntegral]

Thank you! The animations were made using a homemade Python library called "Morpho". You can find the project here: github.com/morpho-matters/morpholib

[@Sanelicv]

@@SerpentineIntegral Thank you!

[@Adventure_fuel]

I get the best education on KZbin. KZbin university the melting pot of universities and independent educators.

[@canowow11]

really good video on a difficult math problem, but visually you made it easy

[@harshal8956]

This just blew my mind. This is what I was looking for. Great work.

[@joaogoncalves-tz2uj]

this is the best video I've seen on this topic and it still doesn't clear all the doubts about it. Why does it when the tangent line at the 3d curve of g is parallel to the plane xy we can say the gradient at f is perpendicular to the "level curve" of g? Also, given that there are infinite lines perpendicular to a given line, how does it guarantees grad f // grad g?

[@alperyldrm4788]

That is wonderful how you visualize and construct the idea step by step! Grateful!

[@Galakyllz]

All of this time is spent describing the problem and then you rush through the end. How did you work out the x, y, and constant values? Where do they come from? Did you end up solving the problem?

[@davidebic]

This is exactly the intuition I had trying to understand Lagrange Multipliers!

[@frankjohnson123]

Bit of a nitpick but I don’t like conflating the terms “flat” and “horizontal.” A curve can be totally flat but have a nonzero gradient.

[@ilong4rennes]

thank you so much for your extraordinary video! this helps me a lot!

[@zacharydavis4398]

Solid content 👍🏾Thanks for spending the time to create and share 🤙🏾

[@dhruvbala4093]

10:40 I lack the cognitive capacity to visualize f gradients at neighboring points on the ellipse. Would have helped to better illustrate why that point is unique Good video though

[@mujtabaalam5907]

But in the case of this function, the gradient of f at its local maximum is zero, so it can't be parallel with any gradient vector of g

[@googleyoutubechannel8554]

"a good way to do that" is to ask a computer, algos to solve these problems have been written 10000 times before, there's no need to make it 10001

[@blower05]

why the g surface is not a cylinder surface? Did it make sth wrong?

[@JakubS]

you could also say that the unit/normalised gradient vector f is equal to the unit/normalised gradient vector g

[@emranhasan7480]

According to 10:49, doesn't the grad(f) on each point on the boundary of f is parallel to the grad(g) ?

[@gabrieledettorre]

10:41: but why does the gradient of f have to be perpendicular if the boundery line is flat at that point?

[@ΠάνοςΚΜ]

Great vid.Only thing that sucked was english is not my mother tongue so i had to search the termology every once in a while in my language and that set my trail of thought back a couple of times

[@andreabonvini]

From the picture at 10:23 it seems that the maximum point (yellow in the picture) on the boundary line should actually be the one closer to the y axis, right?

[@boutainabenhmida6071]

never seen a visual explanation better than this

[@marlongrau246]

It seems the vector fields of g and f are both of straight lines. And the degree of field forces define the entire shape of the source of g functions which is g vector fields and project it to the f function or the f vector fields. That seems to my understanding in terms of gravitational fields or magnetic fields. That somehow the g force fields exerts force to the projected f force fields? Like the em forces?

[@gossipGirlMegan]

Excellent work I ever met ! Tanks a lot ,deer professor!!!

[@henryholsten8802]

The idea of "assume a solution exists" is just bonkers in terms of how well it works vs how well you would expect it to work in maths

[@rylieweaver1516]

This was so great… but how about when constraints are inequalities?

[@thomasjefferson6225]

Hey don't ignore lambda. It has a role in duality!!!!!

[@bijoychandraroy]

I wish I could rewire my mind to the point I understood this, I wish for the me that was once in love with math

[@camel2666]

single-handedly saving my vector calc grade!

[@user-wr4yl7tx3w]

Wow, that is really well and clearly explained.

[@plekkchand]

Wonderful, direct, lucid, free of affected cuteness and cosmic background music. Thank you!

[@cadedulaney1522]

Incredible explanation this helped me so much

[@TheZoneTakesYou]

omg 1 semester in 13 minutes. jk but that was a really good summary

[@user-qs3ih3ll5f]

Thank you. I love this explanation.

[@Vikraman99]

@11:04 but aren't all tangents to g(x,y) is parrallel to all tangents of the boundary curve f(x,y). So won't we consider every point on f(x,y) a maxima or minima?

[@vikraal6974]

f(x,y) is flat on the boundary at max and min points only. At those points gradient of f is parallel to g.

[@shankhasinha1444]

Thank you so much for making this video.

[@egeecagan]

best explanation ever without killing some of my brain cells

[@verracaelum5258]

agam bu tarz animasyonlarla anlatan başka bildiğin kanallar var mı bu adamın az videosu varmış böyle