Symmetric Rank 1 | Exact Line Search | Theory and Python Code | Optimization Techniques #7

[@kerimetasc6981]

Best lecture in quasi-newton method which I have so far found on internet!

[@yaglz4584]

2:50 the animations are very nice. Thank you for taking time to record the lecture.

[@gamingboychannel4992]

Using LaTeX generated equations like a boss. Thank you sir Ahmad !

[@techguru4792]

It's rare when less viewed video gives best explanation. Your presentations are almost like 3Blue1Brown or Khan academy! Don't know why this video has this less view!!

[@robloxeren7527]

Ten minutes of this video explains better than an hour of lecture in the course I’m taking🤣 thanks for saving my brain!

[@utpalgaming8189]

Hello Ahmad. Many thanks for your support! To be honest, I don't know much about gradient methods. I often use search-based optimization methods in my research such as GA, PSO ...

[@awesomegameplays3126]

I am a PhD student and I will be using optimization methods in my research.

[@walak3955]

Honestly, this guy is incredible. He explains everything soo precisely and efficiently without any unnecessary information. Thanks a lot for this video. You made my life easier.

[@mehmetakif2211]

Thanks so much for posting!!

[@efey2605]

I have been watching your videos regularly and they are very informative. Thanking you for taking time to enlighten us. Would you mind making videos on conventional optimizationmethods like conjugate gradient methods?

[@patronkral7664]

Thank you so much for wonderful series of videos. Can you please make a video to solve a bi-level optimization problem with a number of variables to solve using different optimization solvers, like GA etc.,? It will be very much appreciated.

[@frycomfort4002]

This is wonderful!

[@gaffarsolihu1617]

Hi Ahmad, how are you doing? Thank you so much for your videos. Personally, they have been very eye-opening and educational. This might be a far reached request, As a graduate student, your videos have been very helpful, especially with implementing which is missing in classes, but I'll like to know if you have any plan for a full-blown project implementation on any of your Playlists, be it ML or Math Optimization. Thank you

[@AhmadBazzi]

Hello Gaffar, I'm doing well, hope are you as well. I'm very glad you found it useful. As a matter of fact, this is a very great idea. I will give this a deep thought, then act accordingly. Thank you for you idea :)

[@bollywoodtalkies5052]

he did all this hard work and sent it to the internet for free. and he doesn't get too much but what he gets is RESPECT and credit for bringing new aspiring engineers to earth.

[@ardaerennaim182]

This guy is the most underrated youtuber on planet earth.

[@essamsayedemam7078]

thank u very much it was soo helpful can i get the pdf version !!

[@benvesly]

To find this whole course freely available on KZbin is such a gift. Seriously, you cover a LOT of ground.

[@furkanefebayrakc8080]

Understandable with example, rather than those who explained long enough using matrix formula only. Thank you 🙏✨

[@ercansarusta677]

I've known this man only for 40 minutes, but I feel like I owe him 40 decades of gratitude. Thank you for this awesome tutorial!

[@purplerain1562]

Thanks for posting these videos. They are quite helpful. So, to ensure that we minimize and not maximize, is it sufficient to ensure that the newton step has the same sign (goes towards the same direction) as the gradient? Is it ok to just change the sign of the step if that's not the case? (my experiments seem to indicate its not, but what should be done then?)

[@ayuuu2920]

I can't believe these type of courses are for free here, it's amazing how education has change.

[@haktankoctv7426]

Your explanation is awesome. Extension from root-finding scenario to minimum-point-finding problem was exactly my question.

[@zmd9678]

this guy, sat for about 1 hour and talked about newton in one video, and then released it for free. legend

[@fatihbiz4105]

This course has literally changed my life. 2 years ago i started learning optimization from this course and now i am a software engineer intern at a great startup. Thanks Ahmad !

[@nihathatipoglu8936]

Dude, I'm less than 2 minutes in and I just want to say thank you so much for creating this absolute monster of a video.

[@flicksstudio4054]

Wonderful video for clearing optimization of newtons method for finding minima of function in machine learning

[@bunyamincc1177]

ABSOLUTELY LOVE your 40 minute video series... Thanks a lot Ahmad :)😍

[@unknowngone782]

What an absolutely epic contribution to the world. Thank you!

[@nurettinefe537]

Just finished watching and following along with this. Ahmad, thank you so much! It took me about 12 hours to actually get through it cus I kept pausing and going back to make sure I got all the things right.

[@jaimahakaal65]

Super clear explanations and very well put together. Thank you!

[@Туганаарга]

man, perfect explanation. clear and intuitive!

[@suleymanozcan6093]

Hats off ! Ahmad I have no words to let you know how grateful I am for this free course, it is not only well designed but also easy to follow, God bless you.

[@origamianddiy4861]

I'm here from yesterday's 3b1b video on Newton's method of finding roots, after wondering if there's any way to use it for minimizing a function. Mainly to see why we can't use it instead of Stochastic Gradiend Descent in Linear Regression. Turns out the Hessian of functions with many components can turn out to be large and computationally intensive, and also that if the second derivative is not a parabola, it can lead you far away from the minima. Still it was nice to see how the operation works in practice, and you mentioned the same points about Hessians too. Good job 😊👍

[@benhammouyt7300]

Loved the graphical presentation

[@KarakterFilm4039]

Thank you for the words of encouragement, I appreciate it!

[@xuramanhmdova1364]

This course is extremely useful. Thanks a lot. You did a great job!

[@onlinestudylaksar1097]

Excellent video, it really helps me for understanding the quasi Newton method, thank you very much!

[@apptutorials2158]

Sir your way of explaining is really good.

[@RandomVideos-hl5kc]

Amazingly presented, thank you.

[@jadolive-fan5103]

Thanks for this tutorial. Awesome explanations perfect for beginners and experts.

[@Epicorstroys]

The way you explain this is so helpful - love the comparison to the linear approximation. Thank you!

[@theworld-stoptime.5800]

Ahmad can really keep you hooked up on the way he explains things. What a legend.

[@oyunking9012]

An incredible work as usual. Congratulations for the whole video.

[@MRBEASTFAN1102]

Amazing explaination! This is very helful for understanding. Thanks a lot sir.

[@zazasabah3378]

Can we just take a moment to appreciate this guy for providing this type of content for free ? great help, Thank you sir! 🙏🙏🙏

[@batuhanuysl6905]

superb,excellent,best video

[@pankajmittal3201]

Thank you for the amazing optimization algorithms tutorial! We Appreciate your time and the effort to teach us coding 😃

[@TechnicalRH]

Very nice and clear explanations

[@Iamdevil.1]

Amazing video. Looking forward to more.

[@talhayavas6640]

I can't even imagine how long it took to complete this video. Thanks a ton for your effort.

[@VRCreations2O]

Gorgeous tutorial ! I have never even saw the pyhton interface in my life before, but with the help of your videos i feel like i understand a lot.

[@islamiclife9391]

Illuminating! Thank you

[@thevowtv1563]

HOLYYYYY FKKK !!!! I really wish I came across your video much before I took the painful ways to learn all this… definitely a big recommendation for all the people I know who just started with optimisation courses. Great work !!!!!

[@ahmetmelihsanl9663]

I love your videos!, having learnt all this in my gcses / a levels, just rewatching it after 4 months after my exams

[@sajalshah6522]

I really appreciate your precious effort ,not to mention how much fun and friendly to learn. Thanks Prof. Ahmad.

[@dammnoe]

This was exactly what I needed, thank you!

[@TvShow-ml3dz]

WoW! This is amasing work man, thank you.

[@007AryanVlogs]

Brillant explanation, thank you so much.

[@eser_bodur9302]

This was such an awesome explanation, so grateful thank you.

[@salihbeyy9864]

Amazing lecture! Muchas gracias!

[@PoringMC]

Hii Dr. Ahmad ,as per my knowledge this methods are used for Machine learning, where gradient descent is a classical algorithm to find minimum of a function( not always zero), If you know basics about ML then you will be familiar with loss function , so we have to minimize that function, for that we need its derivative to be zero, for finding that we use gradient as direction where the change in function is maximum.Now we have the direction but the we dont have the magnitude , for that we use a learning rate as a constant which is what 1st order does.In 2nd order we would use the magnitude which gives us the magnitude for which the point where derivative of function is 0 can be reached in less iterations.Thus 3rd order will ultimately result in finding the minimum of dervative of the loss function , but we need to find minimum of the loss function so ,it will be useless. Hope this was helpul

[@superiorarmy416]

Thank you very much for your suggestion! I will try my best.

[@AJ-et3vf]

Awesome video! Thank you!

[@Mitchyyy92]

Very good explanation

[@gangadharparate531]

Really appreciate your course! Your tutorials are always so helpful.

[@dinivideolarpaylasmlar8256]

Another problem is for a negative curvature, the method climbs uphill. E.g. ML Loss functions tend to have a lot of saddle points, which attract the method, so gradient descent is used, because it can find the direction down from the saddle

[@husarr5111]

Thank you soo much for the amazing lecture.

[@critcalops3107]

I think that the visualization makes sense if we think about approximating the function f(x) by its second order Taylor expansion around x_t. Taking the derivative of the second order Taylor expansion and setting it equal to zero leads us to the formula of the Newton's method for optimization. This operation is the same as minimizing the second order approximation of the function at x_t as depicted in the video.

[@sujunprodhanwordpress]

Very great content

[@Babapr.o]

Amazing job! Thanks a lot!!

[@muhammadazan5540]

lovely explanation 🤩🤩🤩🤩🤩🤩

[@Brns_8399]

thanks , very informative

[@madkar988]

This was actually quite helpful :)

[@AbhishekKumar-kt6yp]

Thank you, Ahmad, for the time and effort you took into making this marvellous tutorial. Much, much appreciated!

[@roshan4542]

Wow, this looks like a great course! 😀

[@herseyburada9288]

Ahmad is a legend !

[@lionjenkins2063]

Awesome, thank you!

[@furkansenol1450]

Your videos are awesome!

[@troguz195]

Thank you Ahmad !

[@gamerhappyonline4175]

What the what?! Even I understood this. Killer tutorial!

[@Abdullahqamar16]

Thank you for the video!

[@oyuntv7174]

This is brilliant thank you, hope you give us more visual insight into calculus related things

[@ahmyahmy9269]

Again amazing

[@user-rt3wl1wv4x]

Ahmad you should write your book , it'll be really helpful for literally a lot of people out there

[@user-bt5zx1li1i]

very good. thank you

[@furkanatukk]

I hope listening to this brings more positive KZbin channels like yours 💜

[@linkbox3117]

Very nice. Thank you for your inside information Ahmad. Always pleased to watch your content.

[@acimasinhd9590]

really appreciate your work :)

[@snxgz2808]

Thank You so much Sir✨

[@afthabrazagaming3547]

Great content

[@mehmetkaymak627]

OMG, this video just saved my homework

[@sggcengizhan2999]

Sure. Consider the quadratic approximation f(x) ~ f(xk) + f'(xk) (x - xk) + 1/2 f''(xk) (x-xk)^2 at the bottom of the screen at 7:06. To minimize the right hand side, we can take the derivative with respect to x and set it to zero (i.e., f'(xk) + f''(xk) (x - xk) = 0). If you solve for x, you get x = xk - 1 / f''(xk) * f'(xk).

[@BrawlStars-hf8hm]

We appreciate you ❤️

[@how2make208]

Good job. I am subscribing !

[@reissoyundaa4901]

Yes, I think that statement on Wikipedia is a little misleading. Any symmetric rank one update can be written as c^T c for some vector c. For a problem in R^n, then c will have n degrees of freedom, but equation (**) gives you n constraints, so it's not surprising that you get a unique solution.

[@openmusicnocopyright3249]

This is great.

[@WayneWarp]

Very nice video

[@radiolight7793]

All the best!!

[@ROBOOTDANCING]

Liked, Subscribed and voted for him 👍

[@BorntobeHiu]

underrated