Gradient Descent in 3 minutes

[@user-sb6os]

This animation is really great for a small channel like this

[@prakhargupta1745]

just a correction, 2:30 is mini-batch stochastic gradient descent since we are iterating over batches

[@kitzelnsiebert]

Great job! I like that you were even able to talk about some of the different types of gradient decent algorithms, tall task for 3 minutes.

[@byronwilliams7977]

Same here. i studied Applied mathematics, so I have to get up to speed on this rather quickly, I find these videos to be excellent.

[@myelinsheathxd]

That's appealing to see these visual explanations after learning the the concept !

[@vivekpujaravp]

Best explanation and visualization I've seen. You have incredible talent. Please keep making more.

[@billyin4771]

Very comprehensive and short, love it! Quick and concise!

[@VisuallyExplained]

Thanks so much!

[@hrishikeshsrivatsa]

First video, where I got clear and precise understanding of the topic

[@Leo-vv3jd]

I really liked the video and the visuals, but I think it would be better without the "generic music" in the background.

[@VisuallyExplained]

Thank you for taking the time to post your feedback, this is very useful for the growth of this channel!

[@209_Violate]

wow. you have such a talent for explaining things so well compared to the rest of the youtube sphere. i hope you will continue to bless us with your talents.

[@snowcamo]

Honestly didn't really help with my questions, but I didn't expect a 3 minute video to answer them. This was very well done, the visualization was great, and everything it touched on (while brief) was concise and accurate. Subbed.

[@ah-rdk]

Hello. Thanks for this great video. Just I believe at 2:20, this variant of Gradient Descent that you explained is called the Mini-Batch Gradient Descent which uses a random subset of the training dataset. Stochastic Gradient Descent is the one that uses just one training record in each iteration.

[@khabib8568]

Yeah i have the same doubt

[@0N3_01]

Thank you for the clear explanation

[@A0G7]

i looooooooooove when the background music stops. it tells you to open your eye and focus your ear, a really important revelation is coming ....

[@AK56fire]

Could you please make a video explaining how you made this video.. That would be very VERY helpful.. I've always wanted to use blender to make such animations as you've made, but couldn't make head or tails of it in blender. Most tutorials(and I've seen more than 100 videos) on blender showcase heavyduty animations which have nothing to do with mathematical explanations, as in how to make animations for maths related videos.. Your's is the first video in which I've seen such a thing. Please consider my request and kindly make a video tutorial about it(for the Blender part).

[@VisuallyExplained]

Hey Amit, I am definitely planning to make a video about my workflow, and in particular, how I make the animations. So stay tuned for that! :)

[@AK56fire]

@@VisuallyExplained Will definitely wait for it.. Thanks for considering it. I appreciate it a lot.

[@photogyulai]

Huge thanks, it would be fantastic (fellow teacher here:-)

[@olivier306]

Literal gift from god this channel

[@juanete69]

It mixes very well the theory and a practical example.

[@unplandsitch]

Wow this is so well, intuitivly explained

[@bilalb95]

Thanks for the amazing explanation and visualization

[@dannyk123]

Great video, was wondering how this works if there are multiple minimum points where the data has high dimensionality?

[@herronproductions829]

It works just fine with multiple minimum points in a high dimension. As long as you configure your hyperparameters (learning rate, batch size, etc) correctly, you should have no problem converging to a "decent" minimum.

[@World_Admiror]

Keep up the good work! This viedeo and the whole chanel are amazing! Bravo 👏🏻

[@na50r24]

For Vanilla GD, are you not supposed to divide by the number of samples in data before performing the update? Or do you just take the sum of this 'accumulated gradient'?

[@hariraj4184]

Brilliant work 👍

[@brainxyz]

Great Job! well done

[@VisuallyExplained]

Thanks a lot!

[@pdebuck1]

My professor said “there is no excuse for gradient descent” when conjugate gradient is so easy to implement

[@MARCELSOCORÓGARRIGOSA-l6h]

I have only seen one video and it is helping me a lot! Keep going!

[@MrWater2]

funcking incredible explanation in just 3 minutes..wow!

[@user-rw6iw8jg2t]

Isn't it awesome in a simplified way, I was juz implementing OLS for Vanilla linear regression to train the neural networks with some weights and bias and this video popped up was doing some stuff with thje matrix and dot product , I luv Mathematics !!! one thing when we have OLS algo directly why we need to implement OLS with GDA again then wts the use of having OLS algo seperately is it coz of Volume of the data points ?

[@eumatheus]

Thanks for this, the visualization helps a lot!

[@sayyidj6406]

excellent vid. do you have a video about PPO in RL?

[@ranjanpal7217]

Amazing explanation and visualization

[@alanmdl]

great video but at around 1:30 my heart dropped it felt like a scary movie since it was all dark lol

[@Hateusernamearentu]

very smart. But I still need another video presenting differential to help understand the slope, opposite direction thing in 2D. But this is clear. Also I like the small step demonstration.

[@error-my9ut]

thanks for visualization it really helped .

[@thebrucecyou]

Excellent video!

[@Darkev77]

Proximal GD next please!

[@Urammar]

How does this actually apply in reverse, though? How do you apply this

[@AK56fire]

Which software did you use to make those animations.

[@VisuallyExplained]

I used Blender3D (with python) for all 3d scenes. The rest is a combination of After Effects and the python library manim

[@abdulghanialmasri5550]

Great video, would you consider some topics in numerical analysis, like gaussian quadrature???

[@rasmil77]

Helpful for revising the topic :)

[@fabricetshinangi5042]

Can you please post a link or titles of materials(books) on this topic tsht one can go through . I really need to learn this topic. Thank you

[@VisuallyExplained]

Great idea! Boyd's book is a good starting point (page 463 of web.stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf). I will try to add more references to the video description in the future.

[@fabricetshinangi5042]

Thank you

[@alirezaakhavi9943]

really nice animation, explanation and content than you very much for sharing! :)

[@TimCrinion-j2r]

Is this the same as Newton's method, or the Newton-Raphson method?

[@jayp9158]

My fav video about this

[@JuanCarlosAraujoCabarcas]

Interesting topic and comparison. Since you are using information from past iterations, it would be very illustrative to include a quasi-newton in your comparison. For example the BFGS.

[@VisuallyExplained]

Thanks, and great suggestion!!!

[@md.sarowarhossainrana4787]

what is the ita?

[@tarekbenzyad6766]

i was about to ask the same thing , he suddenly introduced it into the cost function as a paramter then never talked about what it meant

[@tsunningwah3471]

simplex method please

[@incrediblekullu7932]

how you make these videos ? manim ??

[@zzzzzz-zzz-789]

great video, thanks!

[@Zethuzzz]

Adam usually works quite good

[@John-wx3zn]

What is gradient descent trying to find?

[@at1with0]

To minimize cost/error of a learning model

[@charlesyang7233]

This is amazing

[@AK56fire]

Great Animation buddy.. Cool..

[@VisuallyExplained]

Thank you! Cheers!

[@wearedoingsomething]

Huge thank you!

[@viddeshk8020]

Wow, 😊❤️ love it

[@VisuallyExplained]

Thank you! Cheers!!!

[@arnaupinto5890]

awesome

[@emirhandemir3872]

Amazing !!!

[@cedricmanouan2333]

magnificent 🔥😧

[@guillecobo_]

great video

[@VisuallyExplained]

Great comment, thanks!

[@andrewt15]

dope

[@Hans_Magnusson]

As you might know I studied this topic in London… I obviously aced it.😂

[@FaberLSH]

Good!

[@yassinom2466]

veeeeery nice

[@fulljsu3glitches]

And how the fuck i get the n ??

[@marinamaged962]

i dont get it lol

[@-leaflet]

Wow

[@Christoo228]

sagapo

[@tsunningwah3471]

房山

[@mohammadnasser9951]

Damn calculus. I understand nothing.