The Most Important Algorithm in Machine Learning

[@ArtemKirsanov]

Join Shortform for awesome book guides and get 5 days of unlimited access! shortform.com/artem

[@TNTsundar]

Can you talk about liquid neural networks? I’m interested to know if that’s a revolutionary work that deserves more recognition and following. arxiv.org/pdf/2006.04439.pdf

[@webgpu]

Artem, i want to thank you, not only for publishing an excellent material (from the 100's of DL/ML videos i've saved, yours is top 5 - really), as well as having a great intonation, that helps A LOT in capturing the attention in this day and age of constant distractions. THANK YOU 🙌

[@mdsadik6987]

That's the most easiest explanation of back prop. Thanks alot. I have done multiple courses of Deep learning. I understood most things. but in the case of backprop I only undertstood few things like finding partial derivative of loss function in respect to some para meter and then adjusting weight parameters based on that and learning rate. but never really knew how it worked under the hood

[@siquod]

You did not explain what happens if a value has multiple paths through the computation graph that lead to the loss. The chain rule for multi-argument functions is an important part of the trick.

[@Mutual_Information]

Back prop is a hard, heavy thing to explain, and this video does it extremely well. I mean, that section 'Computational Graph and Autodiff' might be the best explanation of that subject on the internet. I'm very impressed - well done!

[@33gbm]

You two are the best channels I have found in the SoME episodes. It's great to see this interaction between you guys.

[@dprophecyguy]

Love your videos

[@michaelcharlesthearchangel]

If there is no mention of sine waves in neural networks then it won't be total.

[@ExtantFrodo2]

Where is that section 'Computational Graph and Autodiff' ?

[@entivreality]

Yeah really helped me get the significance of autodiff

[@CuriousLad]

Funnily enough, the calculus portion of the video is probably one of the best explained I've seen

[@George70220]

Why would that be 'funnily enough'? What a diss lmao.

[@balu6923]

@@George70220 I don't think CuriousLad meant it as a diss, it's just that when Artem made the video, he explained the Calculus section as a background information. The partial derivates and gradient descent wasn't the main topic of the vid, yet you could show this to Calculus I student and they would be thanking him for the explanation, even if they have not interest in learning back propagation! That's why funnily enough, while the intro Calc topics wasn't the main part of the video, that portion would be very helpful to anyone starting out int Calc!

[@veritas7010]

I dont agree for example the act of minimizing loss function and gradient descend were not properly linked there were just two pieces of information unprocessed dumped in series

[@tonic4120]

I found it unnecessary. Anyone who clicks on a video about back propagation likely already knows calculus, and if they don’t, that short primer is not going to be enough foundation for the rest of the content.

[@MphoMotlokwemampuru]

Nasdaq please buy card 0:42

[@undertheshadow]

"Wait, It's all derivatives?" "Always has been" Great work pal. Provides excellent clarity. Looking forward to the second part.

[@rad6626]

😂 Turns out back propagation isn’t just magic

[@shikhargairola5815]

It’s probably the best explanation of backward propagation. Hats off to your hard work and saving this so valuable content.

[@aminebouramada]

this's by far the most clearer explaination and simplification of backpropagation i have watched

[@vastabyss6496]

It makes sense that you would cover both computational neuroscience AND machine learning since they both play a significant role in AI research. The sort of content you're making is definitely 3Blue1Brown level. Keep up the good work!

[@nickwissler6811]

He also managed to squeeze an entire calc 1 course into this single video. It's amazing

[@matheusmendonca1332]

By far the best ML explanation I have seen on internet.

[@ReighKnight]

The visuals on this video is from another planet . So Good !!!!!!!!

[@black_crest]

This just might be the most underrated video on Back Propagation that I've ever seen! I hope more people come across this

[@keithwallace5277]

This has to be one of the greatest explanation of the inner working of learning in ML, I love it!

[@dianasong4594]

indeed

[@ggsap]

what about 3b1b's series?

[@keithwallace5277]

@@ggsap both are great, but for me this is better

[@priteshtadvi4946]

I knew that calculus is important for machine learning but never knew that 12th grade derivatives are that much important. When you said about chain rule, that bring me back to my school days , I never thought that derivatives, integration and probabilities will be used this way in future. Well explained video. Thanks for sharing this knowledge and conveying process much simply.

[@arjunrao9978]

Very True!

[@mb2776]

Remember when people said nobody needs higher dimensions expect those stupid quantum scientist and nothing useful would come out of it? yeah...^^

[@maheshwaransivagnanam6452]

I've been trying to get into ML for quite a while now. This is by far the best explanation of gradient descent and back propagation hands down!!! Amazing work!!!

[@Redant1Redant]

That was an outstanding explanation. Your ability to explain higher mathematical concepts in such simple terms is really an amazing service to the rest of us who wanna understand these subjects but don’t have a mathematics degree. Thank you.

[@Alwaysiamcaesar]

I actually pictured this all in my head successfully where I thought I had everything in a canonical deep neural network figured out the other day. It’s one thing to hold it, it’s another to do the detailed, gritty work of explaining it in video format. Very well done.

[@pradhumnkanase8381]

There could not have been a better explanation. Hats off to you

[@K9Megahertz]

This is a visual masterpiece! Well done! Much of this was a review for me as I took the time to go through all this last year. I did an implementation of the MNIST handwritten number neural network and had to learn all the calculus covered here to work out the backpropagation math. You really do have to dig in to it to get a good handle on it but it's fun stuff.

[@bungerwow7963]

I've seen probably 20 videos on this and your explanation of the derivatives for someone not in calculus was really helpful. thanks.

[@AlexKelleyD]

This is one of, if not the, best videos I’ve seen that throughly explains back propagation. It will definitely help me to be able to better explain the algorithm to others, so thank you for creating it.

[@gianlucanordio7200]

I just have to say this goes way beyond the quality of the many chainrule videos I've seen so far. Good job man, you've got some impressive skills to keep me watching a math video and take notes past my usual bedtime

[@marc_frank]

you take notes?

[@kevinscales]

@@marc_frank It's generally a good idea if you are trying to learn. Don't be passive if you want it to stick.

[@ZavierBanerjea]

Taking notes, making sketches of the ideas, doing the math are excellent learning techniques. Old timers like me always do that 👍

[@nerdvananc]

I don't know maths at this level but your explanations are so clear and concise that every step of the way felt simple to understand - thank you!

[@ZavierBanerjea]

Guru of Fundamentals. I can't resist subscribing to your channel and watch all of your videos. The way you explained Chain Rule : the logic behind it is awesome. I am trying to visualize the Quotient Rule of Derivatives in your way. A good Teacher always makes you THINK 🙏

[@Anonymous-fr2op]

Damn, I was wondering where you've been since over half a year, whilst I was stuck in backpropagation😂 and here you came back like a true mind reader. Glad to see you back❤

[@highchiller]

He was calculating your backward step so you can make your next forward step (sorry, couldnt resist) XD

[@David123456789345]

@@highchillerhe just gave you the right explanation gradient so that you can optimize your learning loss function 😂

[@ChaseGartner]

Absolutely one of the best videos explaining data points and regression formulas I have ever seen. Amazing work

[@naveen_malla]

Dude, this is the most beautiful ML video i've ever seen. Highly informative yes, but also beautifully made. Thank you for your work.

[@aabiddd]

all these basic concepts such as derivatives, least square method, I'm learning it in my college. watching these kind of machine learning videos has made me understand the practical applications of these theoretical concepts a bit better now 😌

[@cachegrk]

This is the best ever explanation I have seen. Thanks for taking the time and doing something extraordinary.

[@ram-my6fl]

Most Comprehensive Explanation EVER my opinion : better than 3b 1b, No offence to 3b 1b Hes great at it and one of the pioneers who did these kind kf visual explanations. But i like your explanation as it is slow paced & comprehensive

[@domorobotics6172]

Yeah 3b1b definitely deserves respect from me, but I think he will to recognize this video is very carefully done. I like that these people just care about the truth and the perfection, and even with a little bit of envy, care about the best product being done.

[@TruthOfZ0]

i just made that in python for a simple quadratic equation.....THANK YOU !!!! i just learned python and machine learning !!!!!!!!!! Using desired y=0 i could also find one solution of the equation... wow i love this so much!! The only different i did was to make x the weight and not the coeficients which i wanted them to be fixed inputs What you helped me realise is that any system that can put in a computational graph like that 30:04 ...it can be embeded backpropagation regardles THANK YOU im out of words Also when the next loss is bigger or equal than the preview loss after one iteration... i divided the learning rate by a factor of 2 or 10 for more accuracy and if the next loss was smaller than the preview one i multiple the learning rate by a factor of 1.1 to 1.5 to speed up the proccess...thus having results in hundreds or even thousands less generations/iterations and less time consuming!!!!! I can use this for optimizing my desired outputs in any system !!! JUST WOW!!

[@tonsetz]

He is back! Greetings from Brazil, we've all been waiting for this release!

[@kltr007]

This video explains the mathematical base of neural networks in a way I understood it the frist time enough to be able to explain it to somebody else. Thank You for that. I can't even imagine how much work you put into the animations. A master piece!

[@winterknight1159]

I have been doing ML research for a few years now but somehow I was drawn to this video. I am glad to say that it did not disappoint! You have done an amazing job, putting things in perspective and showing respect to calculus where it is due. We forget how a simple derivatives powers all of ML. Thank you for reminding that!

[@ArtemKirsanov]

Thank you! That’s really nice to hear!

[@pavanmamidi7705]

Understanding something complex requires high intelligence. Explaining it simply requires even higher intelligence. You are one of the best teachers that I have encountered in my life! I'm grateful!

[@techshivanik]

This is the best resource I have found to learn backpropagation. The visualization of each concept made this very clear. I can't even imagine the amount of effort you have put into this video.

[@ahmeterdonmez9195]

The best and most understandable explanation I have ever seen. You explained the essential basis of Artificial Neural Networks so beautifully. I really congratulate you

[@sparkle2575]

Excellent explanation!! You have done a selfless service to humanity.

[@moralboundaries1]

So clear and concise! Thank you for creating this.

[@hasanrants]

I just watched this video after completing my first lecture of Deep Learning on Backpropagation and Gradient Descent. thanks man! appreciated. Really solid content.

[@Master_of_Chess_Shorts]

This has to be the best explanation of the chain rule ever! Thanks

[@MrRhainer]

The best explanation about Deep Learning. Grateful.

[@shekharkumar1902]

This is the best educational video i ever seen on internet.. explaining the Backpropogation with visualisation. Amazingly super😊😊

[@TheMankeer]

What a great explanation and clarification especially for all mathematics required to understand Back prop algorithm, appreciate this so much

[@omarbadr9469]

Man, you really nailed it, especially the Computational Graph and Autodiff part. I heard so many times about them in lectures on Stanford and others. However, this was impressive.

[@Vinsce_lives]

This is incredibly well done and helped me visualize derivatives comprehensively. Thank you.

[@Binue-n5g]

The world needs more of you bro

[@HeatherRoberson-vx5eh]

As a student in this business, who has passed through a bunch of professors, I can say with confidence! With this trader, you will both learn and earn and, importantly, receive advice. Everything is competent and clear, without a bunch of any unnecessary movements! Keep up the good work!🤣

[@XxIgnirFirexX]

I think I just found my favourite channel of all times. I've been on YT since 2011 and never had a crush for a YT channel before today é.è

[@pankajgoikar4158]

Where you were so far.... I've been trying to understand this concept from past 2 years, and now it's cleared after watching this video. Honestly, my maths was not up to the mark. After seeing your video so many important concepts are cleared. Don't have enough words to thank you.... God bless you. I'll share your videos with my friends. Please keep it up.....🙏🙏🙏🙏🙏🙏🙏🙏🙏

[@仁科欣]

It's very very nice to see that are you updating.

[@chilledpepsi]

Hands down the best explanation there is to backprop

[@HimanshuPakhale-n3i]

in simple words, backpropagation is method of finding gradient descent. to minimising the losses their is need to find the correct values of each function and it is get by chain rule which is based on darivatives and calculus. it's direction in backward direction hence it is known as backward propagation. still so much confusion in mind regarding this process. video is very useful and editing is extraordinary.

[@MultiMojo]

Another gem of a video, well done Artem!! This channel deserves 1M+ subscribers, there's nothing else like it on KZbin.

[@halilzabun]

One of the best visual explanations of the backpropagation algorithm I've seen! The animations are really good.

[@javastream5015]

Sure that it was the back propagation algorithm?

[@HozanKano]

The best explanation of machime learning i have ever seen on you tube ,amazing work .thank you👍

[@slopesmonte]

Finally a solid explantation of backpropagation. Thank you!!

[@ndungikyalo]

Amazing how you can explain it so well, so simply. You have a subscriber !

[@ElSenorEls]

This is the best video about this topic. Learned a lot of things. Took me 2 or more hours but I understand it now. Thank you!

[@asdasd-yr7wi]

31 years now, had like 13 years of math in school and another 5 years at university, first time i really understood how derivatives work, bcs visualisation instead of "you calculate it this way and derive it that way, now memorize"

[@ArnaudMEURET]

May I ask which university you went to?

[@WsciekleMleko]

@@ArnaudMEURET Sorry, but I don't believe anybody who have no idea what a tangent line of a function in point x looks like, and what it means (despite milion of excersises teaching you it's meaning), and dozens of graphs in literally every workbook, could actually go through 5 years of math related subject. This guy is straight up lying or trolling.

[@abyssmage6979]

​​@@WsciekleMleko exactly. People shit on school because "muh education system bad" and forget about all of the interesting stuff they actually teach. No, you're not heroes trying to fight against the big bad. You're just lazy and want an excuse to keep being one.

[@Dr_Larken]

Um, how long did it take you to graduate grade school? 13 years of math! Even if you did start learning maths in PK or K to 12, it’s basic mathematics until about grade 4 when you’re starting to build off the foundation of that such as algebra, geometry etc In other words, the maths ain’t mathing!

[@brahmatejachilumula2668]

Beat graphical experience with a clear information, Really enjoyed throughout the video !!!

[@kakandeemmanuel7410]

I cannot tell how much excited this video has got me once I realized I am understanding every single step effortlessly.😂😂😂 Thanks so much for the explanation. God bless you!🙏🙏🙏🙏🙏🙏

[@Bhuvan_D]

That was fire bro! Gonna have to rewatch to understand the back step, but a lot clearer than most videos

[@krishnagupta31]

this is the most intuitive video I have ever come across. Amazing work!!!!!

[@RoodJood]

simply the best presentation on the subject

[@yewdimer1465]

On the computational graph @35:22 can anyone explain why the partial derivatives w.r.t the polynomial terms multiplied by the constants (i.e k_0, k_1 * x, ..., k_5 * x^5) suddenly had a negative sign (-2 * delta_y1)? The partials were back-propagated from a summation operation so I'd think they would remain positive... Or is it just an error?

[@ArtemKirsanov]

whoops, you're right, there shouldn't be a minus sign there! sorry about that. Good catch!

[@eradubbo]

Best description on the topic on the internet!

[@balajimarisetti4245]

Excellent explanation of back-propagation, the building block of machine learning. Thanks a lot.

[@4th_wall511]

bro im 2 minutes in and your graphics are insanely good I can already tell this is going to be a treat. Holy smokes man I'm having a graphicgasm

[@obsidianSt6761]

this video has amazing animations. You/your team clearly have a very high attention to details

[@andrewshort6440]

Magnificent work, from the beautiful, creative, elegant design, to the mastery in teaching. Thank you!

[@shizzm1990]

Some people just want to see the world learning. Great Video Artem!

[@1ProsperousPlanet]

Wow amazing thank you. Ive read and watched many videos on this topic and this is the one where I finally "got it"

[@maxwell77176]

Thanks

[@fosowl]

Glad to see ML related video from you ! As you have neuroscience background I would love to see some video that compare the current state of the art architecture work in ML with some of the inner working of the brain. For exemple if there are any structure in the brain with some ressemblance with GPT/transformers architecture, even thought the brain is light-years away I think that could be interesting :)

[@developersteve1658]

27:27. It clicked here. Seriously amazing video. Honestly, all your videos are. Thank you so much.

[@delete7316]

As soon as I saw this video, I knew it was going to be the best of this kind on the Internet. And it was. Fantastic video!

[@OmarElghamry1]

So much effort, in this video, the quality of the content at the same level of 3B1B, keep it going man.

[@wrtcookiedelta7560]

I should be watching gameplays but here I am procrastinating. Jokes aside, im 13 minutes into the video and astonished by your crystal clear explanations and the quality of your material, this is gold.

[@TurinBeats]

Waiting patiently for the second video 🫰♥️. Much love from Kenya, thank you for making me understand back propagation. Started watching your channel because of Obsidian, stayed for the AI lessons 🫰.

[@sonessz]

What an amazing video. I hope one day they come up with some world prize for 'free education heroes'. 173k views for a video like this is simply disgusting. This guy deserves maybe 2 billion views. God damn it, that makes me mad.

[@noth606]

Ehm, ya do realize this flies over the head of most people, you'll have to stack thousands up to find one person who is interested and can understand this properly. It is also not really needed for a plumber or a bakery cashier to understand ML improvement/approach velocity which is what I'd call this in a sense. Or, a visual way to pick a good method for it.

[@eurob12]

Very well explained how backpropagation and how the loss function helps in determining the optimal minimum by using calculus, great detail which helps newbies like me understand this complex topic much better.

[@nayanahgowda3219]

Hands down the best explanation I have seen so far! So clear and easy to understand!!

[@StratosFair]

This video is an absolute masterpiece, congratulations

[@arjunrao9978]

This video has an amazing and easy-to-understand explanation of the basics of Calculus. Many Thanks to the Creator 🙏🏼

[@imran_v1.0]

Brilliant video! The math, detailed visuals and explanation are excellent. Thank you.

[@connormcmk]

Thanks!

[@WayneLopez-w9d]

Your approach to trading is truly impressive. Thank you for teaching me so much!

[@simaitools]

Watching this video was like a breath of fresh air after some heavy math calculations! The visual explanations really helped solidify my understanding of backpropagation. I appreciate how clear and easy to follow the graphs were. Keep up the fantastic work! Can't wait for more graphic doses like this.

[@philipm3173]

This is just superb, thank you Artem! Timing couldn't be any better as the gradient descent algorithm was mentioned in Grahaene's "How We Learn" which I'm currently reading.

[@martonbalassa8128]

This is the best youtube channel in my feed, and I have many.

[@kalebnegussie8140]

Excellent explanation. I am going to rewatch this a few more times. Well done and thank you.

[@dejavus]

Huge thanks to you, now, I fully understand how the Backpropagation is implemented in code!!!

[@AkshayKumar-sg8qm]

That's the most amazing way of explaining such hard things to understand

[@chakravarthyelumalai8408]

A million dollar explanation. Thank you @Artem

[@benjamin6729]

Such a good video. Have liked and subscribed! I love the Curve Fitter 6000 machine in the animations to explain these concepts. Most textbooks are just too abstract and confusing but you have done a great job.

[@Тима-щ2ю]

WOW!!! The amount of animation you have made is just incredible. I would really like to not know about backprop again in order to fully appreciate this video!

[@TheForbiddenLOL]

You're doing pure ML content now? Excellent! Always glad to see more of your work, looking forward to watching the beautiful manim visuals and clear explanations as usual.

[@ArtemKirsanov]

thanks! ;) Yep! The channel so far has been a reflection of my research interests, and since i've joined an more computational theory neuro-AI lab, i figured more ML content with relevant topics of what i'm learning could be a nice addition

[@Amejonah]

@@ArtemKirsanov the reason, why I am watching your videos, is exactly because of the fact you draw common traits and differences between biology (neuroscience) and ML/"models" of it. Thank you for these!

[@francescobranca653]

Very insightful video. Can't wait to see the second part. I would really love to see a video from you on spiking neural networks too!

[@ahumanperson3649]

Great video! Very elegant explanation of back propagation, and I’m super excited to see the different mechanics of biological neural networks! Keep up the good work.

[@ks0ni]

Wow, hats off to you! Can't even imagine how long it takes to make something like this

[@soniferous]

Mindblowing. Just the video I was looking for. TBH, initially, I was a bit put off by your English as I am not a mothertongue myself. However, your knowledge, competence, hard work and research behind this video got me hooked. Liked and subscribed. And I will be watching this video many times.Well done!