Backpropagation, step-by-step | DL3

[@rbm0307]

You might think that your videos are fodder for university students boning up on a subject, or mathematicians/engineers in the early stages of their careers - basically that you cater to a younger audience. I'll have to prove you wrong. I'm in my early 60's and have been involved with information technology in some form or fashion my entire career. I enjoy learning; always have. I've viewed many of your videos only because they interest me and have you subscribed on my KZbin account so as to get notifications of updates. I find the topics about which you speak fascinating and am a bit jealous of those university grads today who now have access to this material at their fingertips. I wish with all my heart that I was able access these videos back when I was in university. It would have made life SOOO much easier for me back then. Your pedagogic skills are astounding, demonstrated by your ability to communicate difficult subjects precisely, concisely and simply. The animation format is integral with the presentation, adding to the delivery of the material. I salute you!! Please keep these videos coming.

[@zottasi]

I could not have said it better.

[@johncoleman1930]

That is very inspiring to hear you say I am a student rn, not in university though, but I love learning and hope to continue to foster a love of learning.

[@jinalin9351]

Well said.

[@richardsleep2045]

This 65 yr old ex s/w developer completely concurs!

[@landwarrior348]

And great thing about democratizing this learning is that kids in Africa or Europe or Asia have access to same content to what kids in America have. This makes me wonder if learning becomes easier or easily accessible, what the kids of future will be learning?

[@kevinconnolly6450]

This series is totally brilliant. I am 73 years old and used to teach mathematics. I am still learning stuff and with the help of sites like yours it makes it so much easier. Have you thought of doing any videos on the really complex subject of real analysis/. Keep up the good work. Kevin Connolly

[@damian_smith]

Cool! I'm mid forties and about to do a CompSci MSc. Gotta keep my own neurons going!

[@yeimarsoto3196]

​@@damian_smithSame! Im 31 getting my PhD in CS! Hopefully i dont need to be in a retirement home at 35.

[@DarkStar-im]

hello, I think you are 76 years old now, if you want, you can become a Muslim and enter paradise forever in the hereafter. "But those who believe and work righteousness,- no burden do We place on any soul, but that which it can bear,- they will be Companions of the Garden, therein to dwell (for ever)". (Al-A'raf [7] :42) ( i am not a bot)

[@miyagi420]

​@@DarkStar-imThis is the wrong place to spread Islam.

[@miyagi420]

No one is converting to Islam from a reply to their KZbin comment.

[@Niki_0001]

I can't claim to have understood everything from the first watch-through of this series, and I will watch these videos again with pen and paper in hand, but even this first viewing has made neural networks go from pure witchcraft and wizardry to something that actually makes sense in my head. I can't possibly thank you enough for posting these videos.

[@wajdanali1354]

and I thought everybody else here except me is a genius

[@joshdominguez2765]

"...has made neural networks go from pure witchcraft and wizardry to something that actually makes sense in my head" I died coz I've never related to something so much xD D:

[@Dr_Neo_Cortex.uka_uka]

I thought that I was the only one who did it.

[@GabrielCarvv]

I did not understand a single word. Please help me.

[@fillthedao]

@@GabrielCarvv If this video did not explain this to you then nothing will I'm afraid... Don't get me wrong - it's not about you - its a tough subject but this is by far the most approachable presentation I have seen out there. I can't even imagine how one would present it to the fellow human being in a more approachable way... ;-) PS. Have you watched the previous 3 vids from his series?

[@ThePRASANTHof1994]

Anyone else smiling through all of his videos because you're understanding so much so well like never before?

[@makingmediocremachines2133]

yup

[@jehanbhathena6270]

Yes!!!!!

[@angelbythewings]

yeah man!

[@vladimirbosinceanu5778]

Every single time.

[@pranavprameshgandhi9392]

Hell yeah

[@kemsekov6331]

I must say it. I am 100% seriously learnt English just to be able watch your videos. This is the content that will help everyone wandering grow stronger in their favourite subjects. I thank you for your work from the bottom of my heart ❤️❤️

[@adityashukzy]

Note to myself: Aditya, if you're having trouble understanding, read this. Scroll to 06:17. Listen to what he's saying, "In a sense, the neurons that are firing while seeing a 2, get more strongly linked to those firing when thinking about a two." Now, pause the video and listen. All this is is just a fancy way of saying: when we show our model a picture of a handwritten 2 and we tell it, "hey listen up this thing is a 2" (like in our training set we have labels for our input pictures) and then we find out the activation units that have a say in influencing the hypothesis value of the 2 label, ie, the activation units which can heavily increase or decrease the value of the output unit for the label 2, we tell those activation units "hey guys when you see something that resembles this thing, fire up the 2 label, ie increase the hypothesis value of the label for 2. Basically, we assign those activation units greater weights (or parameters) which influence the hypothesis value of the 2 label more (so that they can actually give us the right answer and say "oh look this is probably a two". I hope this helped and didn't complicate it further. If you don't get it, go over the video a few more times and review Andrew Ng's notes on this in the ML course on the Backpropagation lecture in week 5. Cheers, bro. Love you.

[@hzmuhabbet]

Thanks dude, it really helped. I am trying to follow Ng but after a point it sounds me like an Indian language. Sometimes it might be a bit difficult to visualize it in mind, and Ng is really a bit 'bad choice' about that part. I am not saying that he is a bad teacher but sometimes I feel like I am surfing through the universe...

[@adityashukzy]

@@hzmuhabbet I understand. Ng gets caught up in notation quite a lot so it's difficult to follow his intuition. I actually just noted this down for myself for future reference but I'm glad that you and other people are finding it helpful in understanding! I wish you well on your ML journey!

[@HARIHaran-ks7wp]

I finished the Andrew NG course except for the programming assignment part. I felt his explanation wasn't quite reaching my head and now watching this series has blown me away at how easy it's to understand things when they are visualized. Thanks for sharing your note snippet, pretty cool!

[@nicolasstencel7775]

nice

[@douglaswolfen7820]

I love that you're being so friendly and positive to yourself in these notes

[@dinub8414]

I find it astonishing how well you convey some of the intricacies here, way better than most of ML practitioners who teach the public, whether as youtubers, online instructors or public speakers who end up on video on the internet. I very much resonate with the way you frame things, the metaphors you choose, your visualizations and of course your evident love for understanding and sharing thereof. Your work is a great gift to all of us - students, engineers, researchers, philosophers, random viewers from all walks of life. Thank you.

[@hbhavsi]

Watching this in Dec 2024. The level of pedagogy in this series is just unmatched. And so is the animation. Kudos!

[@Kaixo]

You are amazing!! Best explanation I've seen on the internet yet!!

[@vgdevi5167]

Hello, I'm impressed by the way he explained this topic too, but I'm looking for more such great quality resources, youtube channels, books on deep learning, and also math and comp science in general, what do you recommend?

[@TopGunMan]

These visualizations are spot-on. Only a few people in the entire world need to make a great explanations backed by powerful visualizations about a topic - the rest of the world just needs to discover these. So much time wasted by learners trying to locate easily-digestible information, among all the inferior presentation methods out there. Glad to have found one of the best for this topic.

[@user-qj6hl5xb8q]

It is challenging to go through books and lectres from authors and professors trying to sound smart.

[@brianevans4]

Your animations are amazing!

[@MeowliciousMeow]

How are the animations made? It is great!

[@florianro.9185]

He has built an animation "engine" in python, which you can find on his GitHub

[@MeowliciousMeow]

Thanks!

[@thescientist7753]

Brian Evans a

[@chinmaydutta3783]

github.com/3b1b/manim

[@chaohongyang]

3blue1brown > Indian guy on youtube >>> my CS teacher

[@MySkittlesRainbow]

First time I caught myself having a moment of awe while watching educational content. The production value is incredibly high... the way the connections twinkle and move to represent adjusting the weights, the small animations, the descent into various shapes, how the little arrows indicating the desired change move and change size. Beautifully put together! Thanks a lot!

[@davidswygart7472]

I am a neuroscientist, and all your comparisons of actual neural networks to artificial neural networks seem pretty spot on to me. This is a great video series.

[@srirams0071]

Great explanation, your team is awesome. "A drunk man stumbling aimlessly downhill,but taking quick steps" is the best analogy ever for Stochastic gradient descent. :-)

[@youtubeviolatedme7123]

what the deuce?

[@PeterGriffinLovesLois]

stewie

[@ivoryas1696]

​@@youtubeviolatedme7123 Nah, it's actually not that uncommon. My material science professor even used it when discussing some Quantum processes! 😁

[@xanthoptica]

"A drunken sailor walking a plank" is often used to describe fixation of an allele by genetic drift in a small population. So next time you've had one too many, explain that you're feeling a little too stochastic to drive.

[@shiladitya7739]

No form of words can express enough what magic you're creating! I don't know how much you actually think you impact us.. but let me tell you Grant, your effect on my life is immeasurable! And the fact that you learnt it the hard way, and made it so simple for us, so that we don't have to go through the same, makes me respect you even more and more every single day.. Thank you so much.. 3B1B is undoubtedly the best channel on KZbin..

[@cameronadams4366]

You are a wizard when it comes to animations and understanding

[@vgdevi5167]

Hello, I'm impressed by the way he explained this topic too, but I'm looking for more such great quality resources, youtube channels, books on deep learning, and also math and comp science in general, what do you recommend?

[@brendawilliams8062]

You would like Quaternions.

[@HexatrackStudios]

Hey! Just want to shout you are my HERO! I'm an AI and ML researcher and I LITERALLY cry watching those awesome explanations! YOU ARE MY HERO!

[@TheSimonvdp]

I disable adblock for this

[@3blue1brown]

+Simon van der Poel Funny, because I disabled ads for this :)

[@BrianFaure1]

What's the first rule of adblock?

[@BrianFaure1]

TIGuardian - Apologies, I was talking to Simon, the first rule is you never talk about adblock.

[@ntwede]

I tried, but my adblock uses neural networks. It decided it would rather re-enable itself.

[@krakenmetzger]

Bro there's ads in the video

[@AishwaryaAR0013]

I've watched these videos 3 times and everytime I watch them, I feel a bit smarter. It starts with understanding little to progressively understanding more and more and finally seeing the big picture. Can I say I've understood everything? No. I am getting there and I'll be coming back for these again. For anyone who's feeling discouraged, I can assure you that you'll get there. Thank you so much for creating quality content that brings the driest, most theoretical concepts to life! You're a hero.

[@vgdevi5167]

Hello, I'm impressed by the way he explained this topic too, but I'm looking for more such great quality resources, youtube channels, books on deep learning, and also math and comp science in general, what do you recommend?

[@TheVilivan]

Thank you for saying this, I was pretty confused by some things in this series and I'm glad to hear that it gets better the more times you watch it. Learning some fundamental stuff like linear algebra and calculus would probably help as well, I really should get onto that

[@thismakesnosense]

​@@vgdevi5167 Hey, this might be a little late but there are some links to learning about Neural networks/Backpropagation in the description if you haven't looked at them already.

[@makebreakrepeat]

I just want to say that I love that you're moving into the mathematics of ML. The visualizations convey the concepts so well!

[@raver8056]

I love these moments in maths or other sciences, where you have this one singular moment of pure enlightenment. You got me at 7:37 ....I got goose bumps when the first column of "+" appeared. This is just brilliant.

[@donniegoodman8679]

All of the videos shown on this channel have been written so well. Even I can understand, mathematics was a complete mystery for me in school. I have a sixth grade education and I feel really smart after watching one of these programs. Thanks so much. I'm really grateful that you have taken the time to educate the people that have a hard time understanding but still want to learn. What you're doing is just as important as any other volunteer or charity work. I'm excited I found this .

[@AyurvedaGyan]

your videos have made it incredibly easier to grasp fundamental concepts of ML. Great work!!

[@lilysu2529]

I love how you enunciate the words as if you are truly interested and into the topic. The art direction and the animations are also on point. Kudos to the animator!

[@Kishimita7204]

Ngl man one of my Goals, once I get a good enough job. Will be to be able to live contribute to your patreon. Your videos have been so entertaining, inspiring, and helpful that I hope the move that I give allows you to continue with this channel.

[@trevinsmall]

All I can say is thank you! Every 3b1b video that I have watched is simply incredible. The chronology of ideas introduced throughout a video makes complex topics easy to follow, and the animations are BEAUTIFUL. I have never been able to visualize math so well before. You are an amazing teacher and have contributed invaluable knowledge to society! I'm still in shock that educational resources this profound are available for free. I truly appreciate the work you are doing!

[@vgdevi5167]

Hello, I'm impressed by the way he explained this topic too, but I'm looking for more such great quality resources, youtube channels, books on deep learning, and also math and comp science in general, what do you recommend?

[@gmish27]

I've been reading the book for quite some time but your explanations using animations have pushed my understanding to new levels. So many thanks to you. PS: Keep the background music. Holds our concentration for a long time.

[@mark.fedorov]

Man, the clarity and the animations make your videos masterpieces

[@FelipeLoeraToledo]

The way you teach such a complex topic is astounding

[@saitaro]

Grant, you're a diamond.

[@General12th]

no hes a human

[@-long-]

you must be fun at the party

[@timangar9771]

@@-long- I actually found it funny. Also, THE PARTY sounds dramatic, am I invited?

[@duykhanh7746]

@@timangar9771 you gotta admit that we have weird sense of humor for watching this video for fun

[@sumedh-girish]

no he's a pi creature

[@bikrammajhi3020]

Thank you for what you are doing. I have just started learning ML and I find your video really helpful. Thank you again from bottom of my heart for everything you are doing. I just wanted to let you know that you are really making changes and inspiring youngsters like me.

[@UtsavMunendra]

Kurzgesagt and 3Blue1Brown in the single hour! I am in heaven.

[@grainfrizz]

Utsav Munendra ikr?!?!?!

[@zbzb-ic1sr]

Utsav Munendra I was about to comment this.

[@shreyashervatte5495]

So true!

[@Martinmarshallmargella]

this is fav youtuber so farrrrr

[@AlexiLaiho227]

haha me too

[@junaidfaruqui5741]

bro .. the real magic is how effectively you make such a hard subject seem understandable to a layperson .. bless you

[@vgdevi5167]

Hello, I'm impressed by the way he explained this topic too, but I'm looking for more such great quality resources, youtube channels, books on deep learning, and also math and comp science in general, what do you recommend?

[@kushh7550]

I never understood neural nets like this before... Thanks!

[@LakTheShadow]

It is stunning how much better and clearer you can explain all this than my professor at university, and you are even a lot faster! Thank you so much for producing all those brilliant videos. Some of some help me understand what my professors are trying to tell me, and some of them are just fascinating. Keep on going!

[@vgdevi5167]

Hello, I'm impressed by the way he explained this topic too, but I'm looking for more such great quality resources, youtube channels, books on deep learning, and also math and comp science in general, what do you recommend?

[@CedricChee]

So far, I think this is the best intuitive intro to backprop I've seen. This channel means a lot to me. Because of its teaching style, I managed to get back to learning and grok maths while I was studying machine learning last year, 13 years since doing my undergrad in CS. It's almost the end of 2017 and I still keep hearing from the people I talked to that they fear/hate maths because they think it's a tough subject to tame. May be this example tell us why our education system (internationally) is still broken? As an aside, David Perkins, in his book "Making Learning Whole" also touch about this widespread diseases of the educational system, namely "elementitis". I think we can do better. Grant is doing great work to lower the barrier and making math more accessible to everyone. This is not an easy feat. I think we need some sort of concerted effort for encouraging more people to teaching maths or any subject through intuition. Visualization is one way to improve the teaching methodology. We can also distill intuition from stories, feelings, situations, etc. More examples, see: distill.pub/ colah.github.io/

[@fygarOnTheRun]

Cedric Chee spot on! And thanks a lot for those links.

[@AR-rg2en]

Thanks

[@zirion8446]

I'm graduated in Statistics an Data Science by my local university. Study the subject for almost a decade. For years I heard about backpropagation. No one knew how to explain it (or worse, knew how it worked). It always was "a magic algorithm that adjusts weights". This video did in 13 minutes what many professors during years couldn't do to me: explain in a simple way the logic behind it. It's just perfect: well done, free an with a relevant sponsor in the end. Best video I ever watched. Congratulations!

[@vgdevi5167]

Hello, I'm impressed by the way he explained this topic too, but I'm looking for more such great quality resources, youtube channels, books on deep learning, and also math and comp science in general, what do you recommend?

[@lianmccc]

This series is better than my in total 6-hour lecture from school.

[@PunmasterSTP]

All of the other commenters beat me to it, but I can't express enough how insanely high the quality of these videos is, and how much they help teach people new things. Thank you so much for making all of them and sharing some of your vast amount of knowledge!

[@CyberCookieMonster]

I am a “seasoned” systems engineer, inventor, and IT professional. IOW, I’m old (40+), lol. Like @Robert MacKinnon has said, you may think that your wonderful videos are only for those new in the field or younger folks in general. This couldn’t be further from the true. Your ability to explain things with clear diction, amazing graphics, and compelling story telling is phenomenal. They help me to not only refresh to get spun up quickly, but also serve as a point of reference for me to direct those newer in my teams; as your ability to run through this at just the right level and pacing far exceeds mine. I would go so far as to suggest that you freelance as a guest speaker or instructor for hire for larger firms having folks wanting to get spun up quickly. Apologies if this is old news and you’re already pursuing this. I simply feel compelled to tell you as we often don’t understand how awesome we are and need to be reminded by others. Consider this your reminder ;) Thanks again for your wonderful videos where you ask for nothing. I would pay for a Patron page if you get one to be able to ask for more content or explanation on topics such as the vanishing gradient problem prone to recurrent neural networks and how LSTM addresses it. I’ve been looking for a good one that does this graphically with intuitions. I haven’t found a good one yet, but I did come here first to check ;) All the best, CCM.

[@erikaambrioso8966]

Im taking a Neural networks course at my university right now and I'm finally starting to understand this stuff after watching this video series! Thank you so much!

[@orbitmarketing-usa]

This video is literal ART. Grant is my favorite artist.

[@jimmieboboyo]

Normally, I never comment on youtube videos, but I'm making an exception for this one. Even though you might not see this, I want to thank you for your contribution to everyone's education. And for FREE access to it too. Amazing videos with amazing presentation, structure, and explanations. I'm 3 years late, but please do keep up the great work!

[@sohamharnale2878]

This man is the eternal fountain of knowledge straight from the heavens

[@hbookreviews]

I am just in awe with the presentation of the concepts. The graphics, the use of animations, ... It is all very helpful in making us visualize such complex concepts. THANK YOU!!!

[@marcotroster8247]

I love the statement "the most confusing part is the notation". This is so true 😂 It would be really, really nice if you did a last video on how to program this network in a comprehensive way 😄 IMO the inventors of DNN chose all those activation functions and matrice multiplications really carefully to make computation feasible. But all this beautiful simplicity is gone once these unnecessarily complicated math formulas come into place. Honestly, you did an amazing job with your videos on DNNs and backpropagation. The first time I seemed to understand it.

[@SamuelJFord]

I have just found your videos. I have enjoyed watching science educators on youtube for many years, but these videos are the best examples of complex ideas being explained clearly whilst still being entertaining. I can't believe I had never heard of this channel before.

[@grainfrizz]

9:00 Eureka moment. I cried.

[@ChopinFlutist]

I actually got touched for real from these videos. A great example of the beauty of the math.

[@adiadiadi333]

Felt obvious to me 😕

[@Kiros37100]

aditya sai You are so intelligent! Wow!

[@sohamharnale2878]

Hahaha I feel you mate

[@milanstevic8424]

@Kiros37100 You're so bitter and sarcastic! Wow!

[@juliangonzalez28jg]

Implementing a neural network from scratch for a class project and this series helped me so much to get started. Thanks!!!

[@Michael-vs1mw]

* eagerly waiting for a video about convolutional neural networks *

[@3blue1brown]

+Michael Incog It'll be a little while. The next few videos won't be for this series, so I'll probably return back to this in a few months.

[@UtsavMunendra]

Computerphile did some videos on convolutional neural network

[@wanderingrogue3039]

3Blue1Brown Please please make one soon

[@alberttamazyan]

@@3blue1brown hasn't the few months passed? The whole community is waiting for a video on convolutional neural networks.

[@ciherrera]

@@alberttamazyan Ikr :(

[@jacqueskirstein9833]

Thank you so much for all the time and effort that you put into making a visual understanding of Neural Networks. I've been trying (admittedly on and off) to get a good intuition on Neural Networks for at least a year now. This is by far the best fundamental video I have come across to get an understanding of what the theory behind the scenes is for. I know that this will increase my (and many others') learning curve by a large amount. Thanks again!

[@JemimaGoodall]

This should be art. I don't normally comment on videos but I feel I must express my gratitude and appreciation for your amazing explanations and animations! Visualisation is so important and helpful and you have nailed it beautifully!

[@newbie4789]

I'm here doing a Masters in computer science and having seen these things in my classes, this video gave me full idea of just how beautiful this whole development is. Someone was really smart

[@veratsien2014]

This is a lifesaver for completing week 5 of Andrew Ng's Machine Learning course!! Been stuck at backpropagation for hours and finally found some clarity in this video. I love your channel! ❤️

[@Endothermia]

Haha, I'm going through exactly the same thing!

[@veratsien2014]

@@Endothermia lol, high five and hang in there mate. :)

[@ammararif7524]

Half the time I'm in awe of the quality of the content. The other half I'm learning. Thank you so much man.

[@shreyanshdarshan3199]

You have made life so much easier.

[@Julieggppppp]

I had attended 3Blue1Brown sessions on Vector and Calculus earlier before recently attending Machine Learning and Neural Networks. It is just amazing to see a mix of great content, great description along with excellent graphic visuals. It just makes it so comprehensible that way. I want to convey my thanks to the whole team and wish them continued success.

[@chriswangux]

Came for the knowledge, stayed for the Animations. Here I'm thinking how it can be done in After Effects, and turned out it's custom engine!

[@solidsnake013579]

best neural network explanation video on youtube hands down

[@ObitoSigma]

Now to watch all three parts at once!! Great video 3b1b, keep up the great work man! δ.δ

[@vgdevi5167]

Hello, I'm impressed by the way he explained this topic too, but I'm looking for more such great quality resources, youtube channels, books on deep learning, and also math and comp science in general, what do you recommend?

[@krikkiteer]

your videos are simply amazing.. awesome visualisation of what actually happens...i imagine a huge effort went into making those.... kudos - i almost never comment in that manner... greatest thanks and respect for that!

[@markdashark3443]

I wish I had the option to pay my college tuition to you instead of my university. Well done mate!

[@shukkkursabzaliev1730]

wow, I am surprise anyone knows this sophisticated concepts to explain them clearly, and yet you also accompany them by amazing visuals which I believe not easier to do. Thank you!

[@bricechivu8573]

I might misunderstand but maybe not. At 7:38, we want to reduce the activation of the neuron responsible for 3 right? So we should decrease the activation of the neurons that have a positive weight to 3, no? In the video, it's actually the opposite. For example, the first neuron has a positive weight with the neuron responsible for 3, so we should decrease its activation right? Can someone help me on that please?

[@nickgardner5641]

I had the same question. It seems like the color-coding of the video at this point is inconsistent? (it applies the same procedure for adjusting the preceding layer of neurons to all the other output neurons as it does to the output neuron for 2, when really the signs in the procedure should be reversed for all the neurons besides 2, since we want their activations to decrease?)

[@xXasd012xX]

same here, i'm quite confused about this

[@TheOnlyKeksSuchti]

You're saving my presentation! I didn't know how to explain neural networks to my fellow students but now I know how to do it! Thanks mate!

[@5yago.960]

Sorry, just noticed a mistake and wanted to point it out. What you are calling Stochastic Gradient Decent is actually called Mini Batch Gradient Decent. SGD is actually when you take a gradient and update the parameters at every single observation, while MBGD is where we split the data into subsets and update the parameters based on each subset (what you were referring to in the video). Loves the video though, a lot was clarified for me. Keep up the good work :)

[@user-fg6ng7ej6w]

this author has an amazing talent to explain complex things in a simple intuitive visual way. that what most of academia lacks. thanks

[@shengchuangfeng227]

At 7:37, shouldn't the activations associated with the digit 0 neuron (whose activation should be decreased) change the opposite direction? For example, if the neuron at the very top of the second-to-last layer increases activation, with a positive weight, the digit 0 neuron's activation would increase. Is that correct?

[@ludd8489]

Same question here...Someone has an answer for that...?

[@cvm7549]

This is the best video on machine learning I have seen so far. Thank you for making this complexity as simple as possible.

[@thevoodooninja]

I've never clicked on a video this fast

[@assaissa]

What for did you clicked on the video?

[@jmaccan11]

one of the better Machine Learning channels! Delivered very nicely. muchly needed for me and appreciated

[@toolegittoquit_001]

You guys just gotta stop blowing my mind ....💥

[@Spektrob]

This is by far the best educational content i've ever seen on youtube. Thank you and go on with it.

[@fakecubed]

Let's just say a quick thank you to all the mathematicians who wrote the highly optimized software libraries that do all this math for us so ordinary programmers can quickly get neural nets up and running on our training data.

[@hans-henrikstrfeldt7854]

I totally like how your commercial sponsor is totally relevant and not just toothpaste !

[@TheFadime123]

I think it'd be extremely beneficial if you were a complete lecture series for deep learning. Your way of teach is billion times better than this somewhat popular Andrew Ng lecture series, and also a lot more intuitive. Please consider this:) Best wishes

[@atul6147]

Sanderson>>> Ng

[@mattiasli]

extremely well made high quality material, every animation, every transition perfectly timed with the explanation. priceless

[@culedood123]

At 7:39 onwards: These are the effects that you would like to have on the activations themselves, not the weights. How would you determine how much you want the weights to change? Do you just go back until you reach the beginning layer, and then adjust the weights between the first and second layer to achieve the desired changes on the activations on the second layer, then adjust the weights between the second and third layers, etc until you reach the output layer?

[@raunak1147]

Have you found this out? I'm having the same confusion

[@philippos4330]

Same confusion here. Anyone with an answer to that?

[@christophallmacher9580]

I was wondering about the same fact. Especially since the desired "new activation level" depends on the weights. Thus if you change (or is just increasing possible?) the weights later, the backpropagated changes regarding the new activation level will be distorted. I would be greatful for any further insight! Thanks in advance! Btw: This video series is an awesome job! Keep on the good work :-)

[@raunak1147]

I found the answer, update each weight according to it's partial differential, so it only takes into account the effect of the nudge of that particular weight, not it's predecessor, nor it's successor, so pretty much start from the last layer, and go till the first one. I just thought I'd come here to tell you guys how to do it (got a convutional network working for me :))

[@cederveltman]

the graphic at 8:00 is mindblowingly good at explaining what you mean.

[@manjunathsastry7540]

Oh dear! Absolutely brilliant. This is one heck of video. How would you consider creating an exhaustive ML and Neural Network video series? I'm sure there are a lot of curious folk out there, waiting for it!

[@karanshete6534]

Thanks

[@m0rjc]

"This is where we were in the 80s/90s" - I did my degree in the 90s, so this is the level I learned to. The more up to date stuff would be interesting.

[@SallyxSm]

This is amazing thank you so much I stopped at 9:08 to leave a comment and someone else already commented about the Eureka moment Thank you so much you're a great teacher!

[@octavioperez8267]

As a visual learner, I can't thank you enough for this kind of video. Reading math notation is a piece of cake after seeing this!!

[@toostoned42069]

I would expect the training data to also include a bunch of random images that the network could classify as "not a number" (not hotdog 😉) That way, instead of just taking its wildest guess at which number it's detecting, it could also intelligently say "this doesn't look like a number to me"

[@kalebbruwer]

Nick V Yes, this would probably force it into more logical methods than what it turned out to be last video.

[@erilgaz]

it could also intelligently say "This doesn't look like anything to me." FTFY

[@skyacaniadev2229]

That's a good idea. Also I wanna see what will happen if alphabet and numbers are trained together.

[@allenwalker4703]

From what i understood i think you could just program it to say that whenever all the values in the last layer are either negative or really low without changing the back propagation process. Not sure tho.

[@tiagotiagot]

You need to be sure your "random non-number" is actually a lot of different things, otherwise the "not a number" result would have similar chances of being triggered as the number results when for example instead of alphabetical characters you present the NN with white noise it didn't train on.

[@sonOfLiberty100]

Man, i addicted to your videos, it is so deep and so easy to understand. Thx a lot

[@conoroneill8067]

I feel like I'm missing something with the functionality of backpropogation. So, I get that you adjust the weights between the last two rows to adjust how the second last row feeds into the last row in the optimum way, and then you change the second last way, and then you backpropogate the second row of weights away from the end, etc. But, what I don't understand is that if you change the second last weights, that's going to change the second last row of inputs, which means that the tweaked weights between the last hidden layer and the output will be wrong again. I'm trying to work out what I'm missing. Is it that all the layers of weights are tweaked simultaneously in order to be accurate? If so, I'm still not sure I understand how that works (I'll probably try and watch the video again in the morning when my brains properly awake.)

[@3blue1brown]

+Conor O'Neill Good question! Backpropagation is just giving the list of adjustments you will make, but you don't make them until all are computed. And of course, since this is part of (stochastic) gradient descent, that's a process you repeat multiple times.

[@alectoperez1383]

You bring up a really good point. The next video, which will cover the actual mathematics of how backpropagation works, does take that into account. If you've taken calculus before, you've heard of the chain rule, which describes how to differentiate an expression like f(g(x)). The chain rule states that d/dx f(g(x)) = f'(g(x)) * g'(x). The chain rule is general enough that it's applicable to pretty much all situations where you can express stuff in terms of an f and g, even when f and g take multiple inputs. The important thing is that it allows you to only analyze one input at a time, making it easy to compute stuff. What happens in the case of backpropagation is that you're taking the derivative in terms of each of the weights of the cost function, and the chain rule accounts for how changes affect other changes. The math is too complex to put in a youtube comment (mainly because youtube doesn't have mathjax, so I can't write a lot of the symbols I'd need, like summations), but the next video should explain all of that!

[@conoroneill8067]

Thanks! That helps! Thanks for a great video - I love your work and the effort and care you put into these videos. Truly incredible. Thanks again.

[@ashirizly]

Consider that the changes merited by each example are quite small (because each example alone doesn't carry enough weight to merit making big changes), so the changes of weights based on these values are still accurate. It doesn't matter if you do a tiny step after each example, or a larger one after a thousand examples, the over all effect of each example is still like a single tiny step in the direction it points to.

[@angelanikolova7354]

I am writing my internal assesment for school about neural networking and thank god your channel exists cuz everything about this subject is now clear to me. You explanations are perfect!

[@bartoszk.6451]

Sorry for a depth and detailed analysis, but shouldn't the colors of the arrows (related to the amount of desired change) appearing at 7:37 be in an opposite color (for digit outputs 0, 1, 3, ..., 9 i.e. last layer)?

[@Mr_Bull]

I think you are right

[@anoriginalnick]

Amazing use of visuals to explain concepts. Well done man ...

[@thewingdings1324]

Why don't you have ads on these. I have no money for patreon and I want to help you out

[@3blue1brown]

+TheWingDings1 I just think it's a nicer experience. If you have no money, don't worry about it, just watching is thanks enough.

[@johncoleman1930]

I know what you mean, the best thing you can do to help the channel is by subscribing,liking, and sharing videos as cliche as it sounds it does help creators, not a creator myself just someone who wants to see these kinds of educational channels grow!

[@codinghub3759]

@@johncoleman1930 I have subscribed, I like every 3blue 1brown video I see. But, I don't know any one who is interested in this kind of stuff. So, I can't really share

[@riddhimanna8437]

This is such a wholesome exchange!🥺 Thank you so much Grant and I wish the best for you and the others who commented here!

[@paskymail]

I had to stop seeing your video for a moment because I needed to write down my admiration for your work. It is outstanding the amount of intelligence, communication skills, careful graphic design and passion in very single frame of your videos. Congratulations and thank you for your contribution to maths education. See you in patreon!

[@maxsmith1733]

Hello, I'd like to start by saying your videos are amazing and beyond intersting. One thing I'm still confused about is how to update the biases through backpropagation? Thanks! and keep it up!

[@ybahman]

This!

[@just_a_duck3371]

I also don't get it what these "nudges" actually mean. Do I simply add value from computed gradient to the weight?

[@ayanSaha13291]

I would like to take the forum to THANK YOU for your efforts to make the extremely complex topics comprehensible. I am new to the subject and found a very good starting point to start my journey towards understanding a little bit of AI. Thank you very much.

[@TheBukkitArea]

AAAAAAAAA YEEEESSSS I'VE BEEN WAITING FOR THIS VID

[@omarcusmafait7202]

This channel is absolutely my favourite!

[@MrDiglenson]

I'm just starting out with Deep learning this quarantine and man...it's a bit difficult to get my head around these concepts.

[@JonesDTaylor]

Me too man! I am doing the course by Andrew Ng and things are getting much clearer.

[@MrDiglenson]

@@JonesDTaylor Me too 😁!

[@parthvasoya3562]

@Winston Mcgee hey..why??

[@plasmasheep4098]

@@JonesDTaylor same!

[@RitikSharma-pc5yj]

Include me too in your category...

[@castrojosua]

I changed my major from cybersecurity to philosophy and mathematics because of these videos. This is much better than any college course I’ve taken.

[@tejas8211]

Dude how are you making these animations

[@screwhalunderhill885]

he makes them in python

[@shaileshupadhyay3920]

Dr tenma how are u