How AI Learns (Backpropagation 101)

Explore the fundamental process of backpropagation in artificial intelligence (AI). This video show how neural networks learn and improve by adapting to data during each training phase. Backpropagation is crucial in calculating errors and updating the network's weights to enhance decision-making within the AI system. This tutorial breaks down the core mechanics of neural network training, making it easier to understand for individuals interested in AI, machine learning, and network training. By understanding backpropagation, viewers can better grasp how neural networks evolve to process information more accurately. Keywords: rosenblatt, AI, Artificial Intelligence, Neural Networks, Backpropagation, Machine Learning, Network Training, Data Adaptation, Error Calculation, Performance Tuning, Decision Making.

[@markheaney]

This is easily the best channel on KZbin.

[@TimBorny]

As always, worth the wait. You are a genius at distillation and visualization.

[@ArtOfTheProblem]

just finished this series, please help me share it: kzbin.info/www/bejne/hXe2amNje71ppsk

[@robertbohrer7501]

This is the best explanation of neural networks I've seen by far, and I've seen most of them.

[@ArtOfTheProblem]

thrilled to hear this

[@robosergTV]

3Blue1Braun is on par I'd say

[@bardes18]

This channel has more than lived up to its name! Turning technical explanations into beautiful art is perhaps the most elegant and pleasant way to abstract and teach complex subjects.

[@ArtOfTheProblem]

just finished this series, please help me share it: kzbin.info/www/bejne/hXe2amNje71ppsk

[@austinvw1988]

WOW!! This is the only video that I've watched that made me finally get it. The inclusion of the physical dimmer switch and weights in the neural net made me finally start to grasp this concept. Thank You! 👏

[@ArtOfTheProblem]

So so happy to hear this, glad I took the time. I need to get this video out there more

[@Aksahnsh]

I just don't understand, why this channel is not popular.

[@ArtOfTheProblem]

I know, i kinda stopped asking myself. I know it's due to algorithm changes in some way. because my videos don't even go to subscribers much at all

[@Aksahnsh]

@@ArtOfTheProblem True, even I didn't got it in my recommendation feed. I just realized that why there is no new video from you from long time, have to open your channel to manually find it. Clicked on bell icon though now.

[@ByteNishi]

@@ArtOfTheProblem Please, don't get disheartened. I really love your videos and eagerly wait for new ones :)

[@roygalaasen]

There are no truer words like these. It baffles me as these videos are at least on the level of other highly popular science/math youtubers. It feels kind of unfair. Even the videos made 8+ years ago are pieces of masterful art. Did any of the other youtubers even exist back then? (I guess some did.)

[@roygalaasen]

@@ByteNishi I am praying for the same. I am happy for the once a year schedule. At least there is something. Edit: I know it is a bit of exaggeration. There is at least 4 videos per year, which seems close to what 3b1b does nowadays as well.

[@TimBorny]

Seriously impressive. As someone currently applying to masters degrees in science communication, you are an inspiration. While a personal inquiry within a public forum is generally not advisable, I'm compelled to wonder if you'd be willing to be available for a brief conversation.

[@ArtOfTheProblem]

I appreciate hearing this. You can reach me [e-mail]

[@ArtOfTheProblem]

just finished this series, please help me share it: kzbin.info/www/bejne/hXe2amNje71ppsk

[@idiosinkrazijske.rutine]

The highlight of this day

[@baechlio]

Yay!!! My favourite channel finally uploads again.. to be honest the quality of your videos makes the wait worth it

[@yomanos]

Brilliant video, as always. The part on the explanation of a deep neural network was really well explained.

[@kriztoperurmeneta7089]

This kind of content is a treasure.

[@iMamoMC]

This video was great! What an awesome introduction to deep learning ^^

[@ssk081]

Great explanation of why we use a smooth activation function

[@ilovett]

This could be a Netflix series. Bravo.

[@raresmircea]

This, along with everything else on this channel, is fantastic material for schools. I hope it gets noticed by teachers

[@KipColeman]

College IT professor here... we are noticing! :)

[@mehdia5176]

Beautiful work coming from a beautiful biological neural network about the beauty of artificial neural networks.

[@NoNTr1v1aL]

Absolutely brilliant video!

[@rj8875]

After reading all day about tensorflow you just inspired me to go deeper on this subject. Thank you

[@ArtOfTheProblem]

woo hoo!

[@KhaliliStudios]

I’m always very impressed at the novel approach to teaching these subjects - another hit, Brit!

[@ArtOfTheProblem]

that you for your ongoing feedback. I worked super hard on this one,

[@chris_1337]

Fantastic work!

[@jayaganthan1]

Just wow. Awesome explanation.

[@ArtOfTheProblem]

STAY TUNED: Next video will be on "History of RL | How AI Learned to Feel" SUBSCRIBE: www.youtube.com/@ArtOfTheProblem?sub_confirmation=1 WATCH AI series: kzbin.info/aero/PLbg3ZX2pWlgKV8K6bFJr5dhM7oOClExUJ

[@TheFirstObserver]

This is a well-done, visual representation of artificial neural networks and how they compare to biological ones. I will say, the only item I might add is that the real reason the "gradual" activation functions mentioned in the latter half of the video are so useful is because they are differentiable. The functions being differentiable are what truly allowed backpropagation to shine, as the chain rule is what allowed the error of a neuron to be determined by the error of the neurons following it, rather than calculating a neuron's error from the output directly each time.

[@ccc3]

Your videos are great at making someone more curious about a subject. They have the right balance of simplification and complexity.

[@ArtOfTheProblem]

appreciate the feedback that's what i'm looking to do with these videos. Stay tuned to the next in this series it took me a long while to write

[@elektrisksitron9054]

Another amazing video!

[@CYON4D]

Excellent video as always.

[@zyugyzarc]

now that's a brilliant explanation of neural networks. better than anything Ive ever seen.

[@ArtOfTheProblem]

glad you found it

[@poweruser64]

Wow. Thank you so much for this

[@interspect_]

Great video as always!!

[@midhunrajr372]

what a nice presentation..

[@acidtears]

Great video! Do you have any idea how these types of Neural Networks would respond to visual illusions? I'm writing my thesis about Neural Networks and biological plausibility and realized that there seems to be a disconnect between human perception and the processing of Neural Networks. Either way, incredibly informative.

[@fungi42021]

always looking for new content to watch on this topic.. great channel

[@ArtOfTheProblem]

i'm so happy you found this series as it isn't ranking well yet. i have more videos coming out in this series soon

[@ByteNishi]

Love your videos, can you please post videos more often. Thanks, your videos are always worth the wait.

[@ArtOfTheProblem]

thank so much. I can't possibly post more often but what I can do is promise to continue for another 10 years :)

[@mridhulml9238]

Wow this is really really great...you are really good at explaining

[@DaveMakes]

great work

[@lalaliri]

amazing work! thank you

[@ArtOfTheProblem]

appreciate the feedback

[@srabansinha3430]

As a Medical Student studying Neural anatomy and Physiology , this is a whole new perspective to me !!! Keep teaching us More !!You are the Best teacher :)

[@ArtOfTheProblem]

this means a lot, thanks for sharing

[@sumitlahiri209]

Amazing Video. It was really worth the wait. I have watched all your videos. Just awesome I would say. Best channel for inspiring computer science enthusiasts

[@ArtOfTheProblem]

that's really cool to hear you've watched them all. thanks for sharing

[@sumitlahiri209]

@@ArtOfTheProblem I watched all of them. They inspired me to take up computer science. I really love the video on Turing Machine. I share your videos in cirlces as well.

[@ArtOfTheProblem]

@@sumitlahiri209 you can offer my no higher compliment

[@shawnbibby2934]

The term Distributed Representation when compared to musical notes makes it seem like it has its own image Resonance or Signature Frequency. As if we really are seeing or feeling the totality of the image of the firing neurons. We seem to be addicted to understanding perceptions from a Human Point of View, imagine if we could begin to find translations to see them from Animal Point of Views, Different Sensory Combination Point of Views and different combinations of Layered Senses. The potential is infinite. I like the addition of the Linear Learning Machine versus one that forgets and uses Feelings. It seems that by combining both memory styles you would have more unique potentialities in the flavor pot of experiences, especially when the two interact with each other. Not to mention the infinite different perspectives they would each carry while traveling through time. Small and large epochs of time. I seem to keep coming back to the Encryption / Decryption videos on how it requires complete Randomness to create strong encryption and how the babies babbling was seemingly random in nature, which begs the question, was it truly random or could we simply just not see the pattern from our limited perspective? What is the scales and size of the pattern? And what conceptions and perspectives need to merge to simply find the Key to interpreting it?

[@ArtOfTheProblem]

yes, i'd say "feeling"

[@yagomg7790]

Best explanation on youtube. Keep it up

[@ArtOfTheProblem]

appreciate the feedback

[@karolakkolo123]

Wow! The most amazing explanation on the internet probably. Will actual calculations be talked about in the series? (e.g. backpropagation calculus, etc) or will the series be mostly conceptual? (Any way I'm sure it will be interesting and of an unmatched quality)

[@ArtOfTheProblem]

great question and thank you. No more details on backprop calculations (there are lots of good videos of that) in order to focus on other key insights. stay tuned!

[@username4441]

11:49 And the narration model took how long to-train?

[@shawnbibby2934]

I would also love to see a video of all the terminologies used together and defined in a single video. Such as Bit, Node, Neuron, Layer, Weight, Deep learning, Entropy, Capacity, Memory etc. I am trying to write them down myself as a little glossary. There meanings are so much greater when they are grouped together.

[@ArtOfTheProblem]

thank you! i was thinking of making a super edit of this series just need to scope it correctly...

[@solsticeprojekt1937]

Hi! Three years late, but at 0:26 where you say "feelings", you describe something much better explained as "realizations". The answer to the "Why?" about this lies behind the saying "an image speaks a thousand words". The part that takes care of logic works in steps, sequentially and can analyse the "whole" of a realization, just like we can put feelings and ideas into words. This works both ways, of course, but the path from words to realizations is a much, much slower one.

[@ArtOfTheProblem]

Took 2 years to finish this one, finally live would love your feedback: kzbin.info/www/bejne/hXe2amNje71ppsk

[@thisaccountisdead9060]

I'm not an expert or anything. But I had just been looking at networks. I was interested in the erdos formula: - erdos number = ln(population size) / ln(average number of friends per person) = degrees of separation for example it is thought there is something like 6 degrees of separation and an average of 30 friends each person among the global population. But I was also looking at Pareto distributions as well: - 1 - 1/Pareto index = ln(1 - P^n) / ln[1 - (1 - P)^n], where P relates to population of wealtheist and (1 - P) is the proportion of wealth they have.. for example if 20% of people have 80% of the wealth then P = 0.2 and (1 - P) = 0.8. n = 1 (but can be any number... if n = 3 it gives 1% of people with 50% wealth) and the Pareto Index would be 1.161. Whether it was a fluke I don't know? I did derive the formula as best I could rather than just guessing. But it seemed as though the following was true: - 1 - 1/Pareto Index = 1/Erdos Number Meaning that the Pareto Index = ln(population size) / [ln(populationn size) - ln(average number of friends per person)] Suggesting that the more friends people had on average then the lower the wealth inequality would be. Which I thought was a fascinating idea... ...But it also seemed as though the wealtheist actually had the most 'friends' or 'connections'. So the poorest would have the least connections while the wealthiest would have the most connections - in effect poor people would be channeling their attention toward the wealtheist. Like the top 1% would have an average of around 2,000 connections each (*and a few million dollars) while the poorest would have as little as 1 or 2 connections each (*with just a few thousand dollars... *based on a share of $300 Trillion). Maybe in like a neural network the most dominant parts of the brain could be the most connected parts? As I say I am not an expert. I was just messing around with it.

[@Arifi070]

Great work! However, although the artificial neural network was inspired from the working of our brains, visualizing the network inside a head, can give a wrong idea that the human brain works that way. In fact, it is not like a feed forward neural network. [Just a side note]

[@KalimbaRlz]

excellent explained

[@ArtOfTheProblem]

thanks for feedback, have you watched the whole series?

[@KalimbaRlz]

@@ArtOfTheProblem yes I did!, thank you for all the information

[@trainer1kali]

a message to the one's responsible for the choices of the background music to translate the mood: "you're pretty good". P.S. In fact - you are AWESOME.

[@ArtOfTheProblem]

thank you, glad it's working

[@zuhail1519]

I want to mention here, I watched the video halfway and I must say, I am a complete noob when it comes to biology but without making things complicated for a person like me, You made it so incredibly clear to me to appreciate how amazing our brain works and generalizes stuff, especially with your example of the short-story (can you please mention that author name, I quite couldn't catch it and cc are not clear either). Thank you for making this content, I'm grateful. Jazakallah hu khayr

[@ArtOfTheProblem]

thrilled to have you, i'm still working on the final video in this series so please stay tuned. Was it "Borges'?

[@zuhail1519]

@@ArtOfTheProblem Already have my seatbelt fastened !

[@KDOERAK]

Simply excellent 👍

[@ArtOfTheProblem]

thanks, stay tuned for more!

[@iamacoder8331]

Very good content.

[@ArtOfTheProblem]

more on the way thanks

[@slazy9219]

holy shit this is some next level explanation thank you so much!

[@ArtOfTheProblem]

super glad you found it, still working on this series

[@harryharpratap]

Biologically speaking, what are the "weights" inside our brains? What physical part of the brain do they represent?

[@emanuelmma2]

Amazing Video.

[@ArtOfTheProblem]

would love if you could help share my newest video: kzbin.info/www/bejne/a3bGgmR_mKqAfLM

[@ahmadsalmankhan3200]

Amazing

[@ArtOfTheProblem]

:))

[@lakeguy65616]

so adding hidden layers allows a NN to solve more complex problems. How many layers is too many? You are limited by the speed of the computer training the NN. I assume too many layers allow the NN to "memorize" instead of generalizing. Any other limits on the number of hidden layers? What about the number of neurons/nodes per layer? Is there a relationship between the number of inputs and the number of neurons/nodes in the network? What about the relationship between the number of rows in your dataset versus the number of columns? As I understand it, the number of rows imposes a limit on the number of columns. Adding rows to your dataset allows you to expand the number of columns too. Do you agree or have a different understanding? OUTSTANDING VIDEOS! John D Deatherage

[@ArtOfTheProblem]

super great questions. I hope others can chime in. just wanted to add that in 'theory' you only need one hidden layer if it was really really wide to solve any problem (see universal approximation theorem), but in practice that doesn't work. and yes if the network is "too deep" it will be too difficult to train, so you need a sweet spot. when it comes to how wide those layers need to be, the most interesting research to me is how 'narrow' you can make them to 'force' the network to abstract (compress/generalize) the information in the middle. you can also make the hidden layers very wide which will cause it to 'memorize' instead of generalize. i didn't quite follow your column / row question though

[@user-eh9jo9ep5r]

What input could be done to heal neurons to basic correct stages, for give correct outputs

[@user-eh9jo9ep5r]

What if one layer behaviour is different from expected, and not recognised as correct, but other layers are give output from sorts geomerical inputs on the level sense impulse, what can be done to filter inputs and recieve correct outputs

[@robosergTV]

Isn't universal approximation theorem the mathematical proof NN can solve and model any problem/function?

[@ArtOfTheProblem]

right but that is only in theory, in practice the number of neurons is impractical to make it "practically impossible" to impliment.

[@robosergTV]

@@ArtOfTheProblem true, but at the end of the video, you were talking something about "we still don't have a mathematical proof of how NN works" or something like that.

[@user-eh9jo9ep5r]

If network recieve input, give output, and answer isnt clear and recognised as not correct. Could it be recognised as network desieas , and if it so could be recognised as consequences influenced from other network or networks outputs

[@abiakhil69]

Consensus mechanism?

[@columbus8myhw]

Link to the Hinton lecture?

[@ArtOfTheProblem]

kzbin.info/www/bejne/sJ2canyQq7xqqKc

[@Trombonauta]

1:13 Cajal is pronounced more closely to /kah'al/ than to that, FYI.

[@KittyBoom360]

This might be more of a tangent to your great video, but my understanding is that intuition and logic aren't really distinct things. The former is merely more hidden in deep webs of logic while the latter is the surface or what is most obvious and intuitive. Ah, see the paradox? It's a false dichotomy resulting from awkward folk terms and their common definitions. I was always like the teacher's pet in college courses of logic and symbolic reasoning while majoring in philosophy maybe partly because anything that was labeled "counter-intuitive" was just something I would never accept until I could make it intuitive for me via study and understanding. But putting me and my possible ego aside, look at the example of a great mathematician such as Ramanujan and how he described his own process of doing math while in dream-like states. His gift of logic was indeed his gift in intuition, or vice versa, depending on your definitions.

[@ArtOfTheProblem]

Yes I had a section in this video i cut which I kinda wish I left it. it was about how intuition is the foundation out of which logic grows. Kids won't learn with "words first" they learn with "sense first" - so mathematicians are of course guided by intuition and then they can later prove things with logic.

[@abiakhil69]

Sir any blockchain related videos in future?

[@ArtOfTheProblem]

have you seen my bitcoin video?

[@abiakhil69]

@@ArtOfTheProblem Yes sir. One of the best video YT.

[@ArtOfTheProblem]

@@abiakhil69 I do plan a follow up video, starting with ETH

[@abiakhil69]

@@ArtOfTheProblem great sir. Another best video coming . Waiting👍.

[@bicates]

Eureka!

[@user-eh9jo9ep5r]

If sensory order was destroyed or noised or anything like this, something like network trafficking , what need to do for to safe all neural network

[@arty4679]

Anyone knows the name of the Borjes story?

[@ArtOfTheProblem]

worth reading: Funes the Memorious

[@Libertariun]

14:45 ... can learn to configure THEIR connections ...

[@fredsmith4134]

it's all a chain of cause and effect from start to finish, each level or layer sharpens and zero's in on the exact match and is refined until a result is locked in, the human brain compares past results to incoming stimuli, and the result is also linked by chains of associations with the result, like result it's a dog, associations : dogs are fury, playful, dangerous, have a master, wage there tail when happy, and so on, but associations are unique to each separate mind ????

[@AceHardy]

👑

[@kmachine5110]

KZbin is a mind reader.

[@fxtech-art8242]

gpt4

[@ArtOfTheProblem]

a lot of progress since this video :)

[@EvenStar303]

After 35 seconds, you are already wrong. We do not think in sentences!!! Thinking is wordless. However, we are translating our thinking into words. But this is not necessary. The point being, is that languageing is only necessary if we want to communicate to another person. But thinking comes first, NOT as a result of sentences. If you get good at meditation and centering yourself, you can drive your car without verbalizing what you are doing. You can make decisions and act them out without verbalization, internally or externally! Language is only a descriptor, not the thinking faculty itself!!!

[@ArtOfTheProblem]

check out the whole series as I built up to this, i agree with you!

[@escapefelicity2913]

Get rid of the background noise

[@escapefelicity2913]

For anything expository, any background sound is unhelpful.

[@vj.joseph]

you are wrong within the first 40 seconds.

[@ArtOfTheProblem]

say more!

[@joe_cock]

good sound design