Why Neural Networks can learn (almost) anything

A video about neural networks, how they work, and why they're useful.
My twitter: / max_romana
SOURCES
Neural network playground: playground.tensorflow.org/
Universal Function Approximation:
Proof: / hornik.pdf
Covering ReLUs: proceedings.neurips.cc/paper/...
Covering discontinuous functions: arxiv.org/pdf/2012.03016.pdf
Turing Completeness:
Networks of infinite size are turing complete: Neural Computability I & II (behind a paywall unfourtunately, but is cited in following paper)
RNNs are turing complete: binds.cs.umass.edu/papers/199...
Transformers are turing complete: arxiv.org/abs/2103.05247
More on backpropagation:
• What is backpropagatio...
More on the mandelbrot set:
• The Mandelbrot Set - N...
Additional Sources:
Neat explanation of universal function approximation proof: • The Universal Approxim...
Where I got the hard coded parameters: towardsdatascience.com/can-ne...
Reviewers:
Andrew Carr / andrew_n_carr
Connor Christopherson
TIMESTAMPS
(0:00) Intro
(0:27) Functions
(2:31) Neurons
(4:25) Activation Functions
(6:36) NNs can learn anything
(8:31) NNs can't learn anything
(9:35) ...but they can learn a lot
MUSIC
• It Was Here

[@Rivali0us]

I been in this field for years and this is by far one of the best explanations I seen. Thank you for this.

[@Songfugel]

I would have to agree

[@zh9664]

i think you need join field english

[@ultimateloser3411]

Any advice on how to build a career in the field without a degree?

[@Phoenix_Core]

@@ultimateloser3411 Take clinical psych classes, take classes on quantum physics, take classes on anthropology, take classes on game theory, take classes on python/lisp/lua/C ++/C #, learn unity and ue engines.

[@ayushupadhyay801]

if you had enough data could NN learn anything ?

[@gerixxx1]

This is the absolute best entry level explanation of AI an NN I have yet seen anywhere. This will surely get millions of views!

[@pon1]

Yeah, I think so too, the video was really good!

[@truth.speaker]

@@pon1 I couldn't understand it

[@pon1]

@@truth.speaker What was it that you were having trouble with?

[@SimoneBellomonte]

Pretty close, it's sitting at 962k views right now.

[@NicholasRenotte]

This is such a brilliantly constructed explanation of NNs. Props to you man, this must’ve taken a ton of effort to create.

[@Skynet_the_AI]

I agree. Nicely compiled points and explanation of information gathered.

[@Diamcreeper]

The part where you combined a bunch of line equations was interesting, I've always heard that activation functions are important but I never quite understood what their role is.

[@ScottBrown124]

ReLU specifically constructs a solution as a bunch of piecewise linear functions.

[@Neoplasie1900]

@@ScottBrown124 While this may be technically correct, had I just read your statement this would mean almost nothing for me. And I'm pretty sure this will be true for most people who did not persue an higher education in mathematics or a related field. Visually showing what this means with the inclusion of how things work without ReLU is what does the trick for a non-professional audience.

[@lsacy8347]

I was always told that you need the combination of linearity and non-linearity to gain complexity. But this videos really did it.

[@mertkaaner8622]

They break linearity in efficient ways and there is nothing much to them(lie). Even though this explains what they do “the why” part needs deeper insights.

[@mertkaaner8622]

Without activation(or with linear activation) NN can approximate linear functions with any dimension. However, most of the subjects in real life are represented by non linear functions. So if NN has any chance of learning some meaningful data set it needs non-linearity at some point in its calculations. You can use any non-linear function but most wouldn’t be computationally efficient. ReLu is simple enough and non-linear to be widely used in many NN applications as the activation function. If you wish you can you use sin cos or polynomials or other functions and compare your results and performance with widely used activation functions. Who knows maybe you find the next best activation function :)

[@marlin3576]

Such a good video mate, Hope your channel grows exponentially.

[@NKY5223]

hehe jk

[@ScottBrown124]

I'm glad you brought up the caveats to the universal approximation theorem. Available data, compute, and architecture drastically narrow the scope of problems that we can approximate to any real degree of success. Approximation errors, estimation errors, and optimization errors all compound to drastically limit the expressive power of neural networks.

[@unh0lys0da16]

Tell that to the managers

[@alexCh-ln2gw]

@@unh0lys0da16 naw. elon musk still wants us to believe neural networks will solve driving. They are very far away from collecting the entire universe of driving data to train on.

[@jazzhpatel]

my man altman just walks in with a 7 trillion dollar pitch. hahaha

[@sucmadik123]

As technology advances, AI approaches neuroscience, which is what I find most fascinating about AI. Our brains are just sugar-driven neural networks, tuned to the inputs of everyday life on earth!

[@activision4170]

I hope one day neuroscience is simulated with computer science. A joint of biological systems would be the construction of conscious things.

[@bfi01youtube11]

I’ve been binging your videos all night but this video has easily earned my subscription. An absolutely fantastic explanation that perfectly captures the essence of neural networks without being bogged down by the maths.

[@SO-th8gc]

Just came across your video from the feed. Dude, you've done an incredible job on propping up the basics. Huge appreciation. Waiting for your future videos.

[@Beerbatter1962]

I've recently become interested in neural networks, and AI in general. This video was spot on for providing a fundamental understanding. The piecewise activation function used to trace along the unknown function was eye opening. Really brought that idea home for me finally. Thanks. Your video is very nicely produced. Right at the top by today's YT standards.

[@MoritzWallis]

Wow, this wasn't anything new to me, but you did an amazing job of explaning the background and building intuition. Truly one of the best youtube videos i've seen.

[@Antagon666]

Just found this channel. Quality of your videos is supreme, and you cover really interesting topics, which I've been studying myself for the better part of last year. Thank you!

[@yusufsalk1136]

What a simplicity! Looking forward to watch more explanations.

[@RCFreibott]

What a marvelous video. I've been watching tutorials and reading books on machine learning for over a year, but I've never really understood the core concepts... Until now. Thank you

[@illustriouschin]

I was recommended this video and wow, great explanation. I hope your channel blows up because this is top quality neural network content.

[@derekcobo1453]

This was so well explained. I love how you briefly dipped into the deep math just giving a taste to do our own searching yet still explaining in depth what is going on.

[@clydeb8221]

High audio quality, slow and clear language, great presentation and most of all very informative! I'm glad this was recommended to me

[@cornevanzyl5880]

What you did here is nothing short of magical. Fantastic explanation, awesome visuals 👌

[@breakdancerQ]

The visualizations are brilliant. Easily one of the most clarifying video I've seen about nns, and I've seen many

[@_parallax]

Thank you for this. Your video is, by far, one of the best videos you can find to get an introduction into Neural Networks. Awesome!

[@brian_kirk]

Excellent layout of the topic from basic first principles -- well done! Hoping to find more videos like this on your channel :)

[@miquelmoorrees6010]

a great voice, a great presentation and a great explanation. you have everything to make it as a succesfull channel how this only has 4k views is beyond me edit: also great thumbnail and title

[@naidoeshacks]

Absolutely love these 3blue1brown style videos! Keep it up man your channel is amazing.

[@balaganesh3440]

You got me when you said "A neural network is basically a Function approximator". No one could explain much simpler than this. This is byfar the best explanation of neural networks I have ever come across. Kudos mate!

[@Vaikilli]

This was probably just an aside in the video, but you got me from just knowing the basic shape of the ReLU-Function to why one would use it in a NN really damn quickly.

[@viky293]

One of the best videos that I watched, I knew nothing about neural networks before this video and now I feel like I act as one who understands the damn thing. Thank you for beautiful explaination.

[@parijatbandyopadhyay8057]

This is awesome! Simplest form of explanation ever seen but does not lack any detail. Kudos to you.

[@_.entropy._]

Brilliant job. One of the best videos introducing the basics of neural networks on KZbin.

[@olivergurney9622]

Really interesting and highquality video. Keep it up!

[@orchidchetiaphukan4658]

It has been years in this field.....One of the best explanations I have seen till now....

[@Rikimkigsck]

This channel is a treasure. A person who knows how to explain things with great knowledge.

[@mxlinuxpl]

Thank you, simple, clear, concise, perfect explanation!

[@Los__Merengues]

The best explanation I've seen

[@zeb4827]

I think this is one of the most valuable videos i've watched about understanding what neural networks are actually doing. Neural networks are kind of built upon the union of two of the most fundamental ML algorithms- the linear regression draws the main function, and logistic regression bends the function. More layers, more bends, more nuance in the description of relationships- all now automatable through this recursive optimization process. It's quite beautiful really

[@TheFinav]

What an absolutely wonderful, clear explanation. Thanks!

[@charliegil2007]

Your explanation is brilliant in so many different levels. Thanks for this.

[@alonsomartinez9588]

Max, any chance you could do one on Transformer architecture? It’s the core of most modern machine learning and I haven’t seen an intuitive explanation. It’s increasingly that it has remained mostly unchanged and undefeated. It would be incredible to hear your breakdown on it!

[@Dante02d12]

This is the introduction to neural networks that I was ever hoping for. I'm an engineer, but sadly, neural networks weren't part of my learning program. But I am interested in learning, and this explanation is a perfect description to the concept of NNs, it really clears my mind about what this is. So a neural network is a function approximator, and due to the way it is "built", it can approximate _any_ existing function, no matter how complex, as long as we have enough processing power (and memory). The diagram with the function where we only see a few dots is a great view of what neural networks do. They have a set of data (a bunch of pictures, for example), and when you ask for a result that isn't in its set, it extrapolates it based on what it does know. EDIT : I was wondering how a neural network could, for example, study a set of images and return a brand new pic corresponding to its database. I have realized that an image is just a sum of pixels, and a pixel is just a sum of several colors (the famous RGB, with an alpha channel for transparency). So the "function of an image" looks like this : (Pixel 1, Color 1) + (Pixel 1, Color 2) + (Pixel 1, Color 3) + (Pixel 1, Alpha) + (Pixel 2, Color 1) + (Pixel 2, Color 2) + (Pixel 2, Color 3) + (Pixel 2, Alpha) + ................. So a neural network can be trained to recognize the "sum of pixels and colors" of images, and then return a similar result! And, of course, the various layers help recognize patterns, like a character in a pic, or the hair, or the hands, and so on...

[@AhmedSam]

The exact info i was looking for finally straightforward! Thank you!

[@mboosiri]

Watching this video will clear all the doubts! Some other videos may try to explain the concept visually and with analogies, but that can often lead to over-fitting, i.e. after watching those videos, will only remember the analogies and not the actual concept. This one nails it!

[@jonathanherrera9956]

Sometimes you may want to use a simpler approximation with a neural network instead of the complete known function, since the function is too complex and computational intensive. Like in the case of acoustic simulations.

[@JordanMetroidManiac]

Fluid dynamics is another example. Mandelbrot set rendering. Cayley graph (in group theory) diameter calculation and pathfinding (optimally solving a Rubik’s cube, for instance, can take several minutes). Markov decision processes. There are so many things that neural networks can help us to solve or understand better.

[@betterbelle29]

This is actually a great point as well. There's a small set of research that has recently delved into using neural networks to approximate the solutions for NP-Hard problems. That's probably one of the most intuitive ones to think about; NP-Hard problems are quite literally defined as being complex and computationally intensive :P

[@HansLemurson]

Great video! I liked how you made sure to talk about the major limitations of Neural Networks. The main thing holding NNs back is how long it takes to train, and how inefficient their computation is. But the thing is, that just turns it into a problem that can be solved by scaling up hardware, which is not something that you can do with genius programmers. With machine learning, you can potentially solve any problem by just throwing money at it.

[@setoelkahfi]

This is by far the best explanation of the subject. I remember trying so hard to grasp the concept at the university. Thanks for putting this one out.

[@stas4112]

Wow what a channel, just checked the rest of the videos and got excited for the upcoming weekend to sit down and take some notes on all your videos! Get ready for a lot of growth, looks like the algorithm picked you up!

[@spicetea4060]

I'm gonna use fuzzy logic

[@aspectreishauntingeurope]

great video! looks a lot like 3blue1brown :D

[@yuyong4569]

This channel deserves more viewers. Thx for the clear explanations.

[@qbitsday3438]

Beautifully Explained !

[@yashtrivedi2717]

We need more videos like this. Most of the YT science videos are so in Layman's terms. This is more like learning real scientific explanation 🧬 🧠

[@ingolf17]

Its not learning, its conforming to the result you want. Billions of small dials, controlled by a logic algorithm, Its no magic, just alot of numbers!

[@Imbalanxd]

Sounds the same as billions of small neurons, controlled by a genetic algorithm. Sounds like learning to me.

[@abdelsamie.bousba]

WOW !!, truly amazing, I got to understand what I couldn't in many years of reading papers and watching tutorials, big thanks to you.

[@raphaelcrespo3725]

this is one of the best and clear explanations. thank you

[@mastershooter64]

can they learn how to train themselves and to modify their own code to make themselves better?

[@hyperteleXii]

Sure, to a degree. Software can't fully optimize itself because a computer is a living ecosystem with dynamic properties. One moment you want to optimize for something, the next for something else. As learning takes time, it will always take *more* time to continously optimize than to brute force. That is to say, LEARNING only accelerates REPETITIVE PROBLEMS. "Become better all the time" sounds repetitive but isn't a well-defined *problem*. So there's a practical limit to self-optimization usefulness.

[@dannyarcher6370]

Fun fact: You can't learn everything.

[@PunmasterSTP]

Your videos are incredible; please keep making them!

[@terryn9450]

Wow!!!! Out of all the videos I watched on deep learning, this video simplified it the best. Amazing work on making this idea easy to understand.!

[@alessandrorossi1294]

Can't learn an XOR 😂😂😂

[@dagmarski4133]

That’s not true, it’s even feasible to manually construct a neural network with just 3 nodes for such a simple task.

[@fatemehcheginisalzmann2189]

Please continue making more videos; you have an excellent talent for creating graphics that make the topic understandable.

[@unpatientes1562]

This is an awesome video, congrats!

[@asier6734]

Best explanation ever, visuals really helped !

[@xxsaivior9639]

Dude i learned so much from this video I was actually shocked like I’ve watched 30 min videos and was still left with so many questions but this one’s amazing ❤

[@willykitheka7618]

Truly one of the BEST VIDEOS I've watched on Neural Networks....so very well explained!

[@tnegras99]

This is amazing, best explanation of neural Networks I've seen, saved it to watch it again and again, might need it

[@anontasofen60]

Very well put together video. Loved it!

[@saikrishnasyamala5720]

That’s a great way to explain a complex concept👏🏼

[@TheNewton]

Subbed. That thumb of the tensorflow playground spiral is why I watched I've only dabbled in other tools or unfinished projects so I've never gotten the thing to cooperate and forgot all about it.

[@Alexander-ms2ct]

Your editing skills and descriptive ability gained you a subscriber. Very good

[@rahulrustagi7029]

Watched many tutorials on NN but this is far better than anyone in terms of content and explanation. Great Work !!

[@lookingaheadoftime]

Thank you for uploading this video! I was taught neural network and this video helps me to clear my doubts and help to understand this topic better. Indeed this is one of the best explanation.

[@zerotwo7319]

I knew it would be turing complete. Thanks immensely for this

[@franklinfeng176]

such a simple and elegant explanation!

[@zahid1909]

I believe, now I got some fundamental ideas how and why the neural network works. Thanks a lot! A great video.

[@mayankkhanna3660]

This is the best it can get, and you did it for us, thank you!

[@pkrozie]

Amazing video, such a intuitive way to explain NNs.

[@lumix854]

Thank you for making this video, very informative and visually engaging :)

[@rgoudie]

This is a great presentation. Thank you.

[@hsslay3237]

Didn’t have any prior knowledge on neural networks and it’s working but a high level overview like this makes it more sensible on how these neural networks works without involving too much background information in it just weights and bias with pre determined input and output to get the function that describes them by adding more and more neurons if that’s what needed for the approximation.This is fascinating id love to go deeper where veretasium talked about this concept but in a different approach.really fascinating stuff

[@antoniomachado3083]

This video is amazingly well done and clear.

[@xhinker]

The relu part is amazing, renewed my understanding of it, thanks!

[@HenryChidgeyLeadershipMethod]

Thanks so much for creating this video! Absolutely awesome!

[@skynet_cyberdyne_systems]

Well explained ! Good work.

[@qwerasdliop2810]

I didn't come here expecting to learn anything but this video was incredible! I learnt: Neural networks are universal approximators, turing complete, and how multiple layers actually work!

[@grkuntzmd]

I started writing neural networks for a research project when I was in medical school in the early 1990's. At the time, I struggled with explaining how they worked to my audience of medical professionals. I wish I had had this video to show them. It would have made things much easier. Bravo!

[@0xAV]

most easy to grasp explanation i have come across till now sir. Thank You 🙏

[@RRP3168]

What a great video! NNs so elegantly explained! I love it! Thanks.

[@notallm]

The explanation and video quality is crazy good. Your voice is so nice too!

[@user-rb2od5bj2w]

Such a good video mate, Hope your channel grows exponentially.. Such a good video mate, Hope your channel grows exponentially..

[@AA-gl1dr]

Brilliant in every way. Thank you so much

[@SeaWaves8]

amazing, thank you so much for making this.

[@scottterry2606]

From one educator to another, A+++ job!

[@HeisenbergHK]

I absolutely loved this video, wish we had more content like this on KZbin

[@rishabhjain2532]

That's a great explanation

[@wiktorwysocki516]

Great explanation, much better than on the uni and even after studying this concept for some time it brings some clarification. Thank you!

[@jordiwang]

Man for real, the visual graphical representation of the piecewise functions sum left me speechless, so freaking good. For someone who has a bad foundation at math, that representation was just everything i need to understand perhaps one of the most pivotal foundation of complex neural network. I don't consider my self dumb, but I've watched several videos that all it does is to explain the concept from an abstract perspective, or if not, making assumptions that we already know some mathematical concepts. With that visual representation, I can finally tell me self that i have understand how really the inner world, the magic box works. Thank you

[@user-rn5fh3jr5p]

Top explanation. By far the best I have ever seen🎉

[@HypnosisBear]

Amazing explanation! Thanks 👍 Neural networks are indeed incredible!!

[@aprocesss]

The visuals you presented are the most eye oppening ive seen in all ML courses ive taken.

[@icecream6256]

Wait dude the first 2 min is already great, thank you so much !!! I cant wait to watch you again once i finish more courses on ml

[@AAlmunia]

Really well explained, thank you.

[@jamescarter9147]

One of THE best explanations of neural networks and how they actually work.