Universal Approximation Theorem - The Fundamental Building Block of Deep Learning

The Universal Approximation Theorem is the most fundamental theorem in deep learning. It says that any continuous function can be approximated, as closely as we want, by a neural networks of only one hidden layer (this layer may be huge).
In this video, we see a very simple explanation of why the Universal Approximation Theorem works, using an analogy with Lego blocks.
Grokking Machine Learning Book:
www.manning.co...
40% discount promo code: serranoyt

[@MrProgrammer-yr1ed]

What a masterpiece, mind blowing!, I am waiting for video like this long ago.

[@asv5769]

Very interesting points at 11:40 about non polynomial functions. Yes on Uni we always learned that we can use Taylor series to approximate nonpolynomial functions with sum of polynomial functions. How beautiful is that?

[@MrProgrammer-yr1ed]

Hey Luis please keep it up, make more video on why neural networks work.👍

[@KumR]

Always has been looking for an answer to this. Thank you Luis. Lego analogy made it so easy to understand.

[@SerranoAcademy]

Thank you so much @KumR, I'm glad you liked it!

[@neeloor2004able]

Absolutely new information and thanks for explaining it in this simple and detail

[@SerranoAcademy]

@@neeloor2004able thank you! I’m glad you liked it :)

[@dr.mikeybee]

You always do a really fine job of explaining difficult material in an easy to understand way. The universal approximation theorem is absolutely key to understanding why neural networks are not stochastic parrots, therefore, the universal approximation theorem is the key to understanding how neural networks learn. Might I suggest that you follow this up with an episode on holistic training?

[@SerranoAcademy]

Thank you so much! Holistic training, that sounds awesome! I don't know much about it, do you know any good resources?

[@MrProgrammer-yr1ed]

Please make a video on how relu makes patterns in neural networks.

[@AJoe-ze6go]

This sounds functionally identical to a Fourrier series - by adding a sufficient number of the right kinds of simple wave functions, you can approximate any continuous curve.

[@diemilio]

Great video. Thank you!

[@SerranoAcademy]

Thank you so much, I'm glad you liked it!

[@sanjayshekhar7]

Wow! Just wow!!

[@SerranoAcademy]

:) Thank you!

[@chessfighter-r5g]

how it maps output and input never seen before , my answer is it splits as chunks and then it produces according to first chunk and then continuation is second chunk + first output and then third chunk + second output and it splits as chunks by cosine similarity if it becomes to big cuts off so this way chunks happens . what do you think about this

[@Pedritox0953]

Great video! Peace out

[@SerranoAcademy]

Thank you!

[@chessfighter-r5g]

Hi , do you explain what is difference between o1 and normal transformers 4o, and why it waits 30 about seconds ,what it makes in that range of time

[@SerranoAcademy]

That's a great question! Two things that o1 does are RAG, and chain of prompting. RAG means before talking, it searches for the answer, either on Google, or other databases. Chain of prompting means that it first generates an answer, then reads it and elaborates on it by expanding it, and may do this a few times. These methods make it more consistent, and remove hallucination.

[@chessfighter-r5g]

@@SerranoAcademy Thank you so much , how it maps output and input never seen before , my answer is it splits as chunks and then it produces according to first chunk and then continuation is second chunk + first output and then third chunk + second output and it splits as chunks by cosine similarity if it becomes to big cuts off so this way chunks happens . what do you think about this

[@neelkamal3357]

crazy video

[@neelkamal3357]

I love how it's 2024 and we still don't know what " values in neural networks " actually represent

[@SerranoAcademy]

Great point! It can mean a lot of things, it could be the outputs, the weights, etc. I'm probably use that term for several things... :)

[@tomoki-v6o]

what happens in 2D . 7:02

[@dmitrypereverzev9884]

waiting for Kolmogorov-Arnold representation theorem descripton as more strong theoretic basis for KAN

[@SerranoAcademy]

Thanks! Here's a video on the Kolmogorov-Arnold theorem! kzbin.info/www/bejne/pISVmaGjZa-FeM0 (in there, there's a link to one on Kolmogorov-Arnold networks too)