Why Neural Networks Can Learn Any Function

[@datamlistic]

Neural nets can learn any function, but they are also prone to overfitting the data. If you are interested in finding out why models overfit and underfit, make sure to check out this video: kzbin.info/www/bejne/a57FiWl_id-hfs0

[@SirGisebert]

Clear, fluid, and approachable - good job!

[@datamlistic]

Thank you!!

[@simonetruglia]

This is a very good video mate. Thanks for it

[@datamlistic]

Thanks! Happy to hear that you liked it! :)

[@kraigochieng6395]

thanks. you explained all the parameters well. the animations for how the weights and biases affect the final output really helped. thanks again.

[@datamlistic]

Thanks for the feedback! Happy to hear that you liked the explanation! :)

[@amatinqureshi4917]

I was looking for this type of explanation. Thank you !!!

[@datamlistic]

Glad it was helpful! :)

[@exxzxxe]

Very well done, sir!

[@datamlistic]

Thanks! Glad you liked it! :)

[@Nzargnalphabet]

I would suggest x/|x| as a step function that works on discontinuous functions, but it is discontinuous, and actually not defined at zero, it’s better to simply go into the base code and access the sign bit of the number, although it really isn’t very generalizable at all

[@datamlistic]

Never heard of x/|x| to be used as a step function. Have you encountered it anywhere implemented in a neural network?

[@Nzargnalphabet]

@@datamlistic no, it’s more of just a thing I’ve been expirimenting with, I was mostly trying to find a better way to approximate the floor function, but it still could have a use here

[@AfreediZ]

Great, please keep going

[@datamlistic]

Will do, thanks!🙏

[@florin-andreirusu6424]

Nicely explained! Thanks!

[@datamlistic]

Many thanks!!

[@neolinksrv]

2:00 What is the operation inside the second neuron?

[@datamlistic]

That's the sigmoid function. :)

[@uc5331]

This was super helpful for me!

[@datamlistic]

Thank you! I am glad it helped you!

[@Nzargnalphabet]

If you could detect a discontinuity and its position, which I know to be possible using limits, you could use such a function in that case

[@iacobsorina6924]

Greart video! Thanks!

[@datamlistic]

Thank you!!!

[@JanKowalski-dm5vr]

Nice video, this is exactly what I was wondering about. So adding one node in a layer gives the function 2 new inflection points. But what happens to the function when we add another layer with one node?

[@datamlistic]

Thanks for this! Hopefully I am not wrong in what I am about to say, but you get 2^(n+1) inflection points if you have n layers. However, the produced regions are symmetrical. Take a look at my 'why we build deep neural networks' video, if you wanna learn more about this. :)

[@kadirbasol82]

How can we fit infinite sine wave or other infinite series ? Every input automatically mapped to modded output or any better solution for infinite series ?

[@datamlistic]

That's a very interesting question! Well, infinite problems require infinite solution? You could theoretically fit an infinite sine wave or other infinite series using the logic explained in this video if you had an infinite number of neurons. However, I think you may need another type of neural networks if you wanted to fit such functions with a finite number of neurons (periodical neural nets?). Unfortunately, I am not aware of such networks, but I can search a little bit in the literature for them if you are still interested.

[@kadirbasol82]

​@@datamlistic Yeah , we are interested in this question.Yes we can make module for sine wave for input.But what if we dont know its infinite series that all the time doing same operation.

[@datamlistic]

Hmm, to the best of my knowledge, and please correct me if I am wrong, then it's impossible to approximate such functions for an infinte number of points, without prior knowledge about the fact that the data is periodical or without an infinite number of neurons. Even more, by using the construction logic in this video, you can't approximate perfectly a function within a certain range without having an infinite number of neurons.

[@florinrusu3409]

Awesome!

[@datamlistic]

Thanks mate!

[@ikartikthakur]

then does that mean if a function needs 30 steps to approximate.. you'll need 30 hidden neurons is that's the analogy.

[@datamlistic]

yeah... but it's easier to visualize it this way

[@ikartikthakur]

@@datamlistic Yes ..I really liked this animation. Thank a lot for sharing..

[@SuperMaDBrothers]

But this isn't how actual NNs learn functions at all! You could just use intuition to reason this, it's pretty obvious!

[@datamlistic]

Thank you for your feedback! This a correct observation. That's indeed not how NNs learn and are used in practice. However, the following is just a theoretical proof which shows that through the intelligent manipulation of NNs weights you can approximate any continuous function. It's a nice thing to know and be aware of when working with NNs, because not all statistical models have this property.