Exactly! Are you studying math?

@MathForLife no, third second time you ask me that... xD

@Albericbaboin LOL XD by interacting with the viewers, I have learned so much in the recent weeks, so that’s why I am asking LOL I will post the video later how ReLU performs on this task)

Yeah, i understand 😅​@@MathForLife arr you programming, and on what language?

Omg, thank you so much for such an awesome comment:) it made my day! Yeah, I like to visualise things a lot. That’s how I think in my head, in pictures. Are you doing math?

Let’s goo Applied Math! What year are you? How do you like it so far?:)

@@MathForLife I'm a third year! I absolutely love it, even though sleep and I have never been friends lol.

Ahah, that’s fair! I will become soon:) What’s your favourite math subject so far?

Hello! Are you studying math?:)

Ahahah, it will appear soon :D

Sure, I will record more videos with my experience going through graduate school and qual exams:) are you doing math PhD?

I am a masters student going to convert to PhD in August 25. I am planning to take qualifying exam in Measure Theory, and Matrix Computation. I am really struggling to create a plan to pass Analysis in first attempt. I am just doing textbook problems but not sure if I am headed in correct direction. Since the professor who taught the course is not setting the test, it is difficult. Any help will be appreciated.

What textbook did you use for your class? Are there any practice quals?

@ we used my professors own notes. That is the problem. In his notes most things are left as an exercise.

Oh, that’s unfortunate. What about talking to older students? Do they have some old exams?

What? How? What comments did you write?

@MathForLife You were asking me about the gaussian regression, but when I was replying about the libraries it deleted the comment. I even didn't publish the links or anything like that

Oh weird, can you please send me the email about it to my [email protected]? I am really interested in that. Thank you!

Ahah, that’s true xD are you studying math?

Har?

Thank you! Yes, I will do different experiments and increase the complexity

What do you mean by a serious of larger networks?

@@MathForLifeLike keeping the height fixed and increasing depth of the network. Comparing performance across different depths. Another parameter for sensitivity analysis might be the learning rate. Because this one seems quite jumpy. All the best with your experiments.

Also, do you think you could come up with a better cost function given that you know exactly what the model you're trying to fit is? I know that's cheating but it could be useful when someone has some idea about their model. Which is usually the case in engineering.

What do you mean by a better cost function? I am using mean square or entropy loss as I do standard examples without anything fancy.

In the perfect world?:)

Do you mean learning rate smaller or bigger? Do you do machine learning?

@@MathForLife nope but increasing the learning rate helps. Not too low and not too high.

I will try! Thanks:)

The function is discontinuous:( it’s the bad boy

@@MathForLife I don't think the typical regression to the mean + neural net is the way to go here, as this will give you a functional result, which is what sin(1/x) already gives you. Of course sin(1/x) is just chosen as a challenge function with asymptote and infinitesimal values in it. You can achieve this with a LM, as these concept are more verbal than numerical to us anyways, and I don't know a current day application that would require approximating those types of functions in a numerical way, as most control systems would fly off the handle away with asymptotes and infinitesimals.

Thank you for the feedback. It sounds really interesting. What’s a LM? Yes, you are correct. I just choose different challenging functions and experimenting with neural networks approximating them to build an intuition for such creatures. It sounds like that you do engineering, correct? What type of functions do people care about in control theory, or ODEs/PDEs? Thanks!

Thank you!! Are you studying math?

@MathForLife i love maths, but i ́m only one the 11th grade...

So, it’s the right time to start to dive into this beautiful world!:) what math class are you currently taking?

Ahah, exactly! I also use so much linear algebra in my research, so we joke sometimes that everything is linear algebra in the end of the day. What are your favourite math classes so far?

Yes! Yes! Exactly) do you like analysis?

@@MathForLife Yeah ^^

Thank you for watching! Are you taking analysis class?

I am a graduate engineer and later studied mathematics (but did not graduate)... almost 40 years ago. Today I continue to study mathematics for pleasure and personal enjoyment, and youtube channels like yours are really helpful. Thanks again.

That’s so awesome! What type of engineering did you do? Do you read some math books? What’s your favourite one?

Civil Engineering. I have quite a few math books, both in paper and electronic format, and I don't have any strict favorites, although the "modern" ones are much more graphically attractive than the classic ones. I'm currently reviewing Measure Theory and that's why I accidentally came across your channel: some math videos you can find on KZbin are really fantastic, both didactically and, very importantly, graphically.

That’s really interesting. Measure theory is a fascinating subject. I am thinking to film more topology videos as I see a lot of demand for them

When the function is continuous, we always can do that! But this one is discontinuous at x = 0. Can we upgrade this function and make a continuous one?:)

​@@MathForLifehow about splitting the x values into 2 parts so that they are all continuous? The other idea is to use some transformation to move x out of denominator, but my calculus is dusty

It’s a good idea to split into two parts. I will take one of that part and try to approximate there! Thank you for the advice! Are you doing math in college?

@@MathForLife I've done some math in college, but not too deep. I've worked with neural networks a bit, and there are a lot of architectures with 2 output heads, where you share hidden layers but predict 2 different things, like centroids and borders in the CV problems. My knowledge is very superficial, but maybe splitting the output heads is the step in the right direction

@@MathForLife but given that the function is symmetrical, you might not need a second head at all, just one, and then mirror

seriously?! that's so awesome! Are you doing math?

@@Albericbaboin I started to explore and visualise feedforward neural networks with one layer. This particular one has a rational activation function and 50 hidden neurons. Later, I want to explore deep architectures and consider different type of neural networks:) thank you!!! If you have some ideas or suggestions, let me know

​@@MathForLife50 is impressive! Yeah, NN are interesting, i'm actually learning the theory of transformers...

Oh, cool. What are you currently learning in particular? Have you trained any of them?

@@MathForLife in fact, i ́m learning to implement it from scratch first, and after, i ́ll train it

Awesome! That’s pretty cool. What materials do you use to learn it? It would be fun to train it on arithmetic to predict sequences for example :-)

Thank you for the support!:) do you remember through what video you discovered my channel?:D

@MathForLife I don't remember! Sorry! 😬

Aha, no worries! I just wondered:)

Oh no, sorry about that:( I will find my mic and improve my audio

Thank you @powersetofx! Are you a math student?:)

​@@MathForLifeaspiring*

Of course! Each red line is a level curve of a function that splits the data into clusters (red and blue). The numbers on the level curves are the values of the function. The values closer to 0 or to 1 means that’s the point that belong to that region to be in one or in another cluster. The entire video is the visualisation how the neural network learns to distinguish different clusters.

Yes! Imagine how it changes if we go one dimension higher xD

What I have learned that linear algebra is the most useful subject, everything literally boils down to it in the end of the day

An apple?! 🍎 or a sun function?

Ahah, bro is chilling xD

Ahah, python the one What is a programming language of your choice?

probably rust

I have the architecture (1,100,1) One dimensional output and input with 100-dimensional hidden layer I train on around 160 points sampled from the interval [-30,30] I did 5 runs for each activation function and picked the best run for the visualisation

Yes! It’s interesting to visualise and compare the learning dynamic of the neural network with different activation functions as well

Same style brother xD what project are you working on?

​@@MathForLife Nothing of note. Playing around with some first order wave equations in both time and space that satisfy the principles of Relativity.

Do you use mathlab? As I understand, you solve some partial differential equations numerically, how do you construct your solution?

Thank you for watching! Yes, start with small and something that you understand and climb up!:)

I don’t remember to be honest xD

@MathForLife if you remember we'd all love to have it. Cheers!

@NYlivinginTN will do! Do you teach math?

@@MathForLife yep, high school and college math.

@NYlivinginTN that’s awesome! Do you recommend any book to read about math teaching?

Thank you!:)

Are you studying math?

Thanks for watching!

Thank you for watching!

​@@MathForLife btw, are you planning on making a video about complex projective line? It would be very cool, I still don't understand homogeneous coordinates in complex numbers - isn't like 4d or smth? Thanks in advance

I actually might! I am thinking to make more topology videos and some stuff on algebraic geometry

Do you study mathematics?

@@MathForLife Well, then I can't wait for the complex projective space video! :) I don't really study mathematics (if you mean in university), I'm just a math olympiad kid(currently in 9th grade) and I learn projective geometry primarily because of the method of moving points, which is one of the most wonderful methods of solving geometry ever. I can advise you some articles on this topic if you want

Happy to help!!

Ahah, thanks:)

Excelente!

Thank you for watching!

Ahah, thanks! I miss that office:)

Yes, the best book for Calculus!

Jupyter notebooks, python 🐍