Why ReLU Is Better Than Other Activation Functions | Tanh Saturating Gradients

[@datamlistic]

ReLU is cool, but its variants are cooler: kzbin.info/www/bejne/iZSqnqV8d9KijKs

[@omaralkhasawneh1968]

great video , thanks

[@datamlistic]

Glad you enjoyed it! :)

[@iacobsorina6924]

Great video! keep going!

[@datamlistic]

Thank you!!!

[@davidryanguasrojas2241]

The content is pretty good, dedicate some time for editing and you may get a very good channel in the area soon enough 👍

[@datamlistic]

Thank you so much for your feedback and your kind words! I have dedicated more time to editing in my last two videos and I really hope they are better. Also, if I am not asking for too much, could you tell me what parts more exactly you think would need more editing in this video? Maybe I missed some points.

[@davidryanguasrojas2241]

@@datamlistic I would recommend to focus on two issues: - Try to explain with more confidence and in a more fluid way. During several moments in the video you slow down or introduce filler words in the explanation (2:35 for instance) which reduces the quality of the result. Taking different small cuts and choosing the best would lead to a way better one. - Try to improve the visuals, use different colors, improve the caligrafy (or switch to digital fonts) and/or introduce animations would lead to a better final product. The topics you are addressing are very interesting and I wish you success with this project. I would recommend you to take as reference channels as 3blue1brown which adresses math topics in a very clear and elegant way. Try improving step by step and you may get something very interesting.

[@datamlistic]

@@davidryanguasrojas2241 Thank you so much for this detailed feedback! I have started to remove the filler words and pauses in my videos, while trying to sound a little bit more confident. Regarding the visuals, this is a bit harder to do, as I don't know exactly which tools are better and it's quite an experimental process. As you said, I am trying to improve step by step and hopefully the quality of the videos will also increase.

[@w3w3w3]

best channels/videos have the least views/subs i find lol

[@tomoki-v6o]

Also adam optimizer works faster with relu

[@datamlistic]

Thanks for bringing this up! I am not fully aware of why the Adam optimizer works faster with ReLU. Could you point me to some references?

[@tomoki-v6o]

@@datamlistic through experimentation only. i dont really why.

[@datamlistic]

@@tomoki-v6o Oh, ok. Maybe there are also other people that have observed that, I will take a deeper look into the problem. This might become a subject for a future video. :)