Deep Learning-Activation Functions-Elu, PRelu,Softmax,Swish And Softplus

[@sukumarroychowdhury4122]

Dear Krish: we all love you, your energy, your enthusiasm. One point about derivative of ReLU activation Function at zero. To express it properly, derivative of ReLU as x tends to zero does not exist because the derivative is a step function and at zero, it is discontinuous. And limit approaching from left is not equal to limit approaching from right. The ReLU function is continuous, not-bounded and not zero-centered. At x = 0, the left hand derivative of ReLU is zero while right hand derivative is 1. Since the left hand derivative and the right hand derivative are not equal at x =0, ReLU function is not differentiable at x = 0. Derivative of Leaky ReLU at zero is still discontinuous. Hence, it is still not differentiable at zero. Generally, in Geophysics, we use Leaky ReLU as follows: f(x) = 0.1*x if x0 Else, I am your Bhakt and I want to start your (iNeuron) - 3 courses as soon as I reach India. Master Machine Learning Masters Deep Learning Masters NLP. Congratulations for your new position as CTO at iNeuron. Cheers, Roy

[@ratulghosh3849]

Thank you sir you really made the concepts related to different activation functions so clear.

[@satwindersingh1797]

U r ultra best teacher Sir .. Ultimate and even better...

[@Sudeepdas313]

Thank you so much sir for taking out time and effort to put out such great content!

[@rahuldey6369]

33:58 just a question- Do we have mutually exclusive results when apply sigmoid like softmax? Because the 60% 40% scenario will be there when we have mutually exclusive results where total probability sums to zero as you've mentioned

[@raghav_birla]

Bro he actually said wrong in the video, sigmoid function doesn't sum up to 1 and also binary classification is that in which we only have one output node in the output layer, not 2 that he described

[@raghav_birla]

And that's the only reason why we can allow sigmoid to be used in the output layer, because it calculates or say squashes the x between 0 and 1 independent of other x values that we have in output layers

[@nguyenngocly1484]

f(x)=x is connect. f(x)=0 is disconnect. ReLU is then a switch. A ReLU neural net is a switched system of dot products. Fast transforms like the FFT and fast Walsh Hadamard transform are fixed systems of dot products that you are free to mix in.

[@pritamH]

Make theoretical video on white board..because most of the people familiar with that..Than you krish..big fan of you🖤🖤

[@hariharans9408]

Such a Amazing sir, Please put a link for this notebook in the discription that will help us to revise more about this

[@arpanghosh3801]

can you share the github link for the code

[@DeepakSaini-sg3pq]

Thank you sir for making this video really very helpful and sir can you please provide us this notebook.

[@kr2ik]

@Alden Lewis 100% working

[@anandhiselvi3174]

Please do video on svm kernels

[@rohanyewale3184]

Really very Helpful !!! Can I get this notebook?

[@the-ghost-in-the-machine1108]

27:45 No, the Swish activation function is not zero-centered

[@kavuluridattasriharsha]

Hi Krish Can you please provide the Activation functions notebook for our reference.

[@imranriaz9752]

I am very thankful to you dear sir for uploading such nice and well explained videos. Engr. Imran Riaz Lecturer Department of Electrical Engineering MUST Mirpur A.K. Pakistan

[@chitramethwani4758]

hi Krish.. can you pls provide the link for this notebook? Great content and nice explanation.. :)

[@randhirpratapsingh9795]

Hi Krish.... Well explained... Could you please help me with the Jupyter notebook for this activation functions script....

[@hashimhafeez21]

thank you brother for such a nice explanation.

[@VisheshPanchal-c6j]

Dear Krish: Can you create playlist or course for pytorch and tesnrflow and jax combine. In that you will overt all concept like in D2l book.

[@arijitmukherjee8293]

Hi Krish, thanks for the lovely explanation. I have one question. Why does zero centered data converge faster. Can anyone explain this?

[@techsavy5669]

How can i get the ipynb file that you described here ! Thank you.

[@abhinaykumar8875]

Very informative video sir Please can you share the link to notebook

[@dr.ratnapatil9272]

Wonderful session.

[@anjubhagat8257]

Sir, you have explained very well. Thank you. I have the data from the year 1987-88 to 2021-22 of mango production and I want to apply ANN. so please give me code for Rstudio and also tell me which activation function, learning rate and how many hidden layers are best for the problem.

[@rafibasha1840]

@15:00 when we multiply negative value with .01 how it became positive value

[@shreesapkota]

Good explanation. Thank you.

[@dr.pushpalathamanagement1276]

very clearly explained .

[@vipnirala]

Great content. Thank you.

[@shubhibansal7016]

Hi, where can we find jupyter notebooks or the notes for the videos?

[@Dragon48OO]

Naik Sir can you make fully explained vedio on YOLO algorithm with its working program

[@nagamanid8926]

Thank you Sir. My question is-Can we use both SVM and Softmax simultaneously in CNN for classification

[@meedoremee9622]

Can you download your machine learning that you used in your teaching?

[@rafibasha1840]

Hi Krish,please share the notebook you are using

[@saurabhtripathi62]

thanks for updating.

[@borgavejs4379]

Thanks Krish, Can you please share this document ?

[@koustavdutta5317]

@Krish Naik, sir please kindly share the notebook. Its very much required for self revision and from notes point of view.

[@arjyabasu1311]

Sir please upload the notebook !!

[@AbhishekMishra-nl6by]

from which repository I can find this file in your GITHUB

[@balasaraswathiyugandher3176]

16:35 ELU

[@hareeshr3979]

Saved my time thanks

[@bcinerd]

could you please upload this notebook in video description?

[@devmaharaj1]

Super , Liked a LOTTTTTTTTTT !!!!!!!!!!!

[@MrKB_SSJ2]

Thanks a lot 😊

[@vimalgupta86]

Hi Krish, can you please provide the link for this notebook?

[@anandvamsi1993]

Hi Krish, Why can't we use mod x (|x|) as an activation function? It will ensure neurons are not completely deactivated + take care of the vanishing gradient issue.

[@raghav_birla]

It's a good question, and in my opinion the reason is that because the slope becomes 0 at the point of convergence i.e at 0, or anywhere on x axis if the activation function is shifted right or left with the help of bias.

[@classictremonti7997]

Hello...When you say the model has "2" or "3" output layers...are you really saying that the "output layer has 2 or 3 neurons"? Just trying to keep the semantics clear in my mind. Thank you!

[@vishaljhaveri7565]

Sir please share the jupyter notebook with all of us. Thank you so much. I hope you reply us with the notebook.

[@saurabhtiwari4989]

Did you get ?

[@nazishiqbal1046]

Great work! From where I can get this notebok?

[@dineshjayakumar6349]

sir , can you please share this activation funtion jupyter notebook ?

[@priyankachore1617]

Thank you sir!!!

[@shynie4986]

I was wondering how to access the notes in your video.

[@smarttaurian30]

If ReLU has zero or one output then why we don't value step function?

[@chinmaybhat9636]

@Krish Naik Sir Can you share the Jupyter Notebook for this in the github ???

[@himanshugoel4573]

Hello Krish I am trying to follow your complete Deep Learning playlist but can you share a link to the Jupyter notebook or the documentation that you used in the video to explain everything? It would be a great help to read those notes showin in the vieo. Thanks in advance !

[@sachinborgave8094]

did u got this notebook?

[@himanshugoel4573]

@@sachinborgave8094 No, I am still waiting for the response

[@louerleseigneur4532]

Thanks krish

[@heecmat4045]

When sigmoid is used only output layer then why you used it in hidden layer in before videos sir?

[@saruaralam2723]

Nice video, Krish, btw where is the video of loss functions, couldn't find it

[@NikhilaRaoLakku]

Hello Sir, Can you please share the notes that you are explaining from?

[@gurucharank5491]

Very nice video sir kindly share the notebook

[@guddubhagat7854]

Link to this notebook file please ?

[@muhammedcansoy1434]

Thank you so much

[@akashm1027]

Is there a notebook link to refer?

[@PidathalaSoujanya]

hello sir,i joined as a member how can I get the material

[@deepaklonare9497]

Can you please share the Github link for this document?

[@rimanshumangal3517]

HI krish, could you please provide link for github where this notebook uploaded and others also.

[@suneel8480]

Can you name the tool with which you are writing on the screen?

[@Morais115]

epicpen

[@МаксимВасильків-к3о]

thx Krish! can I have access to your notebook?

[@vivekkumarshaw6495]

Where can i get these slides

[@p_saini]

can someone provide me link of this notebook ?

[@datasciencewallah9620]

I don't understand why you are saying we are not trying to find derivative of 0. Why can't we find derivative of zero in ELu, ReLu, Leaky ReLu function

[@KK-rh6cd]

sir, please share this notebook

[@mdmynuddin1888]

Can i get the notebook?

[@jokerislive1631]

bro can you share this jupyter notebook

[@lamis_18]

Can we have this amazing ipynp file?????????????????????????????????????

[@nazishiqbal1046]

had you get this file?

[@AbcAbc-kx3xm]

Great!

[@smarttaurian30]

You should have made single video for each activation function as it is difficult to get from this video. You are going fast and bit confused in understanding

[@RehmanKhan-gb6sk]

can you share this file please.

[@laxmankusuma8994]

playback speed 1.5

[@gamingultimate7489]

Fooling layer in nlp

[@ramakrishnayellela7455]

Sir,Can you send notebook pdf

[@shaikrahamatulla-jy2cg]

please provide jupyter notebook

[@ameenali1837]

Sir ko nahi pta h kya ki derivative of 0 is 0 itself?

[@KrishnaMishra-fl6pu]

Are 0 jab aa jayega toh dot product karte waqt weight update kaise hoga...dead activation ho jayega na

[@ganjirao7094]

Good

[@pratibhasawant9349]

Hi Krish Naik, Your videos are as usual excellent. Could you help me to know what kind of activation function is this? ψ (x) = ( 1 0 ≤ x −1 x < 0 the operator is chosen as ψt (x) = −1 + r (x + t) /|t| − r (x − t)/ |t|

[@nikextracool]

signam, but here the values will have zero impact on the weights as the derivative is 0 on all the regions except 0 (non-defferntiable.)

[@ArunKumar-sg6jf]

Bro unavailable join u telegram group

[@rageadigaming8162]

share the notes sir >>>>>>

[@iskrabesamrtna]

every sentence you say two times, good lecture but so much repeating is annoying.

[@laykefindley6604]

Holy moly, you jump around more than a locust after snorting the entire amount of cocaine Charlie Sheen snorted before each episode of Two and a Half Men.

[@sachinborgave8094]

Thanks Krish, Can you please share this document ?