Custom Activation and Loss Functions in Keras and TensorFlow with Automatic Differentiation

[@mockingbird3809]

Great Video on Auto Grads, Amazing as Always, Loved it Dr.Jeff

[@NisseOhlsen]

Small correction @1:36 : You don't "take the partial derivative of each weight". You do take the partial derivative of the loss function with respect to each weight. Also @7:24, the derivative of x^2 is 2x, not x. Also, @7:46, that IS the definition of ANALYTIC derivation. It is also used in the discrete case, the difference being that the jumps are finite, not infinitesimal.

[@ChandraShekhar-rn9ty]

Hi Jeff. Thank you so much. I spent couple of hours figuring out how the hell Keras is managing any changes in custom loss so easily. I was worried if it is even checking if the function is differentiable. With this video, things are pretty clear now.

[@tanyajain3461]

does gradient tape break when math operations are applied on custom indexes of input_tensor? also while stacking tensors and then using in our loss function? Please suggest a workaround, I've been trying to implement it but it returns all gradients as NaN

[@SrEngr]

Which version of tensorflow is this?

[@kbd2820]

learning from the legend. It was an amazing experience. Thank you

[@tonsandes]

Hi Jeff, is there a way to access y_pred information? I want to build my loss function, but not using a conventional way, which pass y_pred and y_true to a tf of backend function. I need to apply a step to access y_pred information and after that apply a function to estimate the std and return the std value as the output of my loss function. Do you know how to do this?

[@heecheolcho3246]

Thank you for a good lecture. I have a question. y = tf.divide(1.0,tf.add(1,tf.exp(tf.negative(x)))) vs y = 1.0/(1+tf.exp(-x)) Is there any difference?

[@subhajitpaul8391]

Thank you so much for this amazing video.

[@prajith3676]

i was actually looking for this gradienttape() everywhere , thankyou..finally my doubt is cleared ... :-)

[@StormiestOdin2]

Hi Jeff, Thank you for all these great videos. I have a question about tensorflow. If I create a model but have no hidden layers does this make my model not a neural network but linear discriminant analysis. Like this : model = keras.Sequential([ keras.layers.Dense(12, activation="relu"), keras.layers.Dense(3, activation="softmax")

[@HeatonResearch]

Its both at that point.

[@StormiestOdin2]

Ahh. thank you. Really appreciate all the videos you put on KZbin has helped me loads with making my own neural network:)

[@zonexo5364]

Stranget, why did I get "Tensor("AddN:0", shape=(), dtype=float32)" as output instead?

[@zonexo5364]

realise that in tensorflow, I have to run a session - with tf.Session() as sess: print(dz_dx.eval())

[@tonihullzer1611]

Awesome work, liked and subscribed, excited to see more.

[@slime121212]

Thank you for this video, this question was very important to me and now I know how to work it out)

[@brubrudsi]

Also, in the beginning you said the derivative of x^2 is x. It is 2x.

[@HeatonResearch]

Yes you are correct, good point. All the more reason for me to use automatic differentiation. 😞

[@maulikmadhavi]

super explanation! subscribed!

[@shunnie8482]

Thanks for the amazing explanation, I finally understand GradientTape (I think at least haha).

[@HeatonResearch]

Glad it helped!

[@luchofrancisco]

Thanks, nice video!

[@brubrudsi]

The derivative of 4^2 is 0, not 8.

[@mockingbird3809]

I think you should place take derivatives of the function, not the Number inputted values. You should take the derivative of the Function and place the value(A Number) into the derivated function to get the numeric output. You were wrong. 0 comes only if you take derivative a constant Derivative of f(x) = C is Zero, In this case, the funtion is f(x)=X^2.

[@HA-pz7mv]

Great video thanks!!