Hi,
I am wondering if there is a way to get gradients of output of activation function with respect to input to the activation. It is needed in backpropagation, so I am sure pytorch does that. I can probably use input values and take derivative as derivative of activation functions are easy, but is there a way to get that from pytorch? Also, I wonder what happens if the input is exactly 0 and our activation function is Relu?
Thanks.