Derivatives, Sigmoid Function, Tanh Function, Relu.
you set the derivative to be equal to 1. It always had to be 0, it doesn’t matter. If you’re an expert in optimization, technically, g prime then becomes what’s called a sub-gradient of the activation function g of z, which is why gradient descent still works. But you can think of it as that, the