Artificial Intelligence, Neural Networks, Calculus, Gradient Descent, Math.
respect to both the weights and biases. What is the partial derivative of neuron(z) with respect to z? Neuron(z)=max(0,z)=max(0, sum(w⊗x)+b). The max(0,z) function simply treats all negative values as 0. The graph would thus look something like this: Image 8: Max(0,z) // Source Looking at that