Artificial Intelligence, Neural Networks, Calculus, Gradient Descent, Math.
simply our neuron function, which we solved earlier. Therefore: Image 15: Derivative of u=max(0, sum(w⊗x)+b) with respect to the weights v(y,u) is simply y-u. Therefore, we can find its derivative (with respect to w) using the distributive property and substituting in the derivative of u: Image 16: