Calculus, Backpropagation, Neural Networks, Mathematics, AI.
piecewise (d = 0 if z < 0, d = 1 if z > 0). At exactly 0 it’s not differentiable, but we still are able to use it since the chance of landing on exactly 0.000… is rare, and also, if that does happen, ReLU automatically shifts it to either < 0 or > 0. Now that we’ve found our derivative w.r.t z2, we