Optimization, Mathematics.
to $\frac{1}{i}$ . This decay was observed in some convolutional network problems and conjectured to hold more generally, see Section 3.5 of Noisy Quadratic Model paper. Because gradient descent on a quadratic is rotation and scale invariant, we can assume $H$ is diagonal with norm 1 without loss