Machine Learning, Neural Networks, Optimization, Mathematics.
needed to get close to the minimum grows as O(condition number): Number of steps grows as O(k). Condition number 1: minimization takes 1 step. Condition number 10: minimization takes 10 steps. The neural network’s optimization surface is badly conditioned (empirical evaluation in this paper), and