Machine Learning, Gradient Descent, Hessian Matrix, Optimization, Artificial Intelligence.
= [[2, 0], [0, -2]] with eigenvalues 2 and -2. This mixed sign combination indicates a saddle point. Simple Analogy Imagine an elastic surface. The Hessian represents forces pulling and pushing that surface in various directions. The eigenvectors are the directions where these forces are “cleanest”