Optimization, Deep Learning, Convex Optimization, Optimal Performance, Approximation.
First, convergence to a bad local minimum may happen. I such a case, we can re-optimize the system with different initialization and/or add extra noise to gradient updates. Second, we may face convergence to a saddle point that can be tackled by finding the hessian and computing a descent