Optimization, Deep Learning, Convex Optimization, Optimal Performance, Approximation.
can converge wherever we initialize. For deep neural networks, it known empirically that it doesn’t happen. Practical Insights Because of this, we almost always can’t prove convergence or anything like that when we run back-propagation SGD on a deep net. In DL as NCO, many things could go wrong.