Optimization, Deep Learning, Convex Optimization, Optimal Performance, Approximation.
direction. Third, getting stuck in a region of low gradient magnitude that can be solved using batchnorm or designing networks efficiently using a rectified linear unit (ReLU) activation function for instance. Fourth, we may take huge steps and diverging because of the high curvature. In that case,