Deep Learning, Optimization, Neural Networks, Convex Optimization, Trends.
it’s nearly impossible for a model that costs hundreds of GPU and train for weeks, to develope a low-cost trainable model is important. For that point, Stochastic gradient descent is NOT a best method for learning Neural Network. Convex problems, if possible, will be one of the best alternative.