Artificial Intelligence, Machine Learning, Deep Neural Networks, Technology, Interview.
incorporate their findings. They argue that SGD seems like an overkill during the diffusion phase, which consumes most of the training epochs, and that much simpler optimization algorithms may be more efficient. The paper Opening the Black Box of Deep Neural Networks via Information is on arXiv.