Deep Learning, Regularisation, Optimization Algorithms, Deep Neural Networks, Gradient Descent.
rate — α Momentum — β Adam’s hyperparameter — β1, β2, ε Number of hidden layers Number of hidden units for different layers Learning rate decay Mini-batch size Learning rate usually proves to be the most important among the above. This is followed by the number of hidden units, momentum, mini-batch