Deep Forest: Towards an Alternative to Deep Neural Networks
"In contrast to deep neural networks which require great effort in hyper-parameter tuning, gcForest is much easier to train."Hyperparameter tuning is not as much of an issue with deep neural networks anymore. Thanks to BatchNorm and more robust optimization algorithms, most of the time you can simply use Adam with a default learning rate of 0.001 and do pretty well. Dropout is not even necessary with many models that use BatchNorm nowadays, so generally tuning there is not an issue either. Many layers of 3x3 conv with stride 1 is still magical.Basically: deep NNs can work pretty well with little to no tuning these days. The defaults just work.