Statistical vs. Deep Learning forecasting methods
It is something that bothers me about the ML literature is that they frequently present a large number of evaluation results such as precision and AUC but these are not qualified by error bars.  Typically they make a table which has different algorithms on one side and different problems on the other side and the highest score for a given problem gets bolded.<p>I know if you did the experiment over and over against with different splits you'd get slightly different scores so I'd like to see some guidance as to significance in terms of ① statistical significance,  and ② is it significant on a business level.  Would customers notice the difference?  Would it make better decisions that move the needle for revenue or other business metrics?<p>This study is an example where a drastically more expensive algorithm seems to produce a practically insignificant improvement.