How simple should a Baseline model be? I was making baseline model and was wondering how much time I should spend on it.
I've found a lot of article about the purpose of a baseline model, for instance;

A baseline takes only 10% of the time to develop, but will get us 90% of the way to achieve reasonably good results.

But how do I properly select one?

*

*I'm I choosing among multiple algorithms/models?

*I'm I suppose to tune my simple model or leave it with its default?


Mainly, how to do this is a question of experience. This will also tell you what kind of model is a good candidate for such a baseline. For instance, in time series forecasting, the simplest models, which are actually surprisingly hard to beat, are the historical average of a time series, and the last observation. Note that both models need precisely zero tuning!
An alternative approach would be to pick the simplest model that can at least output some "reasonable" figure. For time series forecasting, the simplest model would be one that always output a flat zero, but that is not "reasonable". So the next most simple would be one of the two above, and either one would be a good start.
Finally, you could also start by time-boxing your entire development effort, then allot the first 5-10% (or whatever is a reasonable number - I would use something closer to 5%, or even 2%, than to 10%) to building this simple benchmark. This allows for a little tuning, but keeps you from over-engineering something that is intended to only be a simple benchmark.
