Thanks for your reply. Yes I see your point. Even with cross validation however, you would still end up training the final model on the whole dataset without being able to test it at the very end. If you have a 5-fold cross validation, for example, you might train your model from scratch 5 times, and each time you keep 1/5th (20%) of the data for validation, which changes every time, so that at the end of the 5 runs, each data sample would have been in the validation set once and in the training set four times. Then you can average the metrics from the 5 runs, and ultimately re-train the model with 100% of your data assuming (which is still an assumption) that the averages of the performance metrics that you got across the 5 runs of the cross validation exercise while you trained the model with 80% of the data each time, is a good estimate of the model performance when trained with 100% of the data. So there are still some assumptions in there, but perhaps they’re more justifiable that simply doing what I was proposing earlier.