Sure thing, I agree that it’s definitely a convention in ML to have train/valid/test split. That’s fine, what I’m arguing is that the examples in the pytorch repo omit completely the validation set and only have a train and test set. Not only that but during training they expose the test set to the model to check its generalization capabilities. This from my perspective seems wrong since the test set should be untouched and not exposed to your model but only once in the end to get the overall performance.
The other thing that I’m arguing about is the conventional split train/valid/test which is fine if there is sufficient data, and here’s where the line gets fuzzy. What do we mean with sufficient data especially for deep learning models? Are 50k samples sufficient for a three way spilt?