I’m also seeing the point you mentioned and think the current approach was chosen because it lets you focus on the fine-tuning mechanics. However, as you explained, it might be clear to experienced users that the “real” test dataset is missing, but might confuse newcomers.
Feel free to create an issue 2 to discuss this further.