I would create separate Datasets for the train, val, and test folders, so that you could stick yo your current Dataset implementation and have a clean cut between the data splits to avoid data leakage.