hi all,
I have a question where I am apparently not able to find any answer.
Let’s say I have a dataset which is relatively small and I want to be able to test on the entire dataset to reduce the bias of my model.
I thought I could use something like k-fold cross-validation, but no matter wher I look, i only find the case where for each fold, the data is split in only training and test set. What I am interested in is to have for each fold a training, validation AND test data, where I train the model on the test data, determine when to stop training on the validation data and then test on the test data. I would then report the average of the scores obtained on the different test sets.
However, I am unsure on how to split the data and if the mentioned procedure is actually correct.
Would this for example be correct? (I fix [p] to be the validation set) and test on the other different test sets)
Dataset = [o, d, p, x]
Fold1: Train = [o, d], Validation = [p], Test=[x]
Fold2: Train = [o, x], Validation = [p], Test=[d]
Fold3: Train = [d, x], Validation = [p], Test=[o]
Thank for help!