I think one easy approach would be to

store the indices of all data samples, which should be removed in a list during training
remove these indices from the complete list of indices to create the “valid indices”
wrap the dataset in a Subset with the new “valid” indices,
and recreate the DataLoader after the epoch.