I would assume the input are not changed by the Dataset or DataLoader, as each sample is loaded and processed on the fly or do you expect them to change somehow?
Are you using any random transformations, which could potentially create the invalid outputs depending on the actual random values used in this transformation?