The error is raised by some batchnorm layers, which cannot calculate the stats from a single sample.
If you leave the model in training mode, you would have to provide at least 2 samples in the batch.
Assuming this error is only raised at the end of the epoch, I guess the last batch might be smaller than the rest and you could drop it via setting drop_last=True in the DataLoader.