Python uses function scoping so unless you wrap the training and validation steps into own functions the tensors from a previous iterations might be kept alive.
Since you are running OOM during the validation I would guess that you are still holding references to some training tensors (and maybe even the computation graph), which would thus need additional memory to be able to perform the validation run.
Let me know, if this might be the case.