You won’t be able to call loss.backward() inside a block, where gradient calculation is disabled via with torch.set_grad_enabled(False).
You could of course enable it, but calculating gradients in the validation set (and updating the model) is a bad idea, as you are leaking the validation dataset into the training and thus make the validation set unusable to validate the training.