swap_swa_sgd just swaps the current parameters for the swa ones in each call as you can see in the source code 8.
If you would like to validate your model, you could call it once before the actual validation, and again afterwards to get your original parameters back.
I’m not sure if you can revert the batchnorm update.