I have met the same problem when using VAE for representation learning.
To solve this problem, you must be know what lead to nan during the training process. I think the  logvar.exp() in the following fomula lead to overflow in the running process
KLD = -0.5 * torch.sum(1 + logvar - mean.pow(2) - logvar.exp())

so, we need to limit logvar in a specific range by some means.
So, you can initialize weights of VAE in a small range, which is recommended in the range [-0.08, 0.08] to make sure the logvar is small, thus exp can not lead to overflow numerically.
Plus, if you are using CNN for image classification, you need to make image zero-means and unit variance.
For details, you can visit this page. Best wishes.
Use of exp in the Reparametrization 329