In GAN hacks 35 and his NIPS 2016 talk 19, Soumith Chintala (@smth)  suggests to check that the network gradients aren’t exploding:

check norms of gradients: if they are over 100 things are screwing up

How might I do that in PyTorch?