Is there any update to this? I am also finding pytorch sampling slow. I am using torch.distributions.Normal(0, 1).sample.
Edit: I found torch.empty().normal_() faster than torch.distributions.Normal(0, 1).sample, so just managing with this now.