Hello,
I am running a training algorithm and in one step, I need to perform Sampling from a Gaussian distribution with a given standard deviation.
Each one epoch in my training takes around 5 seconds if I don’t perform the sampling step.
However, If I do the sampling, it becomes too slow (1 epoch = 120 seconds)!!.
I am doing it using .
Before starting the training, I create a normal distribution object.
import torch.distributions as tdist
normal_dist = tdist.Normal(0.0, sigma ) 

then within my training loop:

noise = normal_dist.sample(w.size())

Any hints why is this happening and if there is a more efficient way?
EDIT:


torch.randn is also as slow.
I did comparison between tensorflow vs pytorch performance on random sampling, when the shape of the output noise small PyTorch tends to be faster, but if we are sampling big tensors, TensorFlow is way faster and Pytorch becomes too slow.
The size of the output in my epxeriment is 1024x128x128.