Could you try to time the noise creation?
Are you using the CPU or GPU?
If you’re using GPU, you should synchronize before starting and stopping the timer:
torch.cuda.synchronize()
t0 = time.time()
noise = normal_dist.sample(w.size())
torch.cuda.synchronize()
t1 = time.time()