Solved by ptrblck in post #2 
              
              
                Inside the forward pass, you could create sampled_noise from the attributes of x: 
sampled_noise = torch.empty_like(x),normal_(mean=0, std=self.sigma)

or push the noise tensor to the device of x: 
sampled_noise = ...
sampled_noise = sampled_noise.to(x.device)