Hi Iam!



 iamctR:

class GaussianModel(nn.Module):
    def __init__(self):
        super(GaussianModel, self).__init__()
        self.v1 = nn.Parameter(torch.zeros(1))
        self.v2 = self.v1*2
        self.g1=D.Normal(torch.tensor([4.0]), self.v1)
        self.g2=D.Normal(torch.tensor([4.0]), self.v2)
    def forward(self,x):
        return torch.mean(self.g1.log_prob(x)+self.g2.log_prob(x))



The problem is that self.v2 = self.v1*2 creates a new tensor
for v2, and does so only once at __init__() time.  Subsequent
updates to v1 (made by the optimizer) never get reflected in v2
(nor in g2).
(I don’t really understand how this leads to the specific autograd
RuntimeError you see, but this is the key error in your code.)
I would probably just do:
class GaussianModel (nn.Module):
    def __init__ (self):
        super (GaussianModel, self).__init__()
        self.v1 = nn.Parameter (torch.zeros (1))
    def forward (self, x):
        return torch.mean (D.Normal (torch.tensor ([4.0]), self.v1).log_prob (x) + D.Normal (torch.tensor ([4.0]), 2 * self.v1).log_prob (x))

I believe you could also implement this as:
class GaussianModel (nn.Module):
    def __init__ (self):
        super (GaussianModel, self).__init__()
        self.v1 = nn.Parameter (torch.zeros (1))
        self.v2 = self.v1 * 2   # these initial values will be overwritten
        self.g1=D.Normal (torch.tensor ([4.0]), self.v1)
        self.g2=D.Normal (torch.tensor ([4.0]), self.v2)
    def forward (self, x):
        self.v2.copy_ (self.v1 * 2)
        return torch.mean (self.g1.log_prob (x)+self.g2.log_prob (x))

but, to me, my first version is more readable.
As an aside, you are initializing g1 and g2 with a scale (standard
deviation) of 0.0.  I would expect this to lead to nans in your first
forward pass.  Up-to-date versions of pytorch will flag this:
>>> import torch
>>> torch.__version__
'1.9.0'
>>> d = torch.distributions.Normal (torch.tensor ([4.0]), torch.zeros (1), validate_args = False)
>>> d
Normal(loc: tensor([4.]), scale: tensor([0.]))
>>> d.log_prob (torch.tensor ([1.0]))
tensor([nan])
>>> d = torch.distributions.Normal (torch.tensor ([4.0]), torch.zeros (1))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\LisaBrown\Documents\admin\programs\Miniconda3\lib\site-packages\torch\distributions\normal.py", line 50, in __init__
    super(Normal, self).__init__(batch_shape, validate_args=validate_args)
  File "C:\Users\LisaBrown\Documents\admin\programs\Miniconda3\lib\site-packages\torch\distributions\distribution.py", line 53, in __init__
    raise ValueError("The parameter {} has invalid values".format(param))
ValueError: The parameter scale has invalid values

Best.
K. Frank