I see. And the following also fails:
x = torch.randn(1, requires_grad=True)
y = torch.randn(1, requires_grad=False)

y[0] = 2 # Works
l1 = x * y
y[0] = 3 # Does not work
l2 = l1 * y
out.backward()

Which makes sense.
Thanks for the reply!