Hi,
thanks for the quick reply. Indeed, this modification solved the issue but leaves me a little bit puzzled. Out of curiosity I tried the following:
a = torch.nn.Parameter(torch.ones([1], device='cpu'))
b = torch.nn.Parameter(torch.ones([1], device='cuda'))
c = torch.nn.Parameter(torch.ones([1])).to('cpu')
d = torch.nn.Parameter(torch.ones([1])).to('cuda')

print(a, end="\n\n")
print(b, end="\n\n")
print(c, end="\n\n")
print(d, end="\n\n")

which gave the following output
Parameter containing:
tensor([1.], requires_grad=True)

Parameter containing:
tensor([1.], device='cuda:0', requires_grad=True)

Parameter containing:
tensor([1.], requires_grad=True)

tensor([1.], device='cuda:0', grad_fn=<CopyBackwards>)

a, b, and c make sense, but what happens to d? From my basic pytorch understanding (and the help of stackoverflow) a and b are straightforward. c would be copied to cpu but as it is already stored there, the .to(‘cpu’) has no effect.
Now to d: My understanding until now was, that  torch.ones([1])  is a “normal” pytorch tensor that becomes a parameter (and thus differentiable) and gets moved to the GPU by the  to('cuda') in the last step. Finally, d points to the address on the GPU. But the print output indicates that something different is going on… Btw, what does  grad_fn=<CopyBackwards>  mean?