Holla,
Considering my previous post 1 I found out I can add new params to the optimizer, but I somehow expected they will be also in the optimizer state dict.
Here is the full code:
import torch
import torch.optim as optim

model = torch.nn.Linear(5, 2)

# Initialize optimizer
optimizer = optim.SGD(model.parameters(), lr=0.001, momentum=0.9)
new = torch.randn(5, 5)
print(new)
optimizer.param_groups.append({'params': new})

print("---")
print(optimizer.param_groups[1])
print("---")
print(optimizer.state_dict())

And here is the output:
tensor([[ 0.7939, -0.3140, -0.3523,  0.8560, -0.4632],
        [-1.0833, -0.3396, -0.3500, -1.1630,  1.1092],
        [ 1.4704,  0.1906, -2.1621,  0.4114,  0.6820],
        [ 0.2702, -0.4855,  0.3020, -1.2136,  0.0537],
        [ 0.1583, -0.4217, -0.3520, -0.5498, -0.9010]])
---
{'params': tensor([[ 0.7939, -0.3140, -0.3523,  0.8560, -0.4632],
        [-1.0833, -0.3396, -0.3500, -1.1630,  1.1092],
        [ 1.4704,  0.1906, -2.1621,  0.4114,  0.6820],
        [ 0.2702, -0.4855,  0.3020, -1.2136,  0.0537],
        [ 0.1583, -0.4217, -0.3520, -0.5498, -0.9010]])}
---
{'state': {}, 'param_groups': [{'lr': 0.001, 'momentum': 0.9, 'dampening': 0, 'weight_decay': 0, 'nesterov': False, 'params': [140147531102680, 140147531102752]}, {'params': [140147531031176, 140147531102896, 140147531031176, 140147531102896, 140147531031176]}]}

The problem is the output. It doesn’t look right the state dict has no correct params. Instead it has some 140147531102680ish numbers? Can you add some comments why?