Try to define a custom module deriving from nn.Module and create the weight parameter as nn.Parameter. Alternatively if you would like to stick to the functional approach, just define weight = nn.Parameter(torch.randn(2, 2)) and pass [weight] to your optimizer.