Solved by ptrblck in post #2 
              
              
                You would just need to wrap it in a torch.no_grad() block and manipulate the parameters as you want: 
model = torch.nn.Sequential(nn.Linear(10, 1, bias=False))

with torch.no_grad():
    model[0].weight = nn.Parameter(torch.ones_like(model[0].weight))
    model[0].weight[0, 0] = 2.
    model[0].weig…