If you want to manipulate the parameters inplace, use copy_ and wrap it into a with torch.no_grad(): guard. Alternatively, if you want to create new parameters and want to pass them to an optimizer afterwards, assign new parameters.