First, set the parameters before the training. You can set it by accessing the .data attribute(which is a tensor) of the parameter. For example, a.data.copy_(b). You can also set it with torch.no_grad():.