Solved by ptrblck in post #2 
              
              
                You are currently manipulating the .data attribute, which is not supported and can yield unwanted side effects such as probably this one. 
If you want to replace the .weight parameter of the linear layer, wrap the assignment in a with torch.no_grad() block and assign a new nn.Parameter to it.