Solved by ptrblck in post #2 
              
              
                You could wrap the code into the no_grad() guard and use the .copy_ operation to fill the parameter. If you are initializing the plain tensor before wrapping it into an nn.Parameter, you could also skip the no_grad() guard.