Solved by albanD in post #2 
              
              
                Hi, 
The thing is to reduce memory usage, all optimizer modify the weights inplace. 
Which means that the content of your list called weights contains the same tensors that are modified inplace by the optimizer. 
You can add a weights.append(param.clone()) to make sure your store a different copy of…