Solved by sunformoon in post #3 
              
              
                Solved. Seems I have a bunch of tensors which need requires_grad to be False while I set it to be True. 
For a reference, one update of the previous error I got: “RuntimeError: Torch: not enough memory: you tried to allocate 1GB. Buy new RAM!”