Solved by JuanFMontesinos in post #8 
              
              
                Well, I don’t know at which point of the code these happens but makes sense since gradients are tracked at a tensor level. Sooo when you call the iterable the parameters are returned only once (you don’t want several backprop updates). 
If you look at pieces of code which uses an already-instantiate…