Solved by ptrblck in post #4 
              
              
                Yes, your approach looks wrong as the new parameter assignment would not keep the gradients, so you should copy them manually back, if you want to apply the previously calculated gradients to a new parameter set (assuming that’s what the linked method really does). 


Don’t use the .data attribute…