ptrblck:

your approach looks wrong as the new parameter assignment would not keep the gradients, so you should copy them manually back,


Could you please tell me what I should copy manually back?  The code in question 1 was wrote acrroding to your advice here, and I think I only change the data of a particular weight while the other parameters are the same still,  thus I don’t know what to copy back.