Hi @MariosOreo,
Thanks for the reply. Unfortunately, redefining the Parameter in this way each time destroys the computation graph associated to it, so it doesn’t get updated by .step after .backward is computed. Whatever the solution is, it must preserve the computational graph for weights (by updating its value without instantiating a brand new Parameter with the same name).