I don’t know when and where you are calling mem_update. Note that you should create the trainable nn.Parameters, pass them to the optimizer, and use them later. Currently you are creating the threshold parameter inside mem_update so if you are repeatedly calling this method, move the parameter creation outside of this method and pass it as an argument to it.