@ptrblck ,
Thank you!



 ptrblck:

self.mask_weight = nn.Parameter(self.mask_weight * temp)


the upper code will generate a new parameter self.mask_weight. so I need to consider whether the new self.mask_weight is added to optimizer. is my understanding correct?
once I do self.mask_weight * temp,  the code return a tensor but not a parameter.  If I want to assign it parameter, I should do self.mask_weight = nn.Parameter(self.mask_weight * temp). right?