Based on your code snippet it seems self.weights is an int and will not be trained, so I’m unsure how the masking is performed as it seems you are scaling the activation instead.



 Marco_Repetto:

Now I would like to be able to modify the new_mask dynamically


If you would like to train the self.weights parameter, create it as an nn.Parameter and pass it to the optimizer.