Glad we figured out how to make it work!
Yes, the parameters will always be the unpruned versions of the tensors, until you .remove() the reparametrization.
One option: grab both the unpruned parameters from self.parameters() and the masks from self.buffers() and do the multiplication manually (con: would require a new logic block to be executed only if the mask buffers exist).
Another option: without directly accessing __dict__, do something like: weights = [module.weight for module in self.modules() if hasattr(module, 'weight')], or whatever this needs to be for your model to work. In your specific case, I guess that, since you don’t have single tensors called 'weight' in a variety of modules but a single module with a variety of tensors called 'weight{i}', then it would look something like (getattr(self, 'weight{}'.format(i)) for i in range(len(self.weights))), instead. Should work fine for both pruned and unpruned version of the model.
Re this:



 ra_schoonhoven:

# This runs fine when I run the network (unpruned) in 1.1.0
weights = (self.__getattr__("weight{}".format(i)) for i in range(len(self.weights)))

# But throws the following Error for the pruned network in 1.4.0
AttributeError: 'MSDBlock2d' object has no attribute 'weight0'



you should do (getattr(self, "weight{}".format(i)) for i in range(len(self.weights))) instead of __getattr__. See this for reference: https://github.com/pytorch/pytorch/blob/master/torch/nn/modules/module.py#L580 3