Thank you for your answer! Actually I am trying to replicate a specific pruning technique which requires backpropagating wrt mask. I will provide you with a simple exmaple, not quite the same as I am implementing, but hopefully it explains my problem and my (maybe strange) solution.
model = torch.nn.Linear(2,3)
mask = torch.tensor([1.,2.], requires_grad=True)

input = torch.tensor([0.5,0.3])
model.weight.data = model.weight * mask  ##this is something like multiplying weights through my pruning mask
output = model.forward(input)
loss = sum(output)

loss.backward()
print(mask.grad)
#None

Please don’t try to make sense from this piece of code, it purpose is to show, from what I’d like to have gradient. At this moment to get desired result I do:
model = torch.nn.Linear(2,3)
mask = torch.tensor([1.,2.], requires_grad=True)

temp = model.weight.data # new code
del model.weight
model.weight = temp

input = torch.tensor([0.5,0.3])
model.weight = model.weight * mask
output = model.forward(input)
loss = sum(output)

loss.backward()
print(mask.grad)
#Actual value

Is there a way to do it better? To ‘force’ Parameter class to be a non leaf node without deleting it? Because it ruins my model a bit.