Hi albanD,
Thank you for your answer, and sorry for bringing back this topic. But here I thought if I pruned a model and then retrained it with backprop,  we are calculating the backward gradient with original weights, and update original weights then mask them? Because I observed that “grad” is only avaliable in “weight_origi”, and those pruned weight is still updating in “weight_origi”. Is there any way to retrain the model as if the pruned weights do not even exist(i.e. not participate in forward inference and backward update)?
Best regards