Let me also add that, in the last scenario, loading the state_dict into a newly instantiated model will make it such that all your weight_origs and weight_masks will be properly filled in with the info from the state_dict, BUT the weight will still be a randomly sampled tensor from the new model instantiation.
Why does this matter?  weight_orig and weight_mask together can be used to recompute the  weight on the fly, through the  forward_pre_hook that PyTorch pruning uses. But this hook needs a forward call to act and recompute the  weight . Without that call, the  weight  will just be some random tensor that has nothing to do with the loaded weight_orig and weight_mask.
Therefore, either serialize you models after removing the pruning parametrization, or remember to set the  weight  correctly (by hand or with a forward call) before trying to prune it again. In practice, this means preferably calling _ = model(X), where X is some (even fake) input data batch, or, alternatively, setting the weight by hand weight = weight_orig * weight_mask (be careful with this).