Check out the pruning functionalities added to PyTorch v1.4 under torch.nn.utils.prune. They were built with lottery ticket hypothesis research in mind. The masks are added as forward pre-hooks, and are therefore automatically included as an operation in the backward pass too, thus effectively freezing pruned parameters.
You might also want to check out the PyTorch pruning tutorial: https://pytorch.org/tutorials/intermediate/pruning_tutorial.html 29