Shameless self-promotion of this helper package of mine. https://github.com/SsnL/PyTorch-Reparam-Module 95
Basically it allows you to activate a network with arbitrary parameter, and minimal code change.