I will try to do something similar to this, unfortunately I don’t think it will work quite as elegantly, since I want to offload the model to cuda in nn.dataparallel, at which point I don’t believe I can access any of the subparameters or functions of the model.