I still think the error is related to the fact that you are not replicating all Parameters, thus these are missing in the replicas.
If you only specify one GPU for DataParallel, the module will just be called without replication (line of code 19).
Maybe I’m not understanding your use case, but currently only the parameters requiring gradients will be replicated, which would create incomplete models.