You are right and I missed the constructor inside MyModule.
You should get a warning using nn.ParameterList and nn.DataParallel, which is not supported in this setup, so you would need to fallback to registering the parameters separately:
UserWarning: nn.ParameterList is being used with DataParallel but this is not supported. This list will appear empty for the models replicated on each GPU except the original one.