Solved by ptrblck in post #2 
              
              
                Using an nn.ModuleList will make sure that all parameters are properly transferred to the device, if you are using model.to() or a data parallel approach. 
The attribute should still be found using a plain list object, but should yield a device mismatch error. 
Are you trying to create these modules…