Solved by ptrblck in post #2 
              
              
                Your modules won’t be properly registered, if you append them in a plain Python list. 
Use nn.ModuleList instead and the model.to() call should push all parameters to the device.