Solved by ptrblck in post #2 
              
              
                .is_cuda is a tensor attribute and cannot be used on a module, as a module can hold parameters on different devices. 
Instead of calling it on the model, you could check some internal parameters. e.g. model.layer.weight.is_cuda.