I trained a model with multiple GPUs using model parallelism. When I tested that model with a single GPU, I got the following error:
RuntimeError: cuda runtime error (10) : invalid device ordinal at torch/csrc/cuda/Module.cpp:81

Does it mean that if I train a model with multiple GPUs, I have to run that model with the same number of GPUs? Is there any way to avoid that? I am using the recommended approach to save my PyTorch model. http://pytorch.org/docs/notes/serialization.html#recommend-saving-models 157