I would recommend to transfer the models first to the CPU and store their state_dict as described here 7.
This would make sure you can recreate the models and load their state_dicts on different systems even if they don’t have a GPU.