I think you’d want to use DistributedDataParallel or somesuch to wrap the model in.
TorchVision, in its git repository, contains the script used for providing the pretrained models in the references directory (an underused/underappreciated resource IMHO): vision/references/detection at main · pytorch/vision · GitHub 1. This does include multi-gpu training.
Best regards
Thomas