Yes, you can use the torch.distributed package. You can find more details on this page 2.
We recommend you to follow the instructions of “3. Use single-machine multi-GPU DistributedDataParallel” on that page.