Does PyTorch provide any options to train a model using two GPUs installed in two different virtual machines parallelly?

Relevant link: Distributed communication package - torch.distributed — PyTorch 1.8.0 documentation 4
Not sure if you can do this with virtual machines.

I found that the DataParallel function is supposed to run multi GPUs present on the same machine, but couldn’t able to find any suitable information for my requirement.

You are right: DataParallel is not designed for multi-node training.