It might be easier to run model parallel on multiple GPUs in the same machine and distributed data parallel across machines. Checkout this section 41 for more details.
For your above use case, you will need to create multiple process groups. Given the above configuration, 4 nodes, and 2 GPUs per node, I assume you will have 8 processes, one process per GPU. Then you can create:

Two process groups of world size 4, which will be responsible for send/recv outputs/gradients across machines.
One process group of world size 2 on EACH machine, which will do the distributed data parallel on the machine.

The reason you need the above setting is because DistributedDataParallel would expect all processes in the same group are training the same model in a synchronized fashion. It won’t work if you only use 2 processes in the same group of size 8.
See the new_group 6 API.
BTW, the torch.distributed.rpc 16 API might make send/recv outputs/grads easier for you, and it also supports distributed autograd.