Hi,
I’m new to distributed training.
When I train with DistributedDataParallel do I get the functionality of DataParallel, meaning can I assume that on a single node if there is more than one GPU then all GPUs will be utilized on that node?
Thanks,
Zlapp