Yes, but DataParallel cannot scale beyond one machine. It is slower than DistributedDataParallel even in a single machine with multiple GPUs due to GIL contention across multiple threads and the extra overhead introduced by scatter and gather and per-iteration model replication.