Hi @tim,
What’s the rationale behind this training architecture over the classical multi-GPU distributed training?
The framework and the underlying libraries are optimized to give you the best throughput in distributed training. You can either use torch.distributed 2 or Horovod 2 to achieve that, and you will certainly have a better GPU occupancy.