Should I use  DataParallel  or  DistributedDataParallel  to train the model on TPU Pod?


General guidance fo DataParallel vs DistributedDataParallel: https://pytorch.org/tutorials/intermediate/ddp_tutorial.html#comparison-between-dataparallel-and-distributeddataparallel 23