Hi,
I am trying to fine-tune the BART model checkpoints on a large dataset(around 1M data points). Since the dataset is large, I want to utilize a multi-GPU setup but I see that because of this line 29 it’s not currently possible to train in a multi-gpu setting. Any work arounds for it?
@sshleifer Tagging you here since you’ve worked with BART and summarization in particular a lot on the repo.