aliosk:

I am trying to test multi-gpu training with the HF Trainer but for training a third party pytorch model.


Since you are not providing the code of that model, I can’t really help you with what is going wrong.