I figured out my issues. The error I was getting was from loading the best model after training using the flag load_best_model_at_end=True. The model is not saved on every node when doing multi instance training, so the FileNoteFound error occurs. This flag will cause issues in the trainer if youre using transformers <= 4.6.
This issue is solved when using version of the AWS Deep Learning containers with a version of Hugging Face greater than 4.6 such as 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-training:1.9.0-transformers4.11.0-gpu-py38-cu111-ubuntu20.04, or by setting load_best_model_at_end=False
Thanks for the help!