Ah ok, I was expecting similar behaviour as trainer.save_model(f’./results/’) where you save the model (and tokenizer etc) to a directory. And as a less experience user the doc string for the .save method sounded like a directory path was required.
For future people who have the same issue: you need to specify a file path like this, where the file name ends with .pth:
training_directory_file = "nli-few-shot/TPU/model_tpu_saved.pth"
accelerator.save(unwrapped_model.state_dict(), f'./results/{training_directory_file}')