When I tried to load the trainer from a specific checkpoint (which were generated during a previous training process)
trainer.train("checkpoint-100")

The model did continue to train from the given checkpoint, but also I encountered this warning:
UserWarning: Please also save or load the state of the optimzer when saving or loading the scheduler.

warnings.warn(SAVE_STATE_WARNING, UserWarning)
Inside the “checkpoint-100” directory, there are 5 files: config.json, optimizer.pt, pytorch_model.bin, scheduler.pt, training_args.bin