Hey @leifan,
Is your question related to inference or training?
If it is related to the question you could download a model from s3 when starting the training job the same way as you would do with data.
huggingface_estimator.fit({
    'train': 's3://<my-bucket>/<prefix>/train',  # containing train files
    'test': 's3://<my-bucket>/<prefix>/test',  # containing test files
    'model':  's3://<another-bucket>/<prefix>/model',  # containing model files (config.json, pytroch_model.bin, etc.)
})

SageMaker will then download when starting the training job all of these files into your container.
The Path of the files can either be accessed from the env var SM_CHANNEL_XXXX, e.g. SM_CHANNEL_TRAIN, SM_CHANNEL_MODEL or directly from, e.g. /opt/ml/input/train
And then you can load your model in your training script with
AutoModelForXXX.from_pretrained(os.environ.get('SM_CHANNEL_MODEL',None))