There are couple public samples showing Hugging Face model deployment on SageMaker, for example those:

amazon-sagemaker-deploy-nlp-huggingface/deploy_pretrained_BART_seq2seq_PyTorch.ipynb at main · aws-samples/amazon-sagemaker-deploy-nlp-huggingface · GitHub 89
Serving PyTorch models in production with the Amazon SageMaker native TorchServe integration | AWS Machine Learning Blog 61
GitHub - aws-samples/amazon-sagemaker-bert-classify-pytorch: This sample show you how to train BERT on Amazon Sagemaker using Spot instances 48