Hello @charlesatftl,
Yes, you can fine-tune transformers anywhere you want and use it in SageMaker for Inference.
There are currently two options to use your model then:

Push your model to Models - Hugging Face 2 and deploy it directory from there, see: Deploy models to Amazon SageMaker 1

Create a model.tar.gz upload it to s3 and deploy it from there, see: Deploy models to Amazon SageMaker 5
In Addition to 2. here is the documentation on how to create a model.tar.gz Deploy models to Amazon SageMaker 16