Hello @kjackson,
How did you try to fine-tune it the code you shared is only on how to deploy it.
We have a detailed “Getting Started” example with video support to run your first training on Amazon SageMaker: Get started 4 This might help you get started.

Does it make sense to finetune an embedding model? Is there a way to pass a loss function and have it included in the model output.

Yes it makes sense to further fine-tune a language model to let it “learn” a new context/domain