I’m looking for a model that will return an embedding vector that can be used in downstream classification tasks. I have been able to deploy the pretrained model sentence-transformers/all-mpnet-base-v2 · Hugging Face 1
to an endpoint and get embeddings from it. However, when I try to finetune the model with huggingface_estimator I get  an error as it doesn’t appear this model returns a loss since it doesn’t have labels (the data I am supplying is simply more domain specific text examples. How can I pass a loss when fine tuning a sentence transformer?
So fist I used the HuggingFaceModel from the sagemaker toolkit
from sagemaker.huggingface import HuggingFaceModel
hub = {
  'HF_MODEL_ID':'sentence-transformers/all-mpnet-base-v2',
  'HF_TASK':'feature-extraction'
}
# create Hugging Face Model Class
huggingface_model = HuggingFaceModel(
    transformers_version='4.6',
    pytorch_version='1.7',
    py_version='py36',
    env=hub,
    role=role,
)

In this example from amazon Fine-tune and host Hugging Face BERT models on Amazon SageMaker | AWS Machine Learning Blog 2 they are first fine tuning the model using the huggingfacee_estimator, and then they create a new HuggingFaceModel object and pass the model_data from the huggingface_estimator fine tuning job. Like this:
from sagemaker.huggingface.model import HuggingFaceModel

huggingface_model = sagemaker.huggingface.HuggingFaceModel(
env={ 'HF_TASK':'sentiment-analysis' },
model_data=huggingface_estimator.model_data,
role=role, # iam role with permissions to create an Endpoint
transformers_version="4.6.1", # transformers version used
pytorch_version="1.7.1", # pytorch version used
py_version='py36', # python version
)

My problem is when I try to finetune on my data, since it is not a classifier, the model output is not return a loss and I get an invalid key error from compute_loss() in transformer/trainer.py transformers/trainer.py at 3977b58437b8ce1ea1da6e31747d888efec2419b · huggingface/transformers · GitHub

KeyError: 'loss'

Does it make sense to finetune an embedding model? Is there a way to pass a loss function and have it included in the model output.
I am just passing it additional unlabelled data. How would one do this in sagemaker given that it is a feature extraction model and not a classification/prediction model?
Thanks