Can it be finetuned for classification tasks?
#15
by
bilalghanem
- opened
Hi,
I was wondering, would it be possible to fine-tune the model for classification tasks? I don't know all the details in the back but I know that it's an embedding model.
Thanks!
Without knowing much more, the simplest thing might be to train a lightweight classifier on top of the embeddings themselves like an SVM
zpn
changed discussion status to
closed
Thanks @zpn ! I meant to add a classification head and fine-tune it (end-to-end), possible?
Yeah you definitely can! It might require some custom code or you might be able to use the NomicBertForSequenceClassification like here: https://github.com/nomic-ai/contrastors/blob/main/src/contrastors/trainers/glue.py#L59-L61.