SentenceTransformers vs Nomic API Embeddings

#16
by arvind-kumar - opened

Using the AutoModel and SentenceTransformer code yields different results vs placing calls to the Nomic API as follows:

Using SentenceTransformer:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("nomic-ai/nomic-embed-text-v1", trust_remote_code=True)
sentences = ['search_query: What is TSNE?', 'search_query: Who is Laurens van der Maaten?']
embeddings = model.encode(sentences)
print(embeddings)

Using Nomic wrapper:

from nomic import embed

sentences = ['search_query: What is TSNE?', 'search_query: Who is Laurens van der Maaten?']

output = embed.text(
texts=sentences,
model='nomic-embed-text-v1',
dimensionality=768,
task_type='search_document'
)

print(output)

Would you know why?

zpn changed discussion status to closed

Sign up or log in to comment