sentence_transformers_support (#3)

- Add support for Sentence Transformer (161751154d1c93b1d0f2e3aee4f9b03aeeb9a2a9)

1_Pooling/config.json

@@ -1,7 +0,0 @@
-{
-  "word_embedding_dimension": 384,
-  "pooling_mode_cls_token": false,
-  "pooling_mode_mean_tokens": false,
-  "pooling_mode_max_tokens": false,
-  "pooling_mode_mean_sqrt_len_tokens": false
-}

1_SpladePooling/config.json

@@ -0,0 +1,5 @@
+{
+    "pooling_strategy": "max",
+    "activation_function": "relu",
+    "word_embedding_dimension": null
+}

README.md

@@ -2,12 +2,16 @@
 language:
 - en
 license: apache-2.0
-library_name: transformers
 tags:
 - language
 - granite
 - embeddings
-pipeline_tag: sentence-similarity
+- sentence-transformers
+- sparse-encoder
+- sparse
+- splade
+pipeline_tag: feature-extraction
+library_name: sentence-transformers
 ---
 # Granite-Embedding-30m-Sparse
 
@@ -117,6 +121,83 @@ for r in res:
     print(r)
 
 ```
+
+**Usage with Sentence Transformers:** 
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+
+from sentence_transformers import SparseEncoder
+
+# Download from the 🤗 Hub
+model = SparseEncoder("ibm-granite/granite-embedding-30m-sparse")
+
+# Run inference
+docs = [
+    "Artificial intelligence was founded as an academic discipline in 1956.",
+    "Alan Turing was the first person to conduct substantial research in AI.",
+    "Born in Maida Vale, London, Turing was raised in southern England.",
+]
+docs_embeddings = model.encode_document(docs, max_active_dims=192)
+print(docs_embeddings.shape)
+# [3, 50265]
+
+queries = ["When was artificial intelligence founded", "Where was Turing born?"]
+queries_embeddings = model.encode_query(queries, max_active_dims=50)
+print(queries_embeddings.shape)
+# [2, 50265]
+
+# Get the similarity scores for the embeddings
+similarities = model.similarity(queries_embeddings, docs_embeddings)
+print(similarities.shape)
+# [2, 3]
+
+for i, query in enumerate(queries):
+    best_doc_index = similarities[i].argmax().item()
+
+    print(f"Query: {query}")
+    print(f"Best doc associate: Similarity: {similarities[i][best_doc_index]:.4f}, Doc: {docs[best_doc_index]}")
+    intersection = model.intersection(queries_embeddings[i], docs_embeddings[best_doc_index])
+    decoded_intersection = model.decode(intersection, top_k=10)
+    print("Top 10 tokens influencing the similarity:")
+    for token, score in decoded_intersection:
+        print(f"Token: {token}, Score: {score:.4f}")
+
+# Query: When was artificial intelligence founded
+# Best doc associate: Similarity: 12.3641, Doc: Artificial intelligence was founded as an academic discipline in 1956.
+# Top 10 tokens influencing the similarity:
+# Token: ĠAI, Score: 2.7591
+# Token: Ġintelligence, Score: 2.2971
+# Token: Ġartificial, Score: 1.7654
+# Token: Ġfounded, Score: 1.3254
+# Token: Ġinvention, Score: 0.9808
+# Token: Ġlearning, Score: 0.4847
+# Token: Ġcomputer, Score: 0.4789
+# Token: Ġrobot, Score: 0.3466
+# Token: Ġestablishment, Score: 0.3371
+# Token: Ġscientific, Score: 0.2804
+# Query: Where was Turing born?
+# Best doc associate: Similarity: 17.1359, Doc: Born in Maida Vale, London, Turing was raised in southern England.
+# Top 10 tokens influencing the similarity:
+# Token: uring, Score: 2.9761
+# Token: ĠTuring, Score: 2.4544
+# Token: Ġborn, Score: 2.4314
+# Token: ing, Score: 1.7760
+# Token: ure, Score: 1.7626
+# Token: Ġcomput, Score: 1.3356
+# Token: Ġraised, Score: 1.3285
+# Token: able, Score: 1.1940
+# Token: Ġphilosopher, Score: 0.4118
+# Token: Ġmachine, Score: 0.3977
+
+```
+
 **Evaluation:**
 
 Granite-Embedding-30m-Sparse is competive in performance to the naver/splade-v3-distilbert despite being half the parameter size. We also compare the sparse model with similar sized dense embedding counterpart `ibm-granite/granite-embedding-30m-english`. The performance of the models on MTEB Retrieval (i.e., BEIR) is reported below. 

config_sentence_transformers.json

@@ -0,0 +1,14 @@
+{
+  "model_type": "SparseEncoder",
+  "__version__": {
+    "sentence_transformers": "5.0.0",
+    "transformers": "4.50.3",
+    "pytorch": "2.6.0+cu124"
+  },
+  "prompts": {
+    "query": "",
+    "document": ""
+  },
+  "default_prompt_name": null,
+  "similarity_fn_name": "dot"
+}

modules.json

@@ -3,18 +3,12 @@
     "idx": 0,
     "name": "0",
     "path": "",
-    "type": "sentence_transformers.models.Transformer"
+    "type": "sentence_transformers.sparse_encoder.models.MLMTransformer"
   },
   {
     "idx": 1,
     "name": "1",
-    "path": "1_Pooling",
-    "type": "sentence_transformers.models.Pooling"
-  },
-  {
-    "idx": 2,
-    "name": "2",
-    "path": "2_Normalize",
-    "type": "sentence_transformers.models.Normalize"
+    "path": "1_SpladePooling",
+    "type": "sentence_transformers.sparse_encoder.models.SpladePooling"
   }
-]
+]

sentence_bert_config.json

@@ -1,4 +1,4 @@
 {
-  "max_seq_length": 512,
-  "do_lower_case": false
-}
+    "max_seq_length": 512,
+    "do_lower_case": false
+}