MesTruck commited on
Commit
ac149b6
·
verified ·
1 Parent(s): 2810552

Upload folder using huggingface_hub

Browse files
.gitattributes CHANGED
@@ -33,3 +33,10 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ sts-multilingual-mpnet-base-v2.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
37
+ sts-multilingual-mpnet-base-v2.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
38
+ sts-multilingual-mpnet-base-v2.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
39
+ sts-multilingual-mpnet-base-v2.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
40
+ sts-multilingual-mpnet-base-v2.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
41
+ sts-multilingual-mpnet-base-v2.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
42
+ sts-multilingual-mpnet-base-v2.bf16.gguf filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,781 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: []
3
+ library_name: sentence-transformers
4
+ tags:
5
+ - mteb
6
+ - sentence-transformers
7
+ - sentence-similarity
8
+ - feature-extraction
9
+ - dataset_size:100K<n<1M
10
+ - loss:AnglELoss
11
+ - autoquant
12
+ - gguf
13
+ base_model: sentence-transformers/paraphrase-multilingual-mpnet-base-v2
14
+ widget:
15
+ - source_sentence: 有些人在路上溜达。
16
+ sentences:
17
+ - Folk går
18
+ - Otururken gitar çalan adam.
19
+ - ארה"ב קבעה שסוריה השתמשה בנשק כימי
20
+ - source_sentence: 緬甸以前稱為緬甸。
21
+ sentences:
22
+ - 缅甸以前叫缅甸。
23
+ - This is very contradictory.
24
+ - 한 남자가 아기를 안고 의자에 앉아 잠들어 있다.
25
+ - source_sentence: אדם כותב.
26
+ sentences:
27
+ - האדם כותב.
28
+ - questa non è una risposta.
29
+ - 7 שוטרים נהרגו ו-4 שוטרים נפצעו.
30
+ - source_sentence: הם מפחדים.
31
+ sentences:
32
+ - liên quan đến rủi ro đáng kể;
33
+ - A man is playing a guitar.
34
+ - A man is playing a piano.
35
+ - source_sentence: 一个女人正在洗澡。
36
+ sentences:
37
+ - A woman is taking a bath.
38
+ - En jente børster håret sitt
39
+ - אדם מחלק תפוח אדמה.
40
+ pipeline_tag: sentence-similarity
41
+ model-index:
42
+ - name: Gameselo/STS-multilingual-mpnet-base-v2
43
+ results:
44
+ - task:
45
+ type: STS
46
+ dataset:
47
+ name: MTEB STS22
48
+ type: mteb/sts22-crosslingual-sts
49
+ config: it
50
+ split: test
51
+ revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
52
+ metrics:
53
+ - type: cosine_spearman
54
+ value: 0.6847049462613332
55
+ - task:
56
+ type: STS
57
+ dataset:
58
+ name: MTEB STS22
59
+ type: mteb/sts22-crosslingual-sts
60
+ config: es
61
+ split: test
62
+ revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
63
+ metrics:
64
+ - type: cosine_spearman
65
+ value: 0.6620948502618977
66
+ - task:
67
+ type: STS
68
+ dataset:
69
+ name: MTEB STS22
70
+ type: mteb/sts22-crosslingual-sts
71
+ config: fr
72
+ split: test
73
+ revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
74
+ metrics:
75
+ - type: cosine_spearman
76
+ value: 0.7875616631597785
77
+ - task:
78
+ type: STS
79
+ dataset:
80
+ name: MTEB STS22
81
+ type: mteb/sts22-crosslingual-sts
82
+ config: pl-en
83
+ split: test
84
+ revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
85
+ metrics:
86
+ - type: cosine_spearman
87
+ value: 0.7510805416538202
88
+ - task:
89
+ type: STS
90
+ dataset:
91
+ name: MTEB STS22
92
+ type: mteb/sts22-crosslingual-sts
93
+ config: ar
94
+ split: test
95
+ revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
96
+ metrics:
97
+ - type: cosine_spearman
98
+ value: 0.6265329479575293
99
+ - task:
100
+ type: STS
101
+ dataset:
102
+ name: MTEB STS22
103
+ type: mteb/sts22-crosslingual-sts
104
+ config: pl
105
+ split: test
106
+ revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
107
+ metrics:
108
+ - type: cosine_spearman
109
+ value: 0.4335552432730643
110
+ - task:
111
+ type: STS
112
+ dataset:
113
+ name: MTEB STS22
114
+ type: mteb/sts22-crosslingual-sts
115
+ config: de
116
+ split: test
117
+ revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
118
+ metrics:
119
+ - type: cosine_spearman
120
+ value: 0.5774252131250034
121
+ - task:
122
+ type: STS
123
+ dataset:
124
+ name: MTEB STS22
125
+ type: mteb/sts22-crosslingual-sts
126
+ config: tr
127
+ split: test
128
+ revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
129
+ metrics:
130
+ - type: cosine_spearman
131
+ value: 0.6383757017928495
132
+ - task:
133
+ type: STS
134
+ dataset:
135
+ name: MTEB STS22
136
+ type: mteb/sts22-crosslingual-sts
137
+ config: es-it
138
+ split: test
139
+ revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
140
+ metrics:
141
+ - type: cosine_spearman
142
+ value: 0.6624635951676386
143
+ - task:
144
+ type: STS
145
+ dataset:
146
+ name: MTEB STS22
147
+ type: mteb/sts22-crosslingual-sts
148
+ config: ru
149
+ split: test
150
+ revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
151
+ metrics:
152
+ - type: cosine_spearman
153
+ value: 0.5866853707548388
154
+ - task:
155
+ type: STS
156
+ dataset:
157
+ name: MTEB STS22
158
+ type: mteb/sts22-crosslingual-sts
159
+ config: en
160
+ split: test
161
+ revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
162
+ metrics:
163
+ - type: cosine_spearman
164
+ value: 0.6385354535483773
165
+ - task:
166
+ type: STS
167
+ dataset:
168
+ name: MTEB STS22
169
+ type: mteb/sts22-crosslingual-sts
170
+ config: zh-en
171
+ split: test
172
+ revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
173
+ metrics:
174
+ - type: cosine_spearman
175
+ value: 0.6537294853166558
176
+ - task:
177
+ type: STS
178
+ dataset:
179
+ name: MTEB STS22
180
+ type: mteb/sts22-crosslingual-sts
181
+ config: zh
182
+ split: test
183
+ revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
184
+ metrics:
185
+ - type: cosine_spearman
186
+ value: 0.6319430830291571
187
+ - task:
188
+ type: STS
189
+ dataset:
190
+ name: MTEB STS22
191
+ type: mteb/sts22-crosslingual-sts
192
+ config: fr-pl
193
+ split: test
194
+ revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
195
+ metrics:
196
+ - type: cosine_spearman
197
+ value: 0.8451542547285167
198
+ - task:
199
+ type: STS
200
+ dataset:
201
+ name: MTEB STS22
202
+ type: mteb/sts22-crosslingual-sts
203
+ config: de-fr
204
+ split: test
205
+ revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
206
+ metrics:
207
+ - type: cosine_spearman
208
+ value: 0.5798716781400349
209
+ - task:
210
+ type: STS
211
+ dataset:
212
+ name: MTEB STS22
213
+ type: mteb/sts22-crosslingual-sts
214
+ config: es-en
215
+ split: test
216
+ revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
217
+ metrics:
218
+ - type: cosine_spearman
219
+ value: 0.7518021273920814
220
+ - task:
221
+ type: STS
222
+ dataset:
223
+ name: MTEB STS22
224
+ type: mteb/sts22-crosslingual-sts
225
+ config: de-en
226
+ split: test
227
+ revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
228
+ metrics:
229
+ - type: cosine_spearman
230
+ value: 0.5749790581441845
231
+ - task:
232
+ type: STS
233
+ dataset:
234
+ name: MTEB STS22
235
+ type: mteb/sts22-crosslingual-sts
236
+ config: de-pl
237
+ split: test
238
+ revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
239
+ metrics:
240
+ - type: cosine_spearman
241
+ value: 0.44220332625465214
242
+ - task:
243
+ type: STS
244
+ dataset:
245
+ name: MTEB STSBenchmark
246
+ type: mteb/stsbenchmark-sts
247
+ config: default
248
+ split: test
249
+ revision: b0fddb56ed78048fa8b90373c8a3cfc37b684831
250
+ metrics:
251
+ - type: cosine_spearman
252
+ value: 0.9762486352335524
253
+ - task:
254
+ type: STS
255
+ dataset:
256
+ name: MTEB STS17
257
+ type: mteb/sts17-crosslingual-sts
258
+ config: en-tr
259
+ split: test
260
+ revision: faeb762787bd10488a50c8b5be4a3b82e411949c
261
+ metrics:
262
+ - type: cosine_spearman
263
+ value: 0.7987027653005363
264
+ - task:
265
+ type: STS
266
+ dataset:
267
+ name: MTEB STS17
268
+ type: mteb/sts17-crosslingual-sts
269
+ config: ko-ko
270
+ split: test
271
+ revision: faeb762787bd10488a50c8b5be4a3b82e411949c
272
+ metrics:
273
+ - type: cosine_spearman
274
+ value: 0.9766336939338607
275
+ - task:
276
+ type: STS
277
+ dataset:
278
+ name: MTEB STS17
279
+ type: mteb/sts17-crosslingual-sts
280
+ config: fr-en
281
+ split: test
282
+ revision: faeb762787bd10488a50c8b5be4a3b82e411949c
283
+ metrics:
284
+ - type: cosine_spearman
285
+ value: 0.9067607122592818
286
+ - task:
287
+ type: STS
288
+ dataset:
289
+ name: MTEB STS17
290
+ type: mteb/sts17-crosslingual-sts
291
+ config: en-ar
292
+ split: test
293
+ revision: faeb762787bd10488a50c8b5be4a3b82e411949c
294
+ metrics:
295
+ - type: cosine_spearman
296
+ value: 0.7703365842088069
297
+ - task:
298
+ type: STS
299
+ dataset:
300
+ name: MTEB STS17
301
+ type: mteb/sts17-crosslingual-sts
302
+ config: nl-en
303
+ split: test
304
+ revision: faeb762787bd10488a50c8b5be4a3b82e411949c
305
+ metrics:
306
+ - type: cosine_spearman
307
+ value: 0.9114826394926738
308
+ - task:
309
+ type: STS
310
+ dataset:
311
+ name: MTEB STS17
312
+ type: mteb/sts17-crosslingual-sts
313
+ config: it-en
314
+ split: test
315
+ revision: faeb762787bd10488a50c8b5be4a3b82e411949c
316
+ metrics:
317
+ - type: cosine_spearman
318
+ value: 0.9246785886944904
319
+ - task:
320
+ type: STS
321
+ dataset:
322
+ name: MTEB STS17
323
+ type: mteb/sts17-crosslingual-sts
324
+ config: ar-ar
325
+ split: test
326
+ revision: faeb762787bd10488a50c8b5be4a3b82e411949c
327
+ metrics:
328
+ - type: cosine_spearman
329
+ value: 0.8124393788492182
330
+ - task:
331
+ type: STS
332
+ dataset:
333
+ name: MTEB STS17
334
+ type: mteb/sts17-crosslingual-sts
335
+ config: es-es
336
+ split: test
337
+ revision: faeb762787bd10488a50c8b5be4a3b82e411949c
338
+ metrics:
339
+ - type: cosine_spearman
340
+ value: 0.872701191632785
341
+ - task:
342
+ type: STS
343
+ dataset:
344
+ name: MTEB STS17
345
+ type: mteb/sts17-crosslingual-sts
346
+ config: en-de
347
+ split: test
348
+ revision: faeb762787bd10488a50c8b5be4a3b82e411949c
349
+ metrics:
350
+ - type: cosine_spearman
351
+ value: 0.9109414091487618
352
+ - task:
353
+ type: STS
354
+ dataset:
355
+ name: MTEB STS17
356
+ type: mteb/sts17-crosslingual-sts
357
+ config: es-en
358
+ split: test
359
+ revision: faeb762787bd10488a50c8b5be4a3b82e411949c
360
+ metrics:
361
+ - type: cosine_spearman
362
+ value: 0.8553203530552356
363
+ - task:
364
+ type: STS
365
+ dataset:
366
+ name: MTEB STS17
367
+ type: mteb/sts17-crosslingual-sts
368
+ config: en-en
369
+ split: test
370
+ revision: faeb762787bd10488a50c8b5be4a3b82e411949c
371
+ metrics:
372
+ - type: cosine_spearman
373
+ value: 0.9378741534997558
374
+ ---
375
+
376
+ ## State-of-the-Art Results Comparison (MTEB STS Multilingual Leaderboard)
377
+
378
+ | Dataset | State-of-the-art (Multi) | STSb-XLM-RoBERTa-base | STS Multilingual MPNet base v2 |
379
+ |-------------------|--------------------------|-----------------------|--------------------------------------|
380
+ | Average | 73.17 | 71.68 | **73.89** |
381
+ | STS17 (ar-ar) | **81.87** | 80.43 | 81.24 |
382
+ | STS17 (en-ar) | **81.22** | 76.3 | 77.03 |
383
+ | STS17 (en-de) | 87.3 | 91.06 | **91.09** |
384
+ | STS17 (en-tr) | 77.18 | **80.74** | 79.87 |
385
+ | STS17 (es-en) | **88.24** | 83.09 | 85.53 |
386
+ | STS17 (es-es) | **88.25** | 84.16 | 87.27 |
387
+ | STS17 (fr-en) | 88.06 | **91.33** | 90.68 |
388
+ | STS17 (it-en) | 89.68 | **92.87** | 92.47 |
389
+ | STS17 (ko-ko) | 83.69 | **97.67** | 97.66 |
390
+ | STS17 (nl-en) | 88.25 | **92.13** | 91.15 |
391
+ | STS22 (ar) | 58.67 | 58.67 | **62.66** |
392
+ | STS22 (de) | **60.12** | 52.17 | 57.74 |
393
+ | STS22 (de-en) | **60.92** | 58.5 | 57.5 |
394
+ | STS22 (de-fr) | **67.79** | 51.28 | 57.99 |
395
+ | STS22 (de-pl) | **58.69** | 44.56 | 44.22 |
396
+ | STS22 (es) | **68.57** | 63.68 | 66.21 |
397
+ | STS22 (es-en) | **78.8** | 70.65 | 75.18 |
398
+ | STS22 (es-it) | **75.04** | 60.88 | 66.25 |
399
+ | STS22 (fr) | **83.75** | 76.46 | 78.76 |
400
+ | STS22 (fr-pl) | 84.52 | 84.52 | **84.52** |
401
+ | STS22 (it) | **79.28** | 66.73 | 68.47 |
402
+ | STS22 (pl) | 42.08 | 41.18 | **43.36** |
403
+ | STS22 (pl-en) | **77.5** | 64.35 | 75.11 |
404
+ | STS22 (ru) | **61.71** | 58.59 | 58.67 |
405
+ | STS22 (tr) | **68.72** | 57.52 | 63.84 |
406
+ | STS22 (zh-en) | **71.88** | 60.69 | 65.37 |
407
+ | STSb | 89.86 | 95.05 | **95.15** |
408
+
409
+ **Bold** indicates the best result in each row.
410
+
411
+ # SentenceTransformer based on sentence-transformers/paraphrase-multilingual-mpnet-base-v2
412
+
413
+ This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/paraphrase-multilingual-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-multilingual-mpnet-base-v2). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
414
+
415
+ ## Model Details
416
+
417
+ ### Model Description
418
+ - **Model Type:** Sentence Transformer
419
+ - **Base model:** [sentence-transformers/paraphrase-multilingual-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-multilingual-mpnet-base-v2) <!-- at revision 79f2382ceacceacdf38563d7c5d16b9ff8d725d6 -->
420
+ - **Maximum Sequence Length:** 128 tokens
421
+ - **Output Dimensionality:** 768 tokens
422
+ - **Similarity Function:** Cosine Similarity
423
+ <!-- - **Training Dataset:** Unknown -->
424
+ <!-- - **Language:** Unknown -->
425
+ <!-- - **License:** Unknown -->
426
+
427
+ ### Model Sources
428
+
429
+ - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
430
+ - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
431
+ - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
432
+
433
+ ### Full Model Architecture
434
+
435
+ ```
436
+ SentenceTransformer(
437
+ (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: XLMRobertaModel
438
+ (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
439
+ )
440
+ ```
441
+
442
+ ## Usage
443
+
444
+ ### Direct Usage (Sentence Transformers)
445
+
446
+ First install the Sentence Transformers library:
447
+
448
+ ```bash
449
+ pip install -U sentence-transformers
450
+ ```
451
+
452
+ Then you can load this model and run inference.
453
+ ```python
454
+ from sentence_transformers import SentenceTransformer
455
+
456
+ # Download from the 🤗 Hub
457
+ model = SentenceTransformer("Gameselo/STS-multilingual-mpnet-base-v2")
458
+ # Run inference
459
+ sentences = [
460
+ '一个女人正在洗澡。',
461
+ 'A woman is taking a bath.',
462
+ 'En jente børster håret sitt',
463
+ ]
464
+ embeddings = model.encode(sentences)
465
+ print(embeddings.shape)
466
+ # [3, 768]
467
+
468
+ # Get the similarity scores for the embeddings
469
+ similarities = model.similarity(embeddings, embeddings)
470
+ print(similarities.shape)
471
+ # [3, 3]
472
+ ```
473
+
474
+ <!--
475
+ ### Direct Usage (Transformers)
476
+
477
+ <details><summary>Click to see the direct usage in Transformers</summary>
478
+
479
+ </details>
480
+ -->
481
+
482
+ <!--
483
+ ### Downstream Usage (Sentence Transformers)
484
+
485
+ You can finetune this model on your own dataset.
486
+
487
+ <details><summary>Click to expand</summary>
488
+
489
+ </details>
490
+ -->
491
+
492
+ <!--
493
+ ### Out-of-Scope Use
494
+
495
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
496
+ -->
497
+
498
+ ## Evaluation
499
+
500
+ ### Metrics
501
+
502
+ #### Semantic Similarity
503
+ * Dataset: `sts-dev`
504
+ * Evaluated with [<code>EmbeddingSimilarityEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.EmbeddingSimilarityEvaluator)
505
+
506
+ | Metric | Value |
507
+ |:--------------------|:-----------|
508
+ | pearson_cosine | 0.9551 |
509
+ | **spearman_cosine** | **0.9593** |
510
+ | pearson_manhattan | 0.927 |
511
+ | spearman_manhattan | 0.9383 |
512
+ | pearson_euclidean | 0.9278 |
513
+ | spearman_euclidean | 0.9394 |
514
+ | pearson_dot | 0.876 |
515
+ | spearman_dot | 0.8865 |
516
+ | pearson_max | 0.9551 |
517
+ | spearman_max | 0.9593 |
518
+
519
+ #### Evalutation results vs SOTA results
520
+ * Dataset: `sts-test`
521
+ * Evaluated with [<code>EmbeddingSimilarityEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.EmbeddingSimilarityEvaluator)
522
+
523
+ | Metric | Value |
524
+ |:--------------------|:-----------|
525
+ | pearson_cosine | 0.948 |
526
+ | **spearman_cosine** | **0.9515** |
527
+ | pearson_manhattan | 0.9252 |
528
+ | spearman_manhattan | 0.9352 |
529
+ | pearson_euclidean | 0.9258 |
530
+ | spearman_euclidean | 0.9364 |
531
+ | pearson_dot | 0.8443 |
532
+ | spearman_dot | 0.8435 |
533
+ | pearson_max | 0.948 |
534
+ | spearman_max | 0.9515 |
535
+
536
+ <!--
537
+ ## Bias, Risks and Limitations
538
+
539
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
540
+ -->
541
+
542
+ <!--
543
+ ### Recommendations
544
+
545
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
546
+ -->
547
+
548
+ ## Training Details
549
+
550
+ ### Training Dataset
551
+
552
+ #### Unnamed Dataset
553
+
554
+
555
+ * Size: 226,547 training samples
556
+ * Columns: <code>sentence_0</code>, <code>sentence_1</code>, and <code>label</code>
557
+ * Approximate statistics based on the first 1000 samples:
558
+ | | sentence_0 | sentence_1 | label |
559
+ |:--------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------|
560
+ | type | string | string | float |
561
+ | details | <ul><li>min: 3 tokens</li><li>mean: 20.05 tokens</li><li>max: 128 tokens</li></ul> | <ul><li>min: 4 tokens</li><li>mean: 19.94 tokens</li><li>max: 128 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 1.92</li><li>max: 398.6</li></ul> |
562
+ * Samples:
563
+ | sentence_0 | sentence_1 | label |
564
+ |:-------------------------------------------------------------------|:----------------------------------------------------------------|:---------------------------------|
565
+ | <code>Bir kadın makineye dikiş dikiyor.</code> | <code>Bir kadın biraz et ekiyor.</code> | <code>0.12</code> |
566
+ | <code>Snowden 'gegeven vluchtelingendocument door Ecuador'.</code> | <code>Snowden staat op het punt om uit Moskou te vliegen</code> | <code>0.24000000953674316</code> |
567
+ | <code>Czarny pies idzie mostem przez wodę</code> | <code>Czarny pies nie idzie mostem przez wodę</code> | <code>0.74000000954</code> |
568
+ * Loss: [<code>AnglELoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#angleloss) with these parameters:
569
+ ```json
570
+ {
571
+ "scale": 20.0,
572
+ "similarity_fct": "pairwise_angle_sim"
573
+ }
574
+ ```
575
+
576
+ ### Training Hyperparameters
577
+ #### Non-Default Hyperparameters
578
+
579
+ - `per_device_train_batch_size`: 256
580
+ - `per_device_eval_batch_size`: 256
581
+ - `num_train_epochs`: 10
582
+ - `multi_dataset_batch_sampler`: round_robin
583
+
584
+ #### All Hyperparameters
585
+ <details><summary>Click to expand</summary>
586
+
587
+ - `overwrite_output_dir`: False
588
+ - `do_predict`: False
589
+ - `prediction_loss_only`: True
590
+ - `per_device_train_batch_size`: 256
591
+ - `per_device_eval_batch_size`: 256
592
+ - `per_gpu_train_batch_size`: None
593
+ - `per_gpu_eval_batch_size`: None
594
+ - `gradient_accumulation_steps`: 1
595
+ - `eval_accumulation_steps`: None
596
+ - `learning_rate`: 5e-05
597
+ - `weight_decay`: 0.0
598
+ - `adam_beta1`: 0.9
599
+ - `adam_beta2`: 0.999
600
+ - `adam_epsilon`: 1e-08
601
+ - `max_grad_norm`: 1
602
+ - `num_train_epochs`: 10
603
+ - `max_steps`: -1
604
+ - `lr_scheduler_type`: linear
605
+ - `lr_scheduler_kwargs`: {}
606
+ - `warmup_ratio`: 0.0
607
+ - `warmup_steps`: 0
608
+ - `log_level`: passive
609
+ - `log_level_replica`: warning
610
+ - `log_on_each_node`: True
611
+ - `logging_nan_inf_filter`: True
612
+ - `save_safetensors`: True
613
+ - `save_on_each_node`: False
614
+ - `save_only_model`: False
615
+ - `no_cuda`: False
616
+ - `use_cpu`: False
617
+ - `use_mps_device`: False
618
+ - `seed`: 42
619
+ - `data_seed`: None
620
+ - `jit_mode_eval`: False
621
+ - `use_ipex`: False
622
+ - `bf16`: False
623
+ - `fp16`: False
624
+ - `fp16_opt_level`: O1
625
+ - `half_precision_backend`: auto
626
+ - `bf16_full_eval`: False
627
+ - `fp16_full_eval`: False
628
+ - `tf32`: None
629
+ - `local_rank`: 0
630
+ - `ddp_backend`: None
631
+ - `tpu_num_cores`: None
632
+ - `tpu_metrics_debug`: False
633
+ - `debug`: []
634
+ - `dataloader_drop_last`: False
635
+ - `dataloader_num_workers`: 0
636
+ - `dataloader_prefetch_factor`: None
637
+ - `past_index`: -1
638
+ - `disable_tqdm`: False
639
+ - `remove_unused_columns`: True
640
+ - `label_names`: None
641
+ - `load_best_model_at_end`: False
642
+ - `ignore_data_skip`: False
643
+ - `fsdp`: []
644
+ - `fsdp_min_num_params`: 0
645
+ - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
646
+ - `fsdp_transformer_layer_cls_to_wrap`: None
647
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'gradient_accumulation_kwargs': None}
648
+ - `deepspeed`: None
649
+ - `label_smoothing_factor`: 0.0
650
+ - `optim`: adamw_torch
651
+ - `optim_args`: None
652
+ - `adafactor`: False
653
+ - `group_by_length`: False
654
+ - `length_column_name`: length
655
+ - `ddp_find_unused_parameters`: None
656
+ - `ddp_bucket_cap_mb`: None
657
+ - `ddp_broadcast_buffers`: False
658
+ - `dataloader_pin_memory`: True
659
+ - `dataloader_persistent_workers`: False
660
+ - `skip_memory_metrics`: True
661
+ - `use_legacy_prediction_loop`: False
662
+ - `push_to_hub`: False
663
+ - `resume_from_checkpoint`: None
664
+ - `hub_model_id`: None
665
+ - `hub_strategy`: every_save
666
+ - `hub_private_repo`: False
667
+ - `hub_always_push`: False
668
+ - `gradient_checkpointing`: False
669
+ - `gradient_checkpointing_kwargs`: None
670
+ - `include_inputs_for_metrics`: False
671
+ - `eval_do_concat_batches`: True
672
+ - `fp16_backend`: auto
673
+ - `push_to_hub_model_id`: None
674
+ - `push_to_hub_organization`: None
675
+ - `mp_parameters`:
676
+ - `auto_find_batch_size`: False
677
+ - `full_determinism`: False
678
+ - `torchdynamo`: None
679
+ - `ray_scope`: last
680
+ - `ddp_timeout`: 1800
681
+ - `torch_compile`: False
682
+ - `torch_compile_backend`: None
683
+ - `torch_compile_mode`: None
684
+ - `dispatch_batches`: None
685
+ - `split_batches`: None
686
+ - `include_tokens_per_second`: False
687
+ - `include_num_input_tokens_seen`: False
688
+ - `neftune_noise_alpha`: None
689
+ - `optim_target_modules`: None
690
+ - `batch_sampler`: batch_sampler
691
+ - `multi_dataset_batch_sampler`: round_robin
692
+
693
+ </details>
694
+
695
+ ### Training Logs
696
+ | Epoch | Step | Training Loss | sts-dev_spearman_cosine | sts-test_spearman_cosine |
697
+ |:------:|:----:|:-------------:|:-----------------------:|:------------------------:|
698
+ | 0.5650 | 500 | 10.9426 | - | - |
699
+ | 1.0 | 885 | - | 0.9202 | - |
700
+ | 1.1299 | 1000 | 9.7184 | - | - |
701
+ | 1.6949 | 1500 | 9.5348 | - | - |
702
+ | 2.0 | 1770 | - | 0.9400 | - |
703
+ | 2.2599 | 2000 | 9.4412 | - | - |
704
+ | 2.8249 | 2500 | 9.3097 | - | - |
705
+ | 3.0 | 2655 | - | 0.9489 | - |
706
+ | 3.3898 | 3000 | 9.2357 | - | - |
707
+ | 3.9548 | 3500 | 9.1594 | - | - |
708
+ | 4.0 | 3540 | - | 0.9528 | - |
709
+ | 4.5198 | 4000 | 9.0963 | - | - |
710
+ | 5.0 | 4425 | - | 0.9553 | - |
711
+ | 5.0847 | 4500 | 9.0382 | - | - |
712
+ | 5.6497 | 5000 | 8.9837 | - | - |
713
+ | 6.0 | 5310 | - | 0.9567 | - |
714
+ | 6.2147 | 5500 | 8.9403 | - | - |
715
+ | 6.7797 | 6000 | 8.8841 | - | - |
716
+ | 7.0 | 6195 | - | 0.9581 | - |
717
+ | 7.3446 | 6500 | 8.8513 | - | - |
718
+ | 7.9096 | 7000 | 8.81 | - | - |
719
+ | 8.0 | 7080 | - | 0.9582 | - |
720
+ | 8.4746 | 7500 | 8.8069 | - | - |
721
+ | 9.0 | 7965 | - | 0.9589 | - |
722
+ | 9.0395 | 8000 | 8.7616 | - | - |
723
+ | 9.6045 | 8500 | 8.7521 | - | - |
724
+ | 10.0 | 8850 | - | 0.9593 | 0.6266 |
725
+
726
+
727
+ ### Framework Versions
728
+ - Python: 3.9.7
729
+ - Sentence Transformers: 3.0.0
730
+ - Transformers: 4.40.1
731
+ - PyTorch: 2.3.0+cu121
732
+ - Accelerate: 0.29.3
733
+ - Datasets: 2.19.0
734
+ - Tokenizers: 0.19.1
735
+
736
+ ## Citation
737
+
738
+ ### BibTeX
739
+
740
+ #### Sentence Transformers
741
+ ```bibtex
742
+ @inproceedings{reimers-2019-sentence-bert,
743
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
744
+ author = "Reimers, Nils and Gurevych, Iryna",
745
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
746
+ month = "11",
747
+ year = "2019",
748
+ publisher = "Association for Computational Linguistics",
749
+ url = "https://arxiv.org/abs/1908.10084",
750
+ }
751
+ ```
752
+
753
+ #### AnglELoss
754
+ ```bibtex
755
+ @misc{li2023angleoptimized,
756
+ title={AnglE-optimized Text Embeddings},
757
+ author={Xianming Li and Jing Li},
758
+ year={2023},
759
+ eprint={2309.12871},
760
+ archivePrefix={arXiv},
761
+ primaryClass={cs.CL}
762
+ }
763
+ ```
764
+
765
+ <!--
766
+ ## Glossary
767
+
768
+ *Clearly define terms in order to be accessible across audiences.*
769
+ -->
770
+
771
+ <!--
772
+ ## Model Card Authors
773
+
774
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
775
+ -->
776
+
777
+ <!--
778
+ ## Model Card Contact
779
+
780
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
781
+ -->
sts-multilingual-mpnet-base-v2.Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:18d51123062246c211623dc192a22139b2b47c0a31c88544342a04040d87c69f
3
+ size 198577088
sts-multilingual-mpnet-base-v2.Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:11f24fe8f17d8aa13b38fdfe3cbe256a81a8386ee5c856e73b21587e7325de33
3
+ size 208677824
sts-multilingual-mpnet-base-v2.Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bc8f572368b5cbd3431211301c9f8f2d22d4ee118eea6e1adba5fce3f733d9f3
3
+ size 218732480
sts-multilingual-mpnet-base-v2.Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:74d720cd704b59dd8208b3aa72b32bf044f6a1af6aed468b62d72e5ae8c2f4c1
3
+ size 227137472
sts-multilingual-mpnet-base-v2.Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3e803d82b7da897feca2564d0c7479b5138fa0c48923b639100e5c3b6e4895cb
3
+ size 236067776
sts-multilingual-mpnet-base-v2.Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d14500c6efe5444f3c49872490073474d2c40e68d69f8b58071e4b1c14a748c8
3
+ size 303138240
sts-multilingual-mpnet-base-v2.bf16.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:24f861fafd5e8fd6a500d6650ff0c0c2800bbe6de9d62a972381ca58cd88a021
3
+ size 562765920