sentence-transformers pytorch gpt_neo feature-extraction sentence-similarity mteb arxiv:2202.08904 model-index autotrain_compatible endpoints_compatible region:us
---
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- feature-extraction
- sentence-similarity
- mteb
model-index:
- name: SGPT-2.7B-weightedmean-msmarco-specb-bitfit
  results:
  - task:
      type: Classification
    dataset:
      name: MTEB AmazonCounterfactualClassification (en)
      type: mteb/amazon_counterfactual
      config: en
      split: test
      revision: 2d8a100785abf0ae21420d2a55b0c56e3e1ea996
    metrics:
    - type: accuracy
      value: 67.56716417910448
    - type: ap
      value: 30.75574629595259
    - type: f1
      value: 61.805121301858655
  - task:
      type: Classification
    dataset:
      name: MTEB AmazonPolarityClassification
      type: mteb/amazon_polarity
      config: default
      split: test
      revision: 80714f8dcf8cefc218ef4f8c5a966dd83f75a0e1
    metrics:
    - type: accuracy
      value: 71.439575
    - type: ap
      value: 65.91341330532453
    - type: f1
      value: 70.90561852619555
  - task:
      type: Classification
    dataset:
      name: MTEB AmazonReviewsClassification (en)
      type: mteb/amazon_reviews_multi
      config: en
      split: test
      revision: c379a6705fec24a2493fa68e011692605f44e119
    metrics:
    - type: accuracy
      value: 35.748000000000005
    - type: f1
      value: 35.48576287186347
  - task:
      type: Retrieval
    dataset:
      name: MTEB ArguAna
      type: arguana
      config: default
      split: test
      revision: 5b3e3697907184a9b77a3c99ee9ea1a9cbb1e4e3
    metrics:
    - type: map_at_1
      value: 25.96
    - type: map_at_10
      value: 41.619
    - type: map_at_100
      value: 42.673
    - type: map_at_1000
      value: 42.684
    - type: map_at_3
      value: 36.569
    - type: map_at_5
      value: 39.397
    - type: mrr_at_1
      value: 26.316
    - type: mrr_at_10
      value: 41.772
    - type: mrr_at_100
      value: 42.82
    - type: mrr_at_1000
      value: 42.83
    - type: mrr_at_3
      value: 36.724000000000004
    - type: mrr_at_5
      value: 39.528999999999996
    - type: ndcg_at_1
      value: 25.96
    - type: ndcg_at_10
      value: 50.491
    - type: ndcg_at_100
      value: 54.864999999999995
    - type: ndcg_at_1000
      value: 55.10699999999999
    - type: ndcg_at_3
      value: 40.053
    - type: ndcg_at_5
      value: 45.134
    - type: precision_at_1
      value: 25.96
    - type: precision_at_10
      value: 7.8950000000000005
    - type: precision_at_100
      value: 0.9780000000000001
    - type: precision_at_1000
      value: 0.1
    - type: precision_at_3
      value: 16.714000000000002
    - type: precision_at_5
      value: 12.489
    - type: recall_at_1
      value: 25.96
    - type: recall_at_10
      value: 78.947
    - type: recall_at_100
      value: 97.795
    - type: recall_at_1000
      value: 99.644
    - type: recall_at_3
      value: 50.141999999999996
    - type: recall_at_5
      value: 62.446999999999996
  - task:
      type: Clustering
    dataset:
      name: MTEB ArxivClusteringP2P
      type: mteb/arxiv-clustering-p2p
      config: default
      split: test
      revision: 0bbdb47bcbe3a90093699aefeed338a0f28a7ee8
    metrics:
    - type: v_measure
      value: 44.72125714642202
  - task:
      type: Clustering
    dataset:
      name: MTEB ArxivClusteringS2S
      type: mteb/arxiv-clustering-s2s
      config: default
      split: test
      revision: b73bd54100e5abfa6e3a23dcafb46fe4d2438dc3
    metrics:
    - type: v_measure
      value: 35.081451519142064
  - task:
      type: Reranking
    dataset:
      name: MTEB AskUbuntuDupQuestions
      type: mteb/askubuntudupquestions-reranking
      config: default
      split: test
      revision: 4d853f94cd57d85ec13805aeeac3ae3e5eb4c49c
    metrics:
    - type: map
      value: 59.634661990392054
    - type: mrr
      value: 73.6813525040672
  - task:
      type: STS
    dataset:
      name: MTEB BIOSSES
      type: mteb/biosses-sts
      config: default
      split: test
      revision: 9ee918f184421b6bd48b78f6c714d86546106103
    metrics:
    - type: cos_sim_pearson
      value: 87.42754550496836
    - type: cos_sim_spearman
      value: 84.84289705838664
    - type: euclidean_pearson
      value: 85.59331970450859
    - type: euclidean_spearman
      value: 85.8525586184271
    - type: manhattan_pearson
      value: 85.41233134466698
    - type: manhattan_spearman
      value: 85.52303303767404
  - task:
      type: Classification
    dataset:
      name: MTEB Banking77Classification
      type: mteb/banking77
      config: default
      split: test
      revision: 44fa15921b4c889113cc5df03dd4901b49161ab7
    metrics:
    - type: accuracy
      value: 83.21753246753246
    - type: f1
      value: 83.15394543120915
  - task:
      type: Clustering
    dataset:
      name: MTEB BiorxivClusteringP2P
      type: mteb/biorxiv-clustering-p2p
      config: default
      split: test
      revision: 11d0121201d1f1f280e8cc8f3d98fb9c4d9f9c55
    metrics:
    - type: v_measure
      value: 34.41414219680629
  - task:
      type: Clustering
    dataset:
      name: MTEB BiorxivClusteringS2S
      type: mteb/biorxiv-clustering-s2s
      config: default
      split: test
      revision: c0fab014e1bcb8d3a5e31b2088972a1e01547dc1
    metrics:
    - type: v_measure
      value: 30.533275862270028
  - task:
      type: Retrieval
    dataset:
      name: MTEB CQADupstackAndroidRetrieval
      type: BeIR/cqadupstack
      config: default
      split: test
      revision: 2b9f5791698b5be7bc5e10535c8690f20043c3db
    metrics:
    - type: map_at_1
      value: 30.808999999999997
    - type: map_at_10
      value: 40.617
    - type: map_at_100
      value: 41.894999999999996
    - type: map_at_1000
      value: 42.025
    - type: map_at_3
      value: 37.0
    - type: map_at_5
      value: 38.993
    - type: mrr_at_1
      value: 37.482
    - type: mrr_at_10
      value: 46.497
    - type: mrr_at_100
      value: 47.144000000000005
    - type: mrr_at_1000
      value: 47.189
    - type: mrr_at_3
      value: 43.705
    - type: mrr_at_5
      value: 45.193
    - type: ndcg_at_1
      value: 37.482
    - type: ndcg_at_10
      value: 46.688
    - type: ndcg_at_100
      value: 51.726000000000006
    - type: ndcg_at_1000
      value: 53.825
    - type: ndcg_at_3
      value: 41.242000000000004
    - type: ndcg_at_5
      value: 43.657000000000004
    - type: precision_at_1
      value: 37.482
    - type: precision_at_10
      value: 8.827
    - type: precision_at_100
      value: 1.393
    - type: precision_at_1000
      value: 0.186
    - type: precision_at_3
      value: 19.361
    - type: precision_at_5
      value: 14.106
    - type: recall_at_1
      value: 30.808999999999997
    - type: recall_at_10
      value: 58.47
    - type: recall_at_100
      value: 80.51899999999999
    - type: recall_at_1000
      value: 93.809
    - type: recall_at_3
      value: 42.462
    - type: recall_at_5
      value: 49.385
    - type: map_at_1
      value: 26.962000000000003
    - type: map_at_10
      value: 36.93
    - type: map_at_100
      value: 38.102000000000004
    - type: map_at_1000
      value: 38.22
    - type: map_at_3
      value: 34.065
    - type: map_at_5
      value: 35.72
    - type: mrr_at_1
      value: 33.567
    - type: mrr_at_10
      value: 42.269
    - type: mrr_at_100
      value: 42.99
    - type: mrr_at_1000
      value: 43.033
    - type: mrr_at_3
      value: 40.064
    - type: mrr_at_5
      value: 41.258
    - type: ndcg_at_1
      value: 33.567
    - type: ndcg_at_10
      value: 42.405
    - type: ndcg_at_100
      value: 46.847
    - type: ndcg_at_1000
      value: 48.951
    - type: ndcg_at_3
      value: 38.312000000000005
    - type: ndcg_at_5
      value: 40.242
    - type: precision_at_1
      value: 33.567
    - type: precision_at_10
      value: 8.032
    - type: precision_at_100
      value: 1.295
    - type: precision_at_1000
      value: 0.17600000000000002
    - type: precision_at_3
      value: 18.662
    - type: precision_at_5
      value: 13.299
    - type: recall_at_1
      value: 26.962000000000003
    - type: recall_at_10
      value: 52.489
    - type: recall_at_100
      value: 71.635
    - type: recall_at_1000
      value: 85.141
    - type: recall_at_3
      value: 40.28
    - type: recall_at_5
      value: 45.757
    - type: map_at_1
      value: 36.318
    - type: map_at_10
      value: 47.97
    - type: map_at_100
      value: 49.003
    - type: map_at_1000
      value: 49.065999999999995
    - type: map_at_3
      value: 45.031
    - type: map_at_5
      value: 46.633
    - type: mrr_at_1
      value: 41.504999999999995
    - type: mrr_at_10
      value: 51.431000000000004
    - type: mrr_at_100
      value: 52.129000000000005
    - type: mrr_at_1000
      value: 52.161
    - type: mrr_at_3
      value: 48.934
    - type: mrr_at_5
      value: 50.42
    - type: ndcg_at_1
      value: 41.504999999999995
    - type: ndcg_at_10
      value: 53.676
    - type: ndcg_at_100
      value: 57.867000000000004
    - type: ndcg_at_1000
      value: 59.166
    - type: ndcg_at_3
      value: 48.516
    - type: ndcg_at_5
      value: 50.983999999999995
    - type: precision_at_1
      value: 41.504999999999995
    - type: precision_at_10
      value: 8.608
    - type: precision_at_100
      value: 1.1560000000000001
    - type: precision_at_1000
      value: 0.133
    - type: precision_at_3
      value: 21.462999999999997
    - type: precision_at_5
      value: 14.721
    - type: recall_at_1
      value: 36.318
    - type: recall_at_10
      value: 67.066
    - type: recall_at_100
      value: 85.34
    - type: recall_at_1000
      value: 94.491
    - type: recall_at_3
      value: 53.215999999999994
    - type: recall_at_5
      value: 59.214
    - type: map_at_1
      value: 22.167
    - type: map_at_10
      value: 29.543999999999997
    - type: map_at_100
      value: 30.579
    - type: map_at_1000
      value: 30.669999999999998
    - type: map_at_3
      value: 26.982
    - type: map_at_5
      value: 28.474
    - type: mrr_at_1
      value: 24.068
    - type: mrr_at_10
      value: 31.237
    - type: mrr_at_100
      value: 32.222
    - type: mrr_at_1000
      value: 32.292
    - type: mrr_at_3
      value: 28.776000000000003
    - type: mrr_at_5
      value: 30.233999999999998
    - type: ndcg_at_1
      value: 24.068
    - type: ndcg_at_10
      value: 33.973
    - type: ndcg_at_100
      value: 39.135
    - type: ndcg_at_1000
      value: 41.443999999999996
    - type: ndcg_at_3
      value: 29.018
    - type: ndcg_at_5
      value: 31.558999999999997
    - type: precision_at_1
      value: 24.068
    - type: precision_at_10
      value: 5.299
    - type: precision_at_100
      value: 0.823
    - type: precision_at_1000
      value: 0.106
    - type: precision_at_3
      value: 12.166
    - type: precision_at_5
      value: 8.767999999999999
    - type: recall_at_1
      value: 22.167
    - type: recall_at_10
      value: 46.115
    - type: recall_at_100
      value: 69.867
    - type: recall_at_1000
      value: 87.234
    - type: recall_at_3
      value: 32.798
    - type: recall_at_5
      value: 38.951
    - type: map_at_1
      value: 12.033000000000001
    - type: map_at_10
      value: 19.314
    - type: map_at_100
      value: 20.562
    - type: map_at_1000
      value: 20.695
    - type: map_at_3
      value: 16.946
    - type: map_at_5
      value: 18.076999999999998
    - type: mrr_at_1
      value: 14.801
    - type: mrr_at_10
      value: 22.74
    - type: mrr_at_100
      value: 23.876
    - type: mrr_at_1000
      value: 23.949
    - type: mrr_at_3
      value: 20.211000000000002
    - type: mrr_at_5
      value: 21.573
    - type: ndcg_at_1
      value: 14.801
    - type: ndcg_at_10
      value: 24.038
    - type: ndcg_at_100
      value: 30.186
    - type: ndcg_at_1000
      value: 33.321
    - type: ndcg_at_3
      value: 19.431
    - type: ndcg_at_5
      value: 21.34
    - type: precision_at_1
      value: 14.801
    - type: precision_at_10
      value: 4.776
    - type: precision_at_100
      value: 0.897
    - type: precision_at_1000
      value: 0.133
    - type: precision_at_3
      value: 9.66
    - type: precision_at_5
      value: 7.239
    - type: recall_at_1
      value: 12.033000000000001
    - type: recall_at_10
      value: 35.098
    - type: recall_at_100
      value: 62.175000000000004
    - type: recall_at_1000
      value: 84.17099999999999
    - type: recall_at_3
      value: 22.61
    - type: recall_at_5
      value: 27.278999999999996
    - type: map_at_1
      value: 26.651000000000003
    - type: map_at_10
      value: 36.901
    - type: map_at_100
      value: 38.249
    - type: map_at_1000
      value: 38.361000000000004
    - type: map_at_3
      value: 33.891
    - type: map_at_5
      value: 35.439
    - type: mrr_at_1
      value: 32.724
    - type: mrr_at_10
      value: 42.504
    - type: mrr_at_100
      value: 43.391999999999996
    - type: mrr_at_1000
      value: 43.436
    - type: mrr_at_3
      value: 39.989999999999995
    - type: mrr_at_5
      value: 41.347
    - type: ndcg_at_1
      value: 32.724
    - type: ndcg_at_10
      value: 43.007
    - type: ndcg_at_100
      value: 48.601
    - type: ndcg_at_1000
      value: 50.697
    - type: ndcg_at_3
      value: 37.99
    - type: ndcg_at_5
      value: 40.083999999999996
    - type: precision_at_1
      value: 32.724
    - type: precision_at_10
      value: 7.872999999999999
    - type: precision_at_100
      value: 1.247
    - type: precision_at_1000
      value: 0.16199999999999998
    - type: precision_at_3
      value: 18.062
    - type: precision_at_5
      value: 12.666
    - type: recall_at_1
      value: 26.651000000000003
    - type: recall_at_10
      value: 55.674
    - type: recall_at_100
      value: 78.904
    - type: recall_at_1000
      value: 92.55799999999999
    - type: recall_at_3
      value: 41.36
    - type: recall_at_5
      value: 46.983999999999995
    - type: map_at_1
      value: 22.589000000000002
    - type: map_at_10
      value: 32.244
    - type: map_at_100
      value: 33.46
    - type: map_at_1000
      value: 33.593
    - type: map_at_3
      value: 29.21
    - type: map_at_5
      value: 31.019999999999996
    - type: mrr_at_1
      value: 28.425
    - type: mrr_at_10
      value: 37.282
    - type: mrr_at_100
      value: 38.187
    - type: mrr_at_1000
      value: 38.248
    - type: mrr_at_3
      value: 34.684
    - type: mrr_at_5
      value: 36.123
    - type: ndcg_at_1
      value: 28.425
    - type: ndcg_at_10
      value: 37.942
    - type: ndcg_at_100
      value: 43.443
    - type: ndcg_at_1000
      value: 45.995999999999995
    - type: ndcg_at_3
      value: 32.873999999999995
    - type: ndcg_at_5
      value: 35.325
    - type: precision_at_1
      value: 28.425
    - type: precision_at_10
      value: 7.1
    - type: precision_at_100
      value: 1.166
    - type: precision_at_1000
      value: 0.158
    - type: precision_at_3
      value: 16.02
    - type: precision_at_5
      value: 11.644
    - type: recall_at_1
      value: 22.589000000000002
    - type: recall_at_10
      value: 50.03999999999999
    - type: recall_at_100
      value: 73.973
    - type: recall_at_1000
      value: 91.128
    - type: recall_at_3
      value: 35.882999999999996
    - type: recall_at_5
      value: 42.187999999999995
    - type: map_at_1
      value: 23.190833333333334
    - type: map_at_10
      value: 31.504916666666666
    - type: map_at_100
      value: 32.64908333333334
    - type: map_at_1000
      value: 32.77075
    - type: map_at_3
      value: 28.82575
    - type: map_at_5
      value: 30.2755
    - type: mrr_at_1
      value: 27.427499999999995
    - type: mrr_at_10
      value: 35.36483333333334
    - type: mrr_at_100
      value: 36.23441666666666
    - type: mrr_at_1000
      value: 36.297583333333336
    - type: mrr_at_3
      value: 32.97966666666667
    - type: mrr_at_5
      value: 34.294583333333335
    - type: ndcg_at_1
      value: 27.427499999999995
    - type: ndcg_at_10
      value: 36.53358333333333
    - type: ndcg_at_100
      value: 41.64508333333333
    - type: ndcg_at_1000
      value: 44.14499999999999
    - type: ndcg_at_3
      value: 31.88908333333333
    - type: ndcg_at_5
      value: 33.98433333333333
    - type: precision_at_1
      value: 27.427499999999995
    - type: precision_at_10
      value: 6.481083333333333
    - type: precision_at_100
      value: 1.0610833333333334
    - type: precision_at_1000
      value: 0.14691666666666667
    - type: precision_at_3
      value: 14.656749999999999
    - type: precision_at_5
      value: 10.493583333333332
    - type: recall_at_1
      value: 23.190833333333334
    - type: recall_at_10
      value: 47.65175
    - type: recall_at_100
      value: 70.41016666666667
    - type: recall_at_1000
      value: 87.82708333333332
    - type: recall_at_3
      value: 34.637583333333325
    - type: recall_at_5
      value: 40.05008333333333
    - type: map_at_1
      value: 20.409
    - type: map_at_10
      value: 26.794
    - type: map_at_100
      value: 27.682000000000002
    - type: map_at_1000
      value: 27.783
    - type: map_at_3
      value: 24.461
    - type: map_at_5
      value: 25.668000000000003
    - type: mrr_at_1
      value: 22.853
    - type: mrr_at_10
      value: 29.296
    - type: mrr_at_100
      value: 30.103
    - type: mrr_at_1000
      value: 30.179000000000002
    - type: mrr_at_3
      value: 27.173000000000002
    - type: mrr_at_5
      value: 28.223
    - type: ndcg_at_1
      value: 22.853
    - type: ndcg_at_10
      value: 31.007
    - type: ndcg_at_100
      value: 35.581
    - type: ndcg_at_1000
      value: 38.147
    - type: ndcg_at_3
      value: 26.590999999999998
    - type: ndcg_at_5
      value: 28.43
    - type: precision_at_1
      value: 22.853
    - type: precision_at_10
      value: 5.031
    - type: precision_at_100
      value: 0.7939999999999999
    - type: precision_at_1000
      value: 0.11
    - type: precision_at_3
      value: 11.401
    - type: precision_at_5
      value: 8.16
    - type: recall_at_1
      value: 20.409
    - type: recall_at_10
      value: 41.766
    - type: recall_at_100
      value: 62.964
    - type: recall_at_1000
      value: 81.682
    - type: recall_at_3
      value: 29.281000000000002
    - type: recall_at_5
      value: 33.83
    - type: map_at_1
      value: 14.549000000000001
    - type: map_at_10
      value: 20.315
    - type: map_at_100
      value: 21.301000000000002
    - type: map_at_1000
      value: 21.425
    - type: map_at_3
      value: 18.132
    - type: map_at_5
      value: 19.429
    - type: mrr_at_1
      value: 17.86
    - type: mrr_at_10
      value: 23.860999999999997
    - type: mrr_at_100
      value: 24.737000000000002
    - type: mrr_at_1000
      value: 24.82
    - type: mrr_at_3
      value: 21.685
    - type: mrr_at_5
      value: 23.008
    - type: ndcg_at_1
      value: 17.86
    - type: ndcg_at_10
      value: 24.396
    - type: ndcg_at_100
      value: 29.328
    - type: ndcg_at_1000
      value: 32.486
    - type: ndcg_at_3
      value: 20.375
    - type: ndcg_at_5
      value: 22.411
    - type: precision_at_1
      value: 17.86
    - type: precision_at_10
      value: 4.47
    - type: precision_at_100
      value: 0.8099999999999999
    - type: precision_at_1000
      value: 0.125
    - type: precision_at_3
      value: 9.475
    - type: precision_at_5
      value: 7.170999999999999
    - type: recall_at_1
      value: 14.549000000000001
    - type: recall_at_10
      value: 33.365
    - type: recall_at_100
      value: 55.797
    - type: recall_at_1000
      value: 78.632
    - type: recall_at_3
      value: 22.229
    - type: recall_at_5
      value: 27.339000000000002
    - type: map_at_1
      value: 23.286
    - type: map_at_10
      value: 30.728
    - type: map_at_100
      value: 31.840000000000003
    - type: map_at_1000
      value: 31.953
    - type: map_at_3
      value: 28.302
    - type: map_at_5
      value: 29.615000000000002
    - type: mrr_at_1
      value: 27.239
    - type: mrr_at_10
      value: 34.408
    - type: mrr_at_100
      value: 35.335
    - type: mrr_at_1000
      value: 35.405
    - type: mrr_at_3
      value: 32.151999999999994
    - type: mrr_at_5
      value: 33.355000000000004
    - type: ndcg_at_1
      value: 27.239
    - type: ndcg_at_10
      value: 35.324
    - type: ndcg_at_100
      value: 40.866
    - type: ndcg_at_1000
      value: 43.584
    - type: ndcg_at_3
      value: 30.898999999999997
    - type: ndcg_at_5
      value: 32.812999999999995
    - type: precision_at_1
      value: 27.239
    - type: precision_at_10
      value: 5.896
    - type: precision_at_100
      value: 0.979
    - type: precision_at_1000
      value: 0.133
    - type: precision_at_3
      value: 13.713000000000001
    - type: precision_at_5
      value: 9.683
    - type: recall_at_1
      value: 23.286
    - type: recall_at_10
      value: 45.711
    - type: recall_at_100
      value: 70.611
    - type: recall_at_1000
      value: 90.029
    - type: recall_at_3
      value: 33.615
    - type: recall_at_5
      value: 38.41
    - type: map_at_1
      value: 23.962
    - type: map_at_10
      value: 31.942999999999998
    - type: map_at_100
      value: 33.384
    - type: map_at_1000
      value: 33.611000000000004
    - type: map_at_3
      value: 29.243000000000002
    - type: map_at_5
      value: 30.446
    - type: mrr_at_1
      value: 28.458
    - type: mrr_at_10
      value: 36.157000000000004
    - type: mrr_at_100
      value: 37.092999999999996
    - type: mrr_at_1000
      value: 37.163000000000004
    - type: mrr_at_3
      value: 33.86
    - type: mrr_at_5
      value: 35.086
    - type: ndcg_at_1
      value: 28.458
    - type: ndcg_at_10
      value: 37.201
    - type: ndcg_at_100
      value: 42.591
    - type: ndcg_at_1000
      value: 45.539
    - type: ndcg_at_3
      value: 32.889
    - type: ndcg_at_5
      value: 34.483000000000004
    - type: precision_at_1
      value: 28.458
    - type: precision_at_10
      value: 7.332
    - type: precision_at_100
      value: 1.437
    - type: precision_at_1000
      value: 0.233
    - type: precision_at_3
      value: 15.547
    - type: precision_at_5
      value: 11.146
    - type: recall_at_1
      value: 23.962
    - type: recall_at_10
      value: 46.751
    - type: recall_at_100
      value: 71.626
    - type: recall_at_1000
      value: 90.93900000000001
    - type: recall_at_3
      value: 34.138000000000005
    - type: recall_at_5
      value: 38.673
    - type: map_at_1
      value: 18.555
    - type: map_at_10
      value: 24.759
    - type: map_at_100
      value: 25.732
    - type: map_at_1000
      value: 25.846999999999998
    - type: map_at_3
      value: 22.646
    - type: map_at_5
      value: 23.791999999999998
    - type: mrr_at_1
      value: 20.148
    - type: mrr_at_10
      value: 26.695999999999998
    - type: mrr_at_100
      value: 27.605
    - type: mrr_at_1000
      value: 27.695999999999998
    - type: mrr_at_3
      value: 24.522
    - type: mrr_at_5
      value: 25.715
    - type: ndcg_at_1
      value: 20.148
    - type: ndcg_at_10
      value: 28.746
    - type: ndcg_at_100
      value: 33.57
    - type: ndcg_at_1000
      value: 36.584
    - type: ndcg_at_3
      value: 24.532
    - type: ndcg_at_5
      value: 26.484
    - type: precision_at_1
      value: 20.148
    - type: precision_at_10
      value: 4.529
    - type: precision_at_100
      value: 0.736
    - type: precision_at_1000
      value: 0.108
    - type: precision_at_3
      value: 10.351
    - type: precision_at_5
      value: 7.32
    - type: recall_at_1
      value: 18.555
    - type: recall_at_10
      value: 39.275999999999996
    - type: recall_at_100
      value: 61.511
    - type: recall_at_1000
      value: 84.111
    - type: recall_at_3
      value: 27.778999999999996
    - type: recall_at_5
      value: 32.591
  - task:
      type: Retrieval
    dataset:
      name: MTEB ClimateFEVER
      type: climate-fever
      config: default
      split: test
      revision: 392b78eb68c07badcd7c2cd8f39af108375dfcce
    metrics:
    - type: map_at_1
      value: 10.366999999999999
    - type: map_at_10
      value: 18.953999999999997
    - type: map_at_100
      value: 20.674999999999997
    - type: map_at_1000
      value: 20.868000000000002
    - type: map_at_3
      value: 15.486
    - type: map_at_5
      value: 17.347
    - type: mrr_at_1
      value: 23.257
    - type: mrr_at_10
      value: 35.419
    - type: mrr_at_100
      value: 36.361
    - type: mrr_at_1000
      value: 36.403
    - type: mrr_at_3
      value: 31.747999999999998
    - type: mrr_at_5
      value: 34.077
    - type: ndcg_at_1
      value: 23.257
    - type: ndcg_at_10
      value: 27.11
    - type: ndcg_at_100
      value: 33.981
    - type: ndcg_at_1000
      value: 37.444
    - type: ndcg_at_3
      value: 21.471999999999998
    - type: ndcg_at_5
      value: 23.769000000000002
    - type: precision_at_1
      value: 23.257
    - type: precision_at_10
      value: 8.704
    - type: precision_at_100
      value: 1.606
    - type: precision_at_1000
      value: 0.22499999999999998
    - type: precision_at_3
      value: 16.287
    - type: precision_at_5
      value: 13.068
    - type: recall_at_1
      value: 10.366999999999999
    - type: recall_at_10
      value: 33.706
    - type: recall_at_100
      value: 57.375
    - type: recall_at_1000
      value: 76.79
    - type: recall_at_3
      value: 20.18
    - type: recall_at_5
      value: 26.215
  - task:
      type: Retrieval
    dataset:
      name: MTEB DBPedia
      type: dbpedia-entity
      config: default
      split: test
      revision: f097057d03ed98220bc7309ddb10b71a54d667d6
    metrics:
    - type: map_at_1
      value: 8.246
    - type: map_at_10
      value: 15.979
    - type: map_at_100
      value: 21.025
    - type: map_at_1000
      value: 22.189999999999998
    - type: map_at_3
      value: 11.997
    - type: map_at_5
      value: 13.697000000000001
    - type: mrr_at_1
      value: 60.75000000000001
    - type: mrr_at_10
      value: 68.70100000000001
    - type: mrr_at_100
      value: 69.1
    - type: mrr_at_1000
      value: 69.111
    - type: mrr_at_3
      value: 66.583
    - type: mrr_at_5
      value: 67.87100000000001
    - type: ndcg_at_1
      value: 49.75
    - type: ndcg_at_10
      value: 34.702
    - type: ndcg_at_100
      value: 37.607
    - type: ndcg_at_1000
      value: 44.322
    - type: ndcg_at_3
      value: 39.555
    - type: ndcg_at_5
      value: 36.684
    - type: precision_at_1
      value: 60.75000000000001
    - type: precision_at_10
      value: 26.625
    - type: precision_at_100
      value: 7.969999999999999
    - type: precision_at_1000
      value: 1.678
    - type: precision_at_3
      value: 41.833
    - type: precision_at_5
      value: 34.5
    - type: recall_at_1
      value: 8.246
    - type: recall_at_10
      value: 20.968
    - type: recall_at_100
      value: 42.065000000000005
    - type: recall_at_1000
      value: 63.671
    - type: recall_at_3
      value: 13.039000000000001
    - type: recall_at_5
      value: 16.042
  - task:
      type: Classification
    dataset:
      name: MTEB EmotionClassification
      type: mteb/emotion
      config: default
      split: test
      revision: 829147f8f75a25f005913200eb5ed41fae320aa1
    metrics:
    - type: accuracy
      value: 49.214999999999996
    - type: f1
      value: 44.85952451163755
  - task:
      type: Retrieval
    dataset:
      name: MTEB FEVER
      type: fever
      config: default
      split: test
      revision: 1429cf27e393599b8b359b9b72c666f96b2525f9
    metrics:
    - type: map_at_1
      value: 56.769000000000005
    - type: map_at_10
      value: 67.30199999999999
    - type: map_at_100
      value: 67.692
    - type: map_at_1000
      value: 67.712
    - type: map_at_3
      value: 65.346
    - type: map_at_5
      value: 66.574
    - type: mrr_at_1
      value: 61.370999999999995
    - type: mrr_at_10
      value: 71.875
    - type: mrr_at_100
      value: 72.195
    - type: mrr_at_1000
      value: 72.206
    - type: mrr_at_3
      value: 70.04
    - type: mrr_at_5
      value: 71.224
    - type: ndcg_at_1
      value: 61.370999999999995
    - type: ndcg_at_10
      value: 72.731
    - type: ndcg_at_100
      value: 74.468
    - type: ndcg_at_1000
      value: 74.91600000000001
    - type: ndcg_at_3
      value: 69.077
    - type: ndcg_at_5
      value: 71.111
    - type: precision_at_1
      value: 61.370999999999995
    - type: precision_at_10
      value: 9.325999999999999
    - type: precision_at_100
      value: 1.03
    - type: precision_at_1000
      value: 0.108
    - type: precision_at_3
      value: 27.303
    - type: precision_at_5
      value: 17.525
    - type: recall_at_1
      value: 56.769000000000005
    - type: recall_at_10
      value: 85.06
    - type: recall_at_100
      value: 92.767
    - type: recall_at_1000
      value: 95.933
    - type: recall_at_3
      value: 75.131
    - type: recall_at_5
      value: 80.17
  - task:
      type: Retrieval
    dataset:
      name: MTEB FiQA2018
      type: fiqa
      config: default
      split: test
      revision: 41b686a7f28c59bcaaa5791efd47c67c8ebe28be
    metrics:
    - type: map_at_1
      value: 15.753
    - type: map_at_10
      value: 25.875999999999998
    - type: map_at_100
      value: 27.415
    - type: map_at_1000
      value: 27.590999999999998
    - type: map_at_3
      value: 22.17
    - type: map_at_5
      value: 24.236
    - type: mrr_at_1
      value: 31.019000000000002
    - type: mrr_at_10
      value: 39.977000000000004
    - type: mrr_at_100
      value: 40.788999999999994
    - type: mrr_at_1000
      value: 40.832
    - type: mrr_at_3
      value: 37.088
    - type: mrr_at_5
      value: 38.655
    - type: ndcg_at_1
      value: 31.019000000000002
    - type: ndcg_at_10
      value: 33.286
    - type: ndcg_at_100
      value: 39.528999999999996
    - type: ndcg_at_1000
      value: 42.934
    - type: ndcg_at_3
      value: 29.29
    - type: ndcg_at_5
      value: 30.615
    - type: precision_at_1
      value: 31.019000000000002
    - type: precision_at_10
      value: 9.383
    - type: precision_at_100
      value: 1.6019999999999999
    - type: precision_at_1000
      value: 0.22200000000000003
    - type: precision_at_3
      value: 19.753
    - type: precision_at_5
      value: 14.815000000000001
    - type: recall_at_1
      value: 15.753
    - type: recall_at_10
      value: 40.896
    - type: recall_at_100
      value: 64.443
    - type: recall_at_1000
      value: 85.218
    - type: recall_at_3
      value: 26.526
    - type: recall_at_5
      value: 32.452999999999996
  - task:
      type: Retrieval
    dataset:
      name: MTEB HotpotQA
      type: hotpotqa
      config: default
      split: test
      revision: 766870b35a1b9ca65e67a0d1913899973551fc6c
    metrics:
    - type: map_at_1
      value: 32.153999999999996
    - type: map_at_10
      value: 43.651
    - type: map_at_100
      value: 44.41
    - type: map_at_1000
      value: 44.487
    - type: map_at_3
      value: 41.239
    - type: map_at_5
      value: 42.659000000000006
    - type: mrr_at_1
      value: 64.30799999999999
    - type: mrr_at_10
      value: 71.22500000000001
    - type: mrr_at_100
      value: 71.57
    - type: mrr_at_1000
      value: 71.59100000000001
    - type: mrr_at_3
      value: 69.95
    - type: mrr_at_5
      value: 70.738
    - type: ndcg_at_1
      value: 64.30799999999999
    - type: ndcg_at_10
      value: 52.835
    - type: ndcg_at_100
      value: 55.840999999999994
    - type: ndcg_at_1000
      value: 57.484
    - type: ndcg_at_3
      value: 49.014
    - type: ndcg_at_5
      value: 51.01599999999999
    - type: precision_at_1
      value: 64.30799999999999
    - type: precision_at_10
      value: 10.77
    - type: precision_at_100
      value: 1.315
    - type: precision_at_1000
      value: 0.153
    - type: precision_at_3
      value: 30.223
    - type: precision_at_5
      value: 19.716
    - type: recall_at_1
      value: 32.153999999999996
    - type: recall_at_10
      value: 53.849000000000004
    - type: recall_at_100
      value: 65.75999999999999
    - type: recall_at_1000
      value: 76.705
    - type: recall_at_3
      value: 45.334
    - type: recall_at_5
      value: 49.291000000000004
  - task:
      type: Classification
    dataset:
      name: MTEB ImdbClassification
      type: mteb/imdb
      config: default
      split: test
      revision: 8d743909f834c38949e8323a8a6ce8721ea6c7f4
    metrics:
    - type: accuracy
      value: 63.5316
    - type: ap
      value: 58.90084300359825
    - type: f1
      value: 63.35727889030892
  - task:
      type: Retrieval
    dataset:
      name: MTEB MSMARCO
      type: msmarco
      config: default
      split: validation
      revision: e6838a846e2408f22cf5cc337ebc83e0bcf77849
    metrics:
    - type: map_at_1
      value: 20.566000000000003
    - type: map_at_10
      value: 32.229
    - type: map_at_100
      value: 33.445
    - type: map_at_1000
      value: 33.501
    - type: map_at_3
      value: 28.504
    - type: map_at_5
      value: 30.681000000000004
    - type: mrr_at_1
      value: 21.218
    - type: mrr_at_10
      value: 32.816
    - type: mrr_at_100
      value: 33.986
    - type: mrr_at_1000
      value: 34.035
    - type: mrr_at_3
      value: 29.15
    - type: mrr_at_5
      value: 31.290000000000003
    - type: ndcg_at_1
      value: 21.218
    - type: ndcg_at_10
      value: 38.832
    - type: ndcg_at_100
      value: 44.743
    - type: ndcg_at_1000
      value: 46.138
    - type: ndcg_at_3
      value: 31.232
    - type: ndcg_at_5
      value: 35.099999999999994
    - type: precision_at_1
      value: 21.218
    - type: precision_at_10
      value: 6.186
    - type: precision_at_100
      value: 0.914
    - type: precision_at_1000
      value: 0.10300000000000001
    - type: precision_at_3
      value: 13.314
    - type: precision_at_5
      value: 9.943
    - type: recall_at_1
      value: 20.566000000000003
    - type: recall_at_10
      value: 59.192
    - type: recall_at_100
      value: 86.626
    - type: recall_at_1000
      value: 97.283
    - type: recall_at_3
      value: 38.492
    - type: recall_at_5
      value: 47.760000000000005
  - task:
      type: Classification
    dataset:
      name: MTEB MTOPDomainClassification (en)
      type: mteb/mtop_domain
      config: en
      split: test
      revision: a7e2a951126a26fc8c6a69f835f33a346ba259e3
    metrics:
    - type: accuracy
      value: 92.56269949840402
    - type: f1
      value: 92.1020975473988
  - task:
      type: Classification
    dataset:
      name: MTEB MTOPIntentClassification (en)
      type: mteb/mtop_intent
      config: en
      split: test
      revision: 6299947a7777084cc2d4b64235bf7190381ce755
    metrics:
    - type: accuracy
      value: 71.8467852257182
    - type: f1
      value: 53.652719348592015
  - task:
      type: Classification
    dataset:
      name: MTEB MassiveIntentClassification (en)
      type: mteb/amazon_massive_intent
      config: en
      split: test
      revision: 072a486a144adf7f4479a4a0dddb2152e161e1ea
    metrics:
    - type: accuracy
      value: 69.00806993947546
    - type: f1
      value: 67.41429618885515
  - task:
      type: Classification
    dataset:
      name: MTEB MassiveScenarioClassification (en)
      type: mteb/amazon_massive_scenario
      config: en
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 75.90114324142569
    - type: f1
      value: 76.25183590651454
  - task:
      type: Clustering
    dataset:
      name: MTEB MedrxivClusteringP2P
      type: mteb/medrxiv-clustering-p2p
      config: default
      split: test
      revision: dcefc037ef84348e49b0d29109e891c01067226b
    metrics:
    - type: v_measure
      value: 31.350109978273395
  - task:
      type: Clustering
    dataset:
      name: MTEB MedrxivClusteringS2S
      type: mteb/medrxiv-clustering-s2s
      config: default
      split: test
      revision: 3cd0e71dfbe09d4de0f9e5ecba43e7ce280959dc
    metrics:
    - type: v_measure
      value: 28.768923695767327
  - task:
      type: Reranking
    dataset:
      name: MTEB MindSmallReranking
      type: mteb/mind_small
      config: default
      split: test
      revision: 3bdac13927fdc888b903db93b2ffdbd90b295a69
    metrics:
    - type: map
      value: 31.716396735210754
    - type: mrr
      value: 32.88970538547634
  - task:
      type: Retrieval
    dataset:
      name: MTEB NFCorpus
      type: nfcorpus
      config: default
      split: test
      revision: 7eb63cc0c1eb59324d709ebed25fcab851fa7610
    metrics:
    - type: map_at_1
      value: 5.604
    - type: map_at_10
      value: 12.379999999999999
    - type: map_at_100
      value: 15.791
    - type: map_at_1000
      value: 17.327
    - type: map_at_3
      value: 9.15
    - type: map_at_5
      value: 10.599
    - type: mrr_at_1
      value: 45.201
    - type: mrr_at_10
      value: 53.374
    - type: mrr_at_100
      value: 54.089
    - type: mrr_at_1000
      value: 54.123
    - type: mrr_at_3
      value: 51.44499999999999
    - type: mrr_at_5
      value: 52.59
    - type: ndcg_at_1
      value: 42.879
    - type: ndcg_at_10
      value: 33.891
    - type: ndcg_at_100
      value: 31.391999999999996
    - type: ndcg_at_1000
      value: 40.36
    - type: ndcg_at_3
      value: 39.076
    - type: ndcg_at_5
      value: 37.047000000000004
    - type: precision_at_1
      value: 44.582
    - type: precision_at_10
      value: 25.294
    - type: precision_at_100
      value: 8.285
    - type: precision_at_1000
      value: 2.1479999999999997
    - type: precision_at_3
      value: 36.120000000000005
    - type: precision_at_5
      value: 31.95
    - type: recall_at_1
      value: 5.604
    - type: recall_at_10
      value: 16.239
    - type: recall_at_100
      value: 32.16
    - type: recall_at_1000
      value: 64.513
    - type: recall_at_3
      value: 10.406
    - type: recall_at_5
      value: 12.684999999999999
  - task:
      type: Retrieval
    dataset:
      name: MTEB NQ
      type: nq
      config: default
      split: test
      revision: 6062aefc120bfe8ece5897809fb2e53bfe0d128c
    metrics:
    - type: map_at_1
      value: 25.881
    - type: map_at_10
      value: 39.501
    - type: map_at_100
      value: 40.615
    - type: map_at_1000
      value: 40.661
    - type: map_at_3
      value: 35.559000000000005
    - type: map_at_5
      value: 37.773
    - type: mrr_at_1
      value: 29.229
    - type: mrr_at_10
      value: 41.955999999999996
    - type: mrr_at_100
      value: 42.86
    - type: mrr_at_1000
      value: 42.893
    - type: mrr_at_3
      value: 38.562000000000005
    - type: mrr_at_5
      value: 40.542
    - type: ndcg_at_1
      value: 29.2
    - type: ndcg_at_10
      value: 46.703
    - type: ndcg_at_100
      value: 51.644
    - type: ndcg_at_1000
      value: 52.771
    - type: ndcg_at_3
      value: 39.141999999999996
    - type: ndcg_at_5
      value: 42.892
    - type: precision_at_1
      value: 29.2
    - type: precision_at_10
      value: 7.920000000000001
    - type: precision_at_100
      value: 1.0659999999999998
    - type: precision_at_1000
      value: 0.117
    - type: precision_at_3
      value: 18.105
    - type: precision_at_5
      value: 13.036
    - type: recall_at_1
      value: 25.881
    - type: recall_at_10
      value: 66.266
    - type: recall_at_100
      value: 88.116
    - type: recall_at_1000
      value: 96.58200000000001
    - type: recall_at_3
      value: 46.526
    - type: recall_at_5
      value: 55.154
  - task:
      type: Retrieval
    dataset:
      name: MTEB QuoraRetrieval
      type: quora
      config: default
      split: test
      revision: 6205996560df11e3a3da9ab4f926788fc30a7db4
    metrics:
    - type: map_at_1
      value: 67.553
    - type: map_at_10
      value: 81.34
    - type: map_at_100
      value: 82.002
    - type: map_at_1000
      value: 82.027
    - type: map_at_3
      value: 78.281
    - type: map_at_5
      value: 80.149
    - type: mrr_at_1
      value: 77.72
    - type: mrr_at_10
      value: 84.733
    - type: mrr_at_100
      value: 84.878
    - type: mrr_at_1000
      value: 84.879
    - type: mrr_at_3
      value: 83.587
    - type: mrr_at_5
      value: 84.32600000000001
    - type: ndcg_at_1
      value: 77.75
    - type: ndcg_at_10
      value: 85.603
    - type: ndcg_at_100
      value: 87.069
    - type: ndcg_at_1000
      value: 87.25
    - type: ndcg_at_3
      value: 82.303
    - type: ndcg_at_5
      value: 84.03699999999999
    - type: precision_at_1
      value: 77.75
    - type: precision_at_10
      value: 13.04
    - type: precision_at_100
      value: 1.5070000000000001
    - type: precision_at_1000
      value: 0.156
    - type: precision_at_3
      value: 35.903
    - type: precision_at_5
      value: 23.738
    - type: recall_at_1
      value: 67.553
    - type: recall_at_10
      value: 93.903
    - type: recall_at_100
      value: 99.062
    - type: recall_at_1000
      value: 99.935
    - type: recall_at_3
      value: 84.58099999999999
    - type: recall_at_5
      value: 89.316
  - task:
      type: Clustering
    dataset:
      name: MTEB RedditClustering
      type: mteb/reddit-clustering
      config: default
      split: test
      revision: b2805658ae38990172679479369a78b86de8c390
    metrics:
    - type: v_measure
      value: 46.46887711230235
  - task:
      type: Clustering
    dataset:
      name: MTEB RedditClusteringP2P
      type: mteb/reddit-clustering-p2p
      config: default
      split: test
      revision: 385e3cb46b4cfa89021f56c4380204149d0efe33
    metrics:
    - type: v_measure
      value: 54.166876298246926
  - task:
      type: Retrieval
    dataset:
      name: MTEB SCIDOCS
      type: scidocs
      config: default
      split: test
      revision: 5c59ef3e437a0a9651c8fe6fde943e7dce59fba5
    metrics:
    - type: map_at_1
      value: 4.053
    - type: map_at_10
      value: 9.693999999999999
    - type: map_at_100
      value: 11.387
    - type: map_at_1000
      value: 11.654
    - type: map_at_3
      value: 7.053
    - type: map_at_5
      value: 8.439
    - type: mrr_at_1
      value: 19.900000000000002
    - type: mrr_at_10
      value: 29.359
    - type: mrr_at_100
      value: 30.484
    - type: mrr_at_1000
      value: 30.553
    - type: mrr_at_3
      value: 26.200000000000003
    - type: mrr_at_5
      value: 28.115000000000002
    - type: ndcg_at_1
      value: 19.900000000000002
    - type: ndcg_at_10
      value: 16.575
    - type: ndcg_at_100
      value: 23.655
    - type: ndcg_at_1000
      value: 28.853
    - type: ndcg_at_3
      value: 15.848
    - type: ndcg_at_5
      value: 14.026
    - type: precision_at_1
      value: 19.900000000000002
    - type: precision_at_10
      value: 8.450000000000001
    - type: precision_at_100
      value: 1.872
    - type: precision_at_1000
      value: 0.313
    - type: precision_at_3
      value: 14.667
    - type: precision_at_5
      value: 12.32
    - type: recall_at_1
      value: 4.053
    - type: recall_at_10
      value: 17.169999999999998
    - type: recall_at_100
      value: 38.025
    - type: recall_at_1000
      value: 63.571999999999996
    - type: recall_at_3
      value: 8.903
    - type: recall_at_5
      value: 12.477
  - task:
      type: STS
    dataset:
      name: MTEB SICK-R
      type: mteb/sickr-sts
      config: default
      split: test
      revision: 20a6d6f312dd54037fe07a32d58e5e168867909d
    metrics:
    - type: cos_sim_pearson
      value: 77.7548748519677
    - type: cos_sim_spearman
      value: 68.19926431966059
    - type: euclidean_pearson
      value: 71.69016204991725
    - type: euclidean_spearman
      value: 66.98099673026834
    - type: manhattan_pearson
      value: 71.62994072488664
    - type: manhattan_spearman
      value: 67.03435950744577
  - task:
      type: STS
    dataset:
      name: MTEB STS12
      type: mteb/sts12-sts
      config: default
      split: test
      revision: fdf84275bb8ce4b49c971d02e84dd1abc677a50f
    metrics:
    - type: cos_sim_pearson
      value: 75.91051402657887
    - type: cos_sim_spearman
      value: 66.99390786191645
    - type: euclidean_pearson
      value: 71.54128036454578
    - type: euclidean_spearman
      value: 69.25605675649068
    - type: manhattan_pearson
      value: 71.60981030780171
    - type: manhattan_spearman
      value: 69.27513670128046
  - task:
      type: STS
    dataset:
      name: MTEB STS13
      type: mteb/sts13-sts
      config: default
      split: test
      revision: 1591bfcbe8c69d4bf7fe2a16e2451017832cafb9
    metrics:
    - type: cos_sim_pearson
      value: 77.23835466417793
    - type: cos_sim_spearman
      value: 77.57623085766706
    - type: euclidean_pearson
      value: 77.5090992200725
    - type: euclidean_spearman
      value: 77.88601688144924
    - type: manhattan_pearson
      value: 77.39045060647423
    - type: manhattan_spearman
      value: 77.77552718279098
  - task:
      type: STS
    dataset:
      name: MTEB STS14
      type: mteb/sts14-sts
      config: default
      split: test
      revision: e2125984e7df8b7871f6ae9949cf6b6795e7c54b
    metrics:
    - type: cos_sim_pearson
      value: 77.91692485139602
    - type: cos_sim_spearman
      value: 72.78258293483495
    - type: euclidean_pearson
      value: 74.64773017077789
    - type: euclidean_spearman
      value: 71.81662299104619
    - type: manhattan_pearson
      value: 74.71043337995533
    - type: manhattan_spearman
      value: 71.83960860845646
  - task:
      type: STS
    dataset:
      name: MTEB STS15
      type: mteb/sts15-sts
      config: default
      split: test
      revision: 1cd7298cac12a96a373b6a2f18738bb3e739a9b6
    metrics:
    - type: cos_sim_pearson
      value: 82.13422113617578
    - type: cos_sim_spearman
      value: 82.61707296911949
    - type: euclidean_pearson
      value: 81.42487480400861
    - type: euclidean_spearman
      value: 82.17970991273835
    - type: manhattan_pearson
      value: 81.41985055477845
    - type: manhattan_spearman
      value: 82.15823204362937
  - task:
      type: STS
    dataset:
      name: MTEB STS16
      type: mteb/sts16-sts
      config: default
      split: test
      revision: 360a0b2dff98700d09e634a01e1cc1624d3e42cd
    metrics:
    - type: cos_sim_pearson
      value: 79.07989542843826
    - type: cos_sim_spearman
      value: 80.09839524406284
    - type: euclidean_pearson
      value: 76.43186028364195
    - type: euclidean_spearman
      value: 76.76720323266471
    - type: manhattan_pearson
      value: 76.4674747409161
    - type: manhattan_spearman
      value: 76.81797407068667
  - task:
      type: STS
    dataset:
      name: MTEB STS17 (en-en)
      type: mteb/sts17-crosslingual-sts
      config: en-en
      split: test
      revision: 9fc37e8c632af1c87a3d23e685d49552a02582a0
    metrics:
    - type: cos_sim_pearson
      value: 87.0420983224933
    - type: cos_sim_spearman
      value: 87.25017540413702
    - type: euclidean_pearson
      value: 84.56384596473421
    - type: euclidean_spearman
      value: 84.72557417564886
    - type: manhattan_pearson
      value: 84.7329954474549
    - type: manhattan_spearman
      value: 84.75071371008909
  - task:
      type: STS
    dataset:
      name: MTEB STS22 (en)
      type: mteb/sts22-crosslingual-sts
      config: en
      split: test
      revision: 2de6ce8c1921b71a755b262c6b57fef195dd7906
    metrics:
    - type: cos_sim_pearson
      value: 68.47031320016424
    - type: cos_sim_spearman
      value: 68.7486910762485
    - type: euclidean_pearson
      value: 71.30330985913915
    - type: euclidean_spearman
      value: 71.59666258520735
    - type: manhattan_pearson
      value: 71.4423884279027
    - type: manhattan_spearman
      value: 71.67460706861044
  - task:
      type: STS
    dataset:
      name: MTEB STSBenchmark
      type: mteb/stsbenchmark-sts
      config: default
      split: test
      revision: 8913289635987208e6e7c72789e4be2fe94b6abd
    metrics:
    - type: cos_sim_pearson
      value: 80.79514366062675
    - type: cos_sim_spearman
      value: 79.20585637461048
    - type: euclidean_pearson
      value: 78.6591557395699
    - type: euclidean_spearman
      value: 77.86455794285718
    - type: manhattan_pearson
      value: 78.67754806486865
    - type: manhattan_spearman
      value: 77.88178687200732
  - task:
      type: Reranking
    dataset:
      name: MTEB SciDocsRR
      type: mteb/scidocs-reranking
      config: default
      split: test
      revision: 56a6d0140cf6356659e2a7c1413286a774468d44
    metrics:
    - type: map
      value: 77.71580844366375
    - type: mrr
      value: 93.04215845882513
  - task:
      type: Retrieval
    dataset:
      name: MTEB SciFact
      type: scifact
      config: default
      split: test
      revision: a75ae049398addde9b70f6b268875f5cbce99089
    metrics:
    - type: map_at_1
      value: 56.39999999999999
    - type: map_at_10
      value: 65.701
    - type: map_at_100
      value: 66.32000000000001
    - type: map_at_1000
      value: 66.34100000000001
    - type: map_at_3
      value: 62.641999999999996
    - type: map_at_5
      value: 64.342
    - type: mrr_at_1
      value: 58.667
    - type: mrr_at_10
      value: 66.45299999999999
    - type: mrr_at_100
      value: 66.967
    - type: mrr_at_1000
      value: 66.988
    - type: mrr_at_3
      value: 64.11099999999999
    - type: mrr_at_5
      value: 65.411
    - type: ndcg_at_1
      value: 58.667
    - type: ndcg_at_10
      value: 70.165
    - type: ndcg_at_100
      value: 72.938
    - type: ndcg_at_1000
      value: 73.456
    - type: ndcg_at_3
      value: 64.79
    - type: ndcg_at_5
      value: 67.28
    - type: precision_at_1
      value: 58.667
    - type: precision_at_10
      value: 9.4
    - type: precision_at_100
      value: 1.087
    - type: precision_at_1000
      value: 0.11299999999999999
    - type: precision_at_3
      value: 24.889
    - type: precision_at_5
      value: 16.667
    - type: recall_at_1
      value: 56.39999999999999
    - type: recall_at_10
      value: 83.122
    - type: recall_at_100
      value: 95.667
    - type: recall_at_1000
      value: 99.667
    - type: recall_at_3
      value: 68.378
    - type: recall_at_5
      value: 74.68299999999999
  - task:
      type: PairClassification
    dataset:
      name: MTEB SprintDuplicateQuestions
      type: mteb/sprintduplicatequestions-pairclassification
      config: default
      split: test
      revision: 5a8256d0dff9c4bd3be3ba3e67e4e70173f802ea
    metrics:
    - type: cos_sim_accuracy
      value: 99.76831683168317
    - type: cos_sim_ap
      value: 93.47124923047998
    - type: cos_sim_f1
      value: 88.06122448979592
    - type: cos_sim_precision
      value: 89.89583333333333
    - type: cos_sim_recall
      value: 86.3
    - type: dot_accuracy
      value: 99.57326732673268
    - type: dot_ap
      value: 84.06577868167207
    - type: dot_f1
      value: 77.82629791363416
    - type: dot_precision
      value: 75.58906691800189
    - type: dot_recall
      value: 80.2
    - type: euclidean_accuracy
      value: 99.74257425742574
    - type: euclidean_ap
      value: 92.1904681653555
    - type: euclidean_f1
      value: 86.74821610601427
    - type: euclidean_precision
      value: 88.46153846153845
    - type: euclidean_recall
      value: 85.1
    - type: manhattan_accuracy
      value: 99.74554455445545
    - type: manhattan_ap
      value: 92.4337790809948
    - type: manhattan_f1
      value: 86.86765457332653
    - type: manhattan_precision
      value: 88.81922675026124
    - type: manhattan_recall
      value: 85.0
    - type: max_accuracy
      value: 99.76831683168317
    - type: max_ap
      value: 93.47124923047998
    - type: max_f1
      value: 88.06122448979592
  - task:
      type: Clustering
    dataset:
      name: MTEB StackExchangeClustering
      type: mteb/stackexchange-clustering
      config: default
      split: test
      revision: 70a89468f6dccacc6aa2b12a6eac54e74328f235
    metrics:
    - type: v_measure
      value: 59.194098673976484
  - task:
      type: Clustering
    dataset:
      name: MTEB StackExchangeClusteringP2P
      type: mteb/stackexchange-clustering-p2p
      config: default
      split: test
      revision: d88009ab563dd0b16cfaf4436abaf97fa3550cf0
    metrics:
    - type: v_measure
      value: 32.5744032578115
  - task:
      type: Reranking
    dataset:
      name: MTEB StackOverflowDupQuestions
      type: mteb/stackoverflowdupquestions-reranking
      config: default
      split: test
      revision: ef807ea29a75ec4f91b50fd4191cb4ee4589a9f9
    metrics:
    - type: map
      value: 49.61186384154483
    - type: mrr
      value: 50.55424253034547
  - task:
      type: Summarization
    dataset:
      name: MTEB SummEval
      type: mteb/summeval
      config: default
      split: test
      revision: 8753c2788d36c01fc6f05d03fe3f7268d63f9122
    metrics:
    - type: cos_sim_pearson
      value: 30.027210161713946
    - type: cos_sim_spearman
      value: 31.030178065751734
    - type: dot_pearson
      value: 30.09179785685587
    - type: dot_spearman
      value: 30.408303252207812
  - task:
      type: Retrieval
    dataset:
      name: MTEB TRECCOVID
      type: trec-covid
      config: default
      split: test
      revision: 2c8041b2c07a79b6f7ba8fe6acc72e5d9f92d217
    metrics:
    - type: map_at_1
      value: 0.22300000000000003
    - type: map_at_10
      value: 1.762
    - type: map_at_100
      value: 9.984
    - type: map_at_1000
      value: 24.265
    - type: map_at_3
      value: 0.631
    - type: map_at_5
      value: 0.9950000000000001
    - type: mrr_at_1
      value: 88.0
    - type: mrr_at_10
      value: 92.833
    - type: mrr_at_100
      value: 92.833
    - type: mrr_at_1000
      value: 92.833
    - type: mrr_at_3
      value: 92.333
    - type: mrr_at_5
      value: 92.833
    - type: ndcg_at_1
      value: 83.0
    - type: ndcg_at_10
      value: 75.17
    - type: ndcg_at_100
      value: 55.432
    - type: ndcg_at_1000
      value: 49.482
    - type: ndcg_at_3
      value: 82.184
    - type: ndcg_at_5
      value: 79.712
    - type: precision_at_1
      value: 88.0
    - type: precision_at_10
      value: 78.60000000000001
    - type: precision_at_100
      value: 56.56
    - type: precision_at_1000
      value: 22.334
    - type: precision_at_3
      value: 86.667
    - type: precision_at_5
      value: 83.6
    - type: recall_at_1
      value: 0.22300000000000003
    - type: recall_at_10
      value: 1.9879999999999998
    - type: recall_at_100
      value: 13.300999999999998
    - type: recall_at_1000
      value: 46.587
    - type: recall_at_3
      value: 0.6629999999999999
    - type: recall_at_5
      value: 1.079
  - task:
      type: Retrieval
    dataset:
      name: MTEB Touche2020
      type: webis-touche2020
      config: default
      split: test
      revision: 527b7d77e16e343303e68cb6af11d6e18b9f7b3b
    metrics:
    - type: map_at_1
      value: 3.047
    - type: map_at_10
      value: 8.792
    - type: map_at_100
      value: 14.631
    - type: map_at_1000
      value: 16.127
    - type: map_at_3
      value: 4.673
    - type: map_at_5
      value: 5.897
    - type: mrr_at_1
      value: 38.775999999999996
    - type: mrr_at_10
      value: 49.271
    - type: mrr_at_100
      value: 50.181
    - type: mrr_at_1000
      value: 50.2
    - type: mrr_at_3
      value: 44.558
    - type: mrr_at_5
      value: 47.925000000000004
    - type: ndcg_at_1
      value: 35.714
    - type: ndcg_at_10
      value: 23.44
    - type: ndcg_at_100
      value: 35.345
    - type: ndcg_at_1000
      value: 46.495
    - type: ndcg_at_3
      value: 26.146
    - type: ndcg_at_5
      value: 24.878
    - type: precision_at_1
      value: 38.775999999999996
    - type: precision_at_10
      value: 20.816000000000003
    - type: precision_at_100
      value: 7.428999999999999
    - type: precision_at_1000
      value: 1.494
    - type: precision_at_3
      value: 25.85
    - type: precision_at_5
      value: 24.082
    - type: recall_at_1
      value: 3.047
    - type: recall_at_10
      value: 14.975
    - type: recall_at_100
      value: 45.943
    - type: recall_at_1000
      value: 80.31099999999999
    - type: recall_at_3
      value: 5.478000000000001
    - type: recall_at_5
      value: 8.294
  - task:
      type: Classification
    dataset:
      name: MTEB ToxicConversationsClassification
      type: mteb/toxic_conversations_50k
      config: default
      split: test
      revision: edfaf9da55d3dd50d43143d90c1ac476895ae6de
    metrics:
    - type: accuracy
      value: 68.84080000000002
    - type: ap
      value: 13.135219251019848
    - type: f1
      value: 52.849999421995506
  - task:
      type: Classification
    dataset:
      name: MTEB TweetSentimentExtractionClassification
      type: mteb/tweet_sentiment_extraction
      config: default
      split: test
      revision: 62146448f05be9e52a36b8ee9936447ea787eede
    metrics:
    - type: accuracy
      value: 56.68647425014149
    - type: f1
      value: 56.97981427365949
  - task:
      type: Clustering
    dataset:
      name: MTEB TwentyNewsgroupsClustering
      type: mteb/twentynewsgroups-clustering
      config: default
      split: test
      revision: 091a54f9a36281ce7d6590ec8c75dd485e7e01d4
    metrics:
    - type: v_measure
      value: 40.8911707239219
  - task:
      type: PairClassification
    dataset:
      name: MTEB TwitterSemEval2015
      type: mteb/twittersemeval2015-pairclassification
      config: default
      split: test
      revision: 70970daeab8776df92f5ea462b6173c0b46fd2d1
    metrics:
    - type: cos_sim_accuracy
      value: 83.04226023722954
    - type: cos_sim_ap
      value: 63.681339908301325
    - type: cos_sim_f1
      value: 60.349184470480125
    - type: cos_sim_precision
      value: 53.437754271765655
    - type: cos_sim_recall
      value: 69.31398416886545
    - type: dot_accuracy
      value: 81.46271681468677
    - type: dot_ap
      value: 57.78072296265885
    - type: dot_f1
      value: 56.28769265132901
    - type: dot_precision
      value: 48.7993803253292
    - type: dot_recall
      value: 66.49076517150397
    - type: euclidean_accuracy
      value: 82.16606067830959
    - type: euclidean_ap
      value: 59.974530371203514
    - type: euclidean_f1
      value: 56.856023506366306
    - type: euclidean_precision
      value: 53.037916857012334
    - type: euclidean_recall
      value: 61.2664907651715
    - type: manhattan_accuracy
      value: 82.16606067830959
    - type: manhattan_ap
      value: 59.98962379571767
    - type: manhattan_f1
      value: 56.98153158451947
    - type: manhattan_precision
      value: 51.41158989598811
    - type: manhattan_recall
      value: 63.90501319261214
    - type: max_accuracy
      value: 83.04226023722954
    - type: max_ap
      value: 63.681339908301325
    - type: max_f1
      value: 60.349184470480125
  - task:
      type: PairClassification
    dataset:
      name: MTEB TwitterURLCorpus
      type: mteb/twitterurlcorpus-pairclassification
      config: default
      split: test
      revision: 8b6510b0b1fa4e4c4f879467980e9be563ec1cdf
    metrics:
    - type: cos_sim_accuracy
      value: 88.56871191834517
    - type: cos_sim_ap
      value: 84.80240716354544
    - type: cos_sim_f1
      value: 77.07765285922385
    - type: cos_sim_precision
      value: 74.84947406601378
    - type: cos_sim_recall
      value: 79.44256236526024
    - type: dot_accuracy
      value: 86.00923662048356
    - type: dot_ap
      value: 78.6556459012073
    - type: dot_f1
      value: 72.7583749109052
    - type: dot_precision
      value: 67.72823779193206
    - type: dot_recall
      value: 78.59562673236834
    - type: euclidean_accuracy
      value: 87.84103698529127
    - type: euclidean_ap
      value: 83.50424424952834
    - type: euclidean_f1
      value: 75.74496544549307
    - type: euclidean_precision
      value: 73.19402556369381
    - type: euclidean_recall
      value: 78.48013550970127
    - type: manhattan_accuracy
      value: 87.9225365777933
    - type: manhattan_ap
      value: 83.49479248597825
    - type: manhattan_f1
      value: 75.67748162447101
    - type: manhattan_precision
      value: 73.06810035842294
    - type: manhattan_recall
      value: 78.48013550970127
    - type: max_accuracy
      value: 88.56871191834517
    - type: max_ap
      value: 84.80240716354544
    - type: max_f1
      value: 77.07765285922385
---

# SGPT-2.7B-weightedmean-msmarco-specb-bitfit

## Usage

For usage instructions, refer to our codebase: https://github.com/Muennighoff/sgpt 

## Evaluation Results

For eval results, refer to the eval folder or our paper: https://arxiv.org/abs/2202.08904

## Training
The model was trained with the parameters:

**DataLoader**:

`torch.utils.data.dataloader.DataLoader` of length 124796 with parameters:
```
{'batch_size': 4, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
```

**Loss**:

`sentence_transformers.losses.MultipleNegativesRankingLoss.MultipleNegativesRankingLoss` with parameters:
  ```
  {'scale': 20.0, 'similarity_fct': 'cos_sim'}
  ```

Parameters of the fit()-Method:
```
{
    "epochs": 10,
    "evaluation_steps": 0,
    "evaluator": "NoneType",
    "max_grad_norm": 1,
    "optimizer_class": "<class 'transformers.optimization.AdamW'>",
    "optimizer_params": {
        "lr": 7.5e-05
    },
    "scheduler": "WarmupLinear",
    "steps_per_epoch": null,
    "warmup_steps": 1000,
    "weight_decay": 0.01
}
```


## Full Model Architecture
```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 300, 'do_lower_case': False}) with Transformer model: GPTNeoModel 
  (1): Pooling({'word_embedding_dimension': 2560, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': True, 'pooling_mode_lasttoken': False})
)
```

## Citing & Authors

```bibtex
@article{muennighoff2022sgpt,
  title={SGPT: GPT Sentence Embeddings for Semantic Search},
  author={Muennighoff, Niklas},
  journal={arXiv preprint arXiv:2202.08904},
  year={2022}
}
```