transformers pytorch bert feature-extraction mteb model-index text-embeddings-inference endpoints_compatible region:us
---
tags:
- mteb
model-index:
- name: ALL_862873
  results:
  - task:
      type: Classification
    dataset:
      name: MTEB AmazonCounterfactualClassification (en)
      type: mteb/amazon_counterfactual
      config: en
      split: test
      revision: e8379541af4e31359cca9fbcf4b00f2671dba205
    metrics:
    - type: accuracy
      value: 50.805970149253746
    - type: ap
      value: 21.350961103104364
    - type: f1
      value: 46.546166439875044
  - task:
      type: Classification
    dataset:
      name: MTEB AmazonPolarityClassification
      type: mteb/amazon_polarity
      config: default
      split: test
      revision: e2d317d38cd51312af73b3d32a06d1a08b442046
    metrics:
    - type: accuracy
      value: 52.567125000000004
    - type: ap
      value: 51.37893936391345
    - type: f1
      value: 51.8411977908125
  - task:
      type: Classification
    dataset:
      name: MTEB AmazonReviewsClassification (en)
      type: mteb/amazon_reviews_multi
      config: en
      split: test
      revision: 1399c76144fd37290681b995c656ef9b2e06e26d
    metrics:
    - type: accuracy
      value: 22.63
    - type: f1
      value: 21.964526516204575
  - task:
      type: Retrieval
    dataset:
      name: MTEB ArguAna
      type: arguana
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 1.991
    - type: map_at_10
      value: 4.095
    - type: map_at_100
      value: 4.763
    - type: map_at_1000
      value: 4.8759999999999994
    - type: map_at_3
      value: 3.3070000000000004
    - type: map_at_5
      value: 3.73
    - type: mrr_at_1
      value: 2.0629999999999997
    - type: mrr_at_10
      value: 4.119
    - type: mrr_at_100
      value: 4.787
    - type: mrr_at_1000
      value: 4.9
    - type: mrr_at_3
      value: 3.331
    - type: mrr_at_5
      value: 3.768
    - type: ndcg_at_1
      value: 1.991
    - type: ndcg_at_10
      value: 5.346
    - type: ndcg_at_100
      value: 9.181000000000001
    - type: ndcg_at_1000
      value: 13.004
    - type: ndcg_at_3
      value: 3.7199999999999998
    - type: ndcg_at_5
      value: 4.482
    - type: precision_at_1
      value: 1.991
    - type: precision_at_10
      value: 0.9390000000000001
    - type: precision_at_100
      value: 0.28700000000000003
    - type: precision_at_1000
      value: 0.061
    - type: precision_at_3
      value: 1.636
    - type: precision_at_5
      value: 1.351
    - type: recall_at_1
      value: 1.991
    - type: recall_at_10
      value: 9.388
    - type: recall_at_100
      value: 28.663
    - type: recall_at_1000
      value: 60.597
    - type: recall_at_3
      value: 4.9079999999999995
    - type: recall_at_5
      value: 6.757000000000001
  - task:
      type: Clustering
    dataset:
      name: MTEB ArxivClusteringP2P
      type: mteb/arxiv-clustering-p2p
      config: default
      split: test
      revision: a122ad7f3f0291bf49cc6f4d32aa80929df69d5d
    metrics:
    - type: v_measure
      value: 14.790995349964428
  - task:
      type: Clustering
    dataset:
      name: MTEB ArxivClusteringS2S
      type: mteb/arxiv-clustering-s2s
      config: default
      split: test
      revision: f910caf1a6075f7329cdf8c1a6135696f37dbd53
    metrics:
    - type: v_measure
      value: 12.248406292959412
  - task:
      type: Reranking
    dataset:
      name: MTEB AskUbuntuDupQuestions
      type: mteb/askubuntudupquestions-reranking
      config: default
      split: test
      revision: 2000358ca161889fa9c082cb41daa8dcfb161a54
    metrics:
    - type: map
      value: 44.88116875696166
    - type: mrr
      value: 56.07439651760981
  - task:
      type: STS
    dataset:
      name: MTEB BIOSSES
      type: mteb/biosses-sts
      config: default
      split: test
      revision: d3fb88f8f02e40887cd149695127462bbcf29b4a
    metrics:
    - type: cos_sim_pearson
      value: 19.26573437410263
    - type: cos_sim_spearman
      value: 21.34145013484056
    - type: euclidean_pearson
      value: 22.39226418475093
    - type: euclidean_spearman
      value: 23.511981519581447
    - type: manhattan_pearson
      value: 22.14346931904813
    - type: manhattan_spearman
      value: 23.39390654000631
  - task:
      type: Classification
    dataset:
      name: MTEB Banking77Classification
      type: mteb/banking77
      config: default
      split: test
      revision: 0fd18e25b25c072e09e0d92ab615fda904d66300
    metrics:
    - type: accuracy
      value: 36.42857142857143
    - type: f1
      value: 34.81640976406094
  - task:
      type: Clustering
    dataset:
      name: MTEB BiorxivClusteringP2P
      type: mteb/biorxiv-clustering-p2p
      config: default
      split: test
      revision: 65b79d1d13f80053f67aca9498d9402c2d9f1f40
    metrics:
    - type: v_measure
      value: 13.94296328377691
  - task:
      type: Clustering
    dataset:
      name: MTEB BiorxivClusteringS2S
      type: mteb/biorxiv-clustering-s2s
      config: default
      split: test
      revision: 258694dd0231531bc1fd9de6ceb52a0853c6d908
    metrics:
    - type: v_measure
      value: 9.790764523161606
  - task:
      type: Retrieval
    dataset:
      name: MTEB CQADupstackAndroidRetrieval
      type: BeIR/cqadupstack
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 0.968
    - type: map_at_10
      value: 2.106
    - type: map_at_100
      value: 2.411
    - type: map_at_1000
      value: 2.4899999999999998
    - type: map_at_3
      value: 1.797
    - type: map_at_5
      value: 1.9959999999999998
    - type: mrr_at_1
      value: 1.717
    - type: mrr_at_10
      value: 3.0349999999999997
    - type: mrr_at_100
      value: 3.4029999999999996
    - type: mrr_at_1000
      value: 3.486
    - type: mrr_at_3
      value: 2.6470000000000002
    - type: mrr_at_5
      value: 2.876
    - type: ndcg_at_1
      value: 1.717
    - type: ndcg_at_10
      value: 2.9059999999999997
    - type: ndcg_at_100
      value: 4.715
    - type: ndcg_at_1000
      value: 7.318
    - type: ndcg_at_3
      value: 2.415
    - type: ndcg_at_5
      value: 2.682
    - type: precision_at_1
      value: 1.717
    - type: precision_at_10
      value: 0.658
    - type: precision_at_100
      value: 0.197
    - type: precision_at_1000
      value: 0.054
    - type: precision_at_3
      value: 1.431
    - type: precision_at_5
      value: 1.059
    - type: recall_at_1
      value: 0.968
    - type: recall_at_10
      value: 4.531000000000001
    - type: recall_at_100
      value: 13.081000000000001
    - type: recall_at_1000
      value: 32.443
    - type: recall_at_3
      value: 2.8850000000000002
    - type: recall_at_5
      value: 3.768
    - type: map_at_1
      value: 0.9390000000000001
    - type: map_at_10
      value: 1.516
    - type: map_at_100
      value: 1.6680000000000001
    - type: map_at_1000
      value: 1.701
    - type: map_at_3
      value: 1.314
    - type: map_at_5
      value: 1.388
    - type: mrr_at_1
      value: 1.146
    - type: mrr_at_10
      value: 1.96
    - type: mrr_at_100
      value: 2.166
    - type: mrr_at_1000
      value: 2.207
    - type: mrr_at_3
      value: 1.72
    - type: mrr_at_5
      value: 1.796
    - type: ndcg_at_1
      value: 1.146
    - type: ndcg_at_10
      value: 1.9769999999999999
    - type: ndcg_at_100
      value: 2.8400000000000003
    - type: ndcg_at_1000
      value: 4.035
    - type: ndcg_at_3
      value: 1.5859999999999999
    - type: ndcg_at_5
      value: 1.6709999999999998
    - type: precision_at_1
      value: 1.146
    - type: precision_at_10
      value: 0.43299999999999994
    - type: precision_at_100
      value: 0.11100000000000002
    - type: precision_at_1000
      value: 0.027999999999999997
    - type: precision_at_3
      value: 0.8699999999999999
    - type: precision_at_5
      value: 0.611
    - type: recall_at_1
      value: 0.9390000000000001
    - type: recall_at_10
      value: 2.949
    - type: recall_at_100
      value: 6.737
    - type: recall_at_1000
      value: 15.604999999999999
    - type: recall_at_3
      value: 1.846
    - type: recall_at_5
      value: 2.08
    - type: map_at_1
      value: 1.28
    - type: map_at_10
      value: 2.157
    - type: map_at_100
      value: 2.401
    - type: map_at_1000
      value: 2.4570000000000003
    - type: map_at_3
      value: 1.865
    - type: map_at_5
      value: 1.928
    - type: mrr_at_1
      value: 1.505
    - type: mrr_at_10
      value: 2.52
    - type: mrr_at_100
      value: 2.782
    - type: mrr_at_1000
      value: 2.8400000000000003
    - type: mrr_at_3
      value: 2.1839999999999997
    - type: mrr_at_5
      value: 2.2689999999999997
    - type: ndcg_at_1
      value: 1.505
    - type: ndcg_at_10
      value: 2.798
    - type: ndcg_at_100
      value: 4.2090000000000005
    - type: ndcg_at_1000
      value: 6.105
    - type: ndcg_at_3
      value: 2.157
    - type: ndcg_at_5
      value: 2.258
    - type: precision_at_1
      value: 1.505
    - type: precision_at_10
      value: 0.5519999999999999
    - type: precision_at_100
      value: 0.146
    - type: precision_at_1000
      value: 0.034999999999999996
    - type: precision_at_3
      value: 1.024
    - type: precision_at_5
      value: 0.7020000000000001
    - type: recall_at_1
      value: 1.28
    - type: recall_at_10
      value: 4.455
    - type: recall_at_100
      value: 11.169
    - type: recall_at_1000
      value: 26.046000000000003
    - type: recall_at_3
      value: 2.6270000000000002
    - type: recall_at_5
      value: 2.899
    - type: map_at_1
      value: 0.264
    - type: map_at_10
      value: 0.615
    - type: map_at_100
      value: 0.76
    - type: map_at_1000
      value: 0.803
    - type: map_at_3
      value: 0.40499999999999997
    - type: map_at_5
      value: 0.512
    - type: mrr_at_1
      value: 0.33899999999999997
    - type: mrr_at_10
      value: 0.718
    - type: mrr_at_100
      value: 0.8880000000000001
    - type: mrr_at_1000
      value: 0.935
    - type: mrr_at_3
      value: 0.508
    - type: mrr_at_5
      value: 0.616
    - type: ndcg_at_1
      value: 0.33899999999999997
    - type: ndcg_at_10
      value: 0.9079999999999999
    - type: ndcg_at_100
      value: 1.9029999999999998
    - type: ndcg_at_1000
      value: 3.4939999999999998
    - type: ndcg_at_3
      value: 0.46499999999999997
    - type: ndcg_at_5
      value: 0.655
    - type: precision_at_1
      value: 0.33899999999999997
    - type: precision_at_10
      value: 0.192
    - type: precision_at_100
      value: 0.079
    - type: precision_at_1000
      value: 0.023
    - type: precision_at_3
      value: 0.22599999999999998
    - type: precision_at_5
      value: 0.22599999999999998
    - type: recall_at_1
      value: 0.264
    - type: recall_at_10
      value: 1.789
    - type: recall_at_100
      value: 6.927
    - type: recall_at_1000
      value: 19.922
    - type: recall_at_3
      value: 0.5459999999999999
    - type: recall_at_5
      value: 0.9979999999999999
    - type: map_at_1
      value: 0.5599999999999999
    - type: map_at_10
      value: 0.9129999999999999
    - type: map_at_100
      value: 1.027
    - type: map_at_1000
      value: 1.072
    - type: map_at_3
      value: 0.715
    - type: map_at_5
      value: 0.826
    - type: mrr_at_1
      value: 0.8710000000000001
    - type: mrr_at_10
      value: 1.331
    - type: mrr_at_100
      value: 1.494
    - type: mrr_at_1000
      value: 1.547
    - type: mrr_at_3
      value: 1.119
    - type: mrr_at_5
      value: 1.269
    - type: ndcg_at_1
      value: 0.8710000000000001
    - type: ndcg_at_10
      value: 1.2590000000000001
    - type: ndcg_at_100
      value: 2.023
    - type: ndcg_at_1000
      value: 3.737
    - type: ndcg_at_3
      value: 0.8750000000000001
    - type: ndcg_at_5
      value: 1.079
    - type: precision_at_1
      value: 0.8710000000000001
    - type: precision_at_10
      value: 0.28600000000000003
    - type: precision_at_100
      value: 0.086
    - type: precision_at_1000
      value: 0.027999999999999997
    - type: precision_at_3
      value: 0.498
    - type: precision_at_5
      value: 0.42300000000000004
    - type: recall_at_1
      value: 0.5599999999999999
    - type: recall_at_10
      value: 1.907
    - type: recall_at_100
      value: 5.492
    - type: recall_at_1000
      value: 18.974
    - type: recall_at_3
      value: 0.943
    - type: recall_at_5
      value: 1.41
    - type: map_at_1
      value: 1.9720000000000002
    - type: map_at_10
      value: 2.968
    - type: map_at_100
      value: 3.2009999999999996
    - type: map_at_1000
      value: 3.2680000000000002
    - type: map_at_3
      value: 2.683
    - type: map_at_5
      value: 2.8369999999999997
    - type: mrr_at_1
      value: 2.406
    - type: mrr_at_10
      value: 3.567
    - type: mrr_at_100
      value: 3.884
    - type: mrr_at_1000
      value: 3.948
    - type: mrr_at_3
      value: 3.2239999999999998
    - type: mrr_at_5
      value: 3.383
    - type: ndcg_at_1
      value: 2.406
    - type: ndcg_at_10
      value: 3.63
    - type: ndcg_at_100
      value: 5.155
    - type: ndcg_at_1000
      value: 7.381
    - type: ndcg_at_3
      value: 3.078
    - type: ndcg_at_5
      value: 3.3070000000000004
    - type: precision_at_1
      value: 2.406
    - type: precision_at_10
      value: 0.635
    - type: precision_at_100
      value: 0.184
    - type: precision_at_1000
      value: 0.048
    - type: precision_at_3
      value: 1.4120000000000001
    - type: precision_at_5
      value: 1.001
    - type: recall_at_1
      value: 1.9720000000000002
    - type: recall_at_10
      value: 5.152
    - type: recall_at_100
      value: 12.173
    - type: recall_at_1000
      value: 28.811999999999998
    - type: recall_at_3
      value: 3.556
    - type: recall_at_5
      value: 4.181
    - type: map_at_1
      value: 0.346
    - type: map_at_10
      value: 0.619
    - type: map_at_100
      value: 0.743
    - type: map_at_1000
      value: 0.788
    - type: map_at_3
      value: 0.5369999999999999
    - type: map_at_5
      value: 0.551
    - type: mrr_at_1
      value: 0.571
    - type: mrr_at_10
      value: 1.0619999999999998
    - type: mrr_at_100
      value: 1.2109999999999999
    - type: mrr_at_1000
      value: 1.265
    - type: mrr_at_3
      value: 0.818
    - type: mrr_at_5
      value: 0.927
    - type: ndcg_at_1
      value: 0.571
    - type: ndcg_at_10
      value: 0.919
    - type: ndcg_at_100
      value: 1.688
    - type: ndcg_at_1000
      value: 3.3649999999999998
    - type: ndcg_at_3
      value: 0.6779999999999999
    - type: ndcg_at_5
      value: 0.7230000000000001
    - type: precision_at_1
      value: 0.571
    - type: precision_at_10
      value: 0.27399999999999997
    - type: precision_at_100
      value: 0.084
    - type: precision_at_1000
      value: 0.029
    - type: precision_at_3
      value: 0.381
    - type: precision_at_5
      value: 0.32
    - type: recall_at_1
      value: 0.346
    - type: recall_at_10
      value: 1.397
    - type: recall_at_100
      value: 5.079000000000001
    - type: recall_at_1000
      value: 18.060000000000002
    - type: recall_at_3
      value: 0.774
    - type: recall_at_5
      value: 0.8340000000000001
    - type: map_at_1
      value: 0.69
    - type: map_at_10
      value: 0.897
    - type: map_at_100
      value: 1.0030000000000001
    - type: map_at_1000
      value: 1.034
    - type: map_at_3
      value: 0.818
    - type: map_at_5
      value: 0.864
    - type: mrr_at_1
      value: 0.767
    - type: mrr_at_10
      value: 1.008
    - type: mrr_at_100
      value: 1.145
    - type: mrr_at_1000
      value: 1.183
    - type: mrr_at_3
      value: 0.895
    - type: mrr_at_5
      value: 0.9560000000000001
    - type: ndcg_at_1
      value: 0.767
    - type: ndcg_at_10
      value: 1.0739999999999998
    - type: ndcg_at_100
      value: 1.757
    - type: ndcg_at_1000
      value: 2.9090000000000003
    - type: ndcg_at_3
      value: 0.881
    - type: ndcg_at_5
      value: 0.9769999999999999
    - type: precision_at_1
      value: 0.767
    - type: precision_at_10
      value: 0.184
    - type: precision_at_100
      value: 0.06
    - type: precision_at_1000
      value: 0.018000000000000002
    - type: precision_at_3
      value: 0.358
    - type: precision_at_5
      value: 0.27599999999999997
    - type: recall_at_1
      value: 0.69
    - type: recall_at_10
      value: 1.508
    - type: recall_at_100
      value: 4.858
    - type: recall_at_1000
      value: 14.007
    - type: recall_at_3
      value: 0.997
    - type: recall_at_5
      value: 1.2269999999999999
    - type: map_at_1
      value: 0.338
    - type: map_at_10
      value: 0.661
    - type: map_at_100
      value: 0.7969999999999999
    - type: map_at_1000
      value: 0.8290000000000001
    - type: map_at_3
      value: 0.5559999999999999
    - type: map_at_5
      value: 0.5910000000000001
    - type: mrr_at_1
      value: 0.482
    - type: mrr_at_10
      value: 0.88
    - type: mrr_at_100
      value: 1.036
    - type: mrr_at_1000
      value: 1.075
    - type: mrr_at_3
      value: 0.74
    - type: mrr_at_5
      value: 0.779
    - type: ndcg_at_1
      value: 0.482
    - type: ndcg_at_10
      value: 0.924
    - type: ndcg_at_100
      value: 1.736
    - type: ndcg_at_1000
      value: 2.926
    - type: ndcg_at_3
      value: 0.677
    - type: ndcg_at_5
      value: 0.732
    - type: precision_at_1
      value: 0.482
    - type: precision_at_10
      value: 0.20600000000000002
    - type: precision_at_100
      value: 0.078
    - type: precision_at_1000
      value: 0.023
    - type: precision_at_3
      value: 0.367
    - type: precision_at_5
      value: 0.255
    - type: recall_at_1
      value: 0.338
    - type: recall_at_10
      value: 1.545
    - type: recall_at_100
      value: 5.38
    - type: recall_at_1000
      value: 14.609
    - type: recall_at_3
      value: 0.826
    - type: recall_at_5
      value: 0.975
    - type: map_at_1
      value: 0.8240000000000001
    - type: map_at_10
      value: 1.254
    - type: map_at_100
      value: 1.389
    - type: map_at_1000
      value: 1.419
    - type: map_at_3
      value: 1.158
    - type: map_at_5
      value: 1.189
    - type: mrr_at_1
      value: 0.9329999999999999
    - type: mrr_at_10
      value: 1.4200000000000002
    - type: mrr_at_100
      value: 1.59
    - type: mrr_at_1000
      value: 1.629
    - type: mrr_at_3
      value: 1.29
    - type: mrr_at_5
      value: 1.332
    - type: ndcg_at_1
      value: 0.9329999999999999
    - type: ndcg_at_10
      value: 1.53
    - type: ndcg_at_100
      value: 2.418
    - type: ndcg_at_1000
      value: 3.7310000000000003
    - type: ndcg_at_3
      value: 1.302
    - type: ndcg_at_5
      value: 1.363
    - type: precision_at_1
      value: 0.9329999999999999
    - type: precision_at_10
      value: 0.271
    - type: precision_at_100
      value: 0.083
    - type: precision_at_1000
      value: 0.024
    - type: precision_at_3
      value: 0.622
    - type: precision_at_5
      value: 0.41000000000000003
    - type: recall_at_1
      value: 0.8240000000000001
    - type: recall_at_10
      value: 2.1999999999999997
    - type: recall_at_100
      value: 6.584
    - type: recall_at_1000
      value: 17.068
    - type: recall_at_3
      value: 1.5859999999999999
    - type: recall_at_5
      value: 1.7260000000000002
    - type: map_at_1
      value: 0.404
    - type: map_at_10
      value: 0.788
    - type: map_at_100
      value: 0.9860000000000001
    - type: map_at_1000
      value: 1.04
    - type: map_at_3
      value: 0.676
    - type: map_at_5
      value: 0.733
    - type: mrr_at_1
      value: 0.5930000000000001
    - type: mrr_at_10
      value: 1.278
    - type: mrr_at_100
      value: 1.545
    - type: mrr_at_1000
      value: 1.599
    - type: mrr_at_3
      value: 1.054
    - type: mrr_at_5
      value: 1.192
    - type: ndcg_at_1
      value: 0.5930000000000001
    - type: ndcg_at_10
      value: 1.1280000000000001
    - type: ndcg_at_100
      value: 2.2689999999999997
    - type: ndcg_at_1000
      value: 4.274
    - type: ndcg_at_3
      value: 0.919
    - type: ndcg_at_5
      value: 1.038
    - type: precision_at_1
      value: 0.5930000000000001
    - type: precision_at_10
      value: 0.296
    - type: precision_at_100
      value: 0.152
    - type: precision_at_1000
      value: 0.05
    - type: precision_at_3
      value: 0.527
    - type: precision_at_5
      value: 0.47400000000000003
    - type: recall_at_1
      value: 0.404
    - type: recall_at_10
      value: 1.601
    - type: recall_at_100
      value: 6.885
    - type: recall_at_1000
      value: 22.356
    - type: recall_at_3
      value: 0.9490000000000001
    - type: recall_at_5
      value: 1.206
    - type: map_at_1
      value: 0.185
    - type: map_at_10
      value: 0.192
    - type: map_at_100
      value: 0.271
    - type: map_at_1000
      value: 0.307
    - type: map_at_3
      value: 0.185
    - type: map_at_5
      value: 0.185
    - type: mrr_at_1
      value: 0.185
    - type: mrr_at_10
      value: 0.20500000000000002
    - type: mrr_at_100
      value: 0.292
    - type: mrr_at_1000
      value: 0.331
    - type: mrr_at_3
      value: 0.185
    - type: mrr_at_5
      value: 0.185
    - type: ndcg_at_1
      value: 0.185
    - type: ndcg_at_10
      value: 0.211
    - type: ndcg_at_100
      value: 0.757
    - type: ndcg_at_1000
      value: 1.928
    - type: ndcg_at_3
      value: 0.185
    - type: ndcg_at_5
      value: 0.185
    - type: precision_at_1
      value: 0.185
    - type: precision_at_10
      value: 0.037
    - type: precision_at_100
      value: 0.039
    - type: precision_at_1000
      value: 0.015
    - type: precision_at_3
      value: 0.062
    - type: precision_at_5
      value: 0.037
    - type: recall_at_1
      value: 0.185
    - type: recall_at_10
      value: 0.246
    - type: recall_at_100
      value: 3.05
    - type: recall_at_1000
      value: 12.5
    - type: recall_at_3
      value: 0.185
    - type: recall_at_5
      value: 0.185
  - task:
      type: Retrieval
    dataset:
      name: MTEB ClimateFEVER
      type: climate-fever
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 0.241
    - type: map_at_10
      value: 0.372
    - type: map_at_100
      value: 0.45999999999999996
    - type: map_at_1000
      value: 0.47600000000000003
    - type: map_at_3
      value: 0.33999999999999997
    - type: map_at_5
      value: 0.359
    - type: mrr_at_1
      value: 0.651
    - type: mrr_at_10
      value: 1.03
    - type: mrr_at_100
      value: 1.2489999999999999
    - type: mrr_at_1000
      value: 1.282
    - type: mrr_at_3
      value: 0.9450000000000001
    - type: mrr_at_5
      value: 1.0030000000000001
    - type: ndcg_at_1
      value: 0.651
    - type: ndcg_at_10
      value: 0.588
    - type: ndcg_at_100
      value: 1.2550000000000001
    - type: ndcg_at_1000
      value: 1.9040000000000001
    - type: ndcg_at_3
      value: 0.547
    - type: ndcg_at_5
      value: 0.549
    - type: precision_at_1
      value: 0.651
    - type: precision_at_10
      value: 0.182
    - type: precision_at_100
      value: 0.086
    - type: precision_at_1000
      value: 0.02
    - type: precision_at_3
      value: 0.434
    - type: precision_at_5
      value: 0.313
    - type: recall_at_1
      value: 0.241
    - type: recall_at_10
      value: 0.63
    - type: recall_at_100
      value: 3.1759999999999997
    - type: recall_at_1000
      value: 7.175
    - type: recall_at_3
      value: 0.46299999999999997
    - type: recall_at_5
      value: 0.543
  - task:
      type: Retrieval
    dataset:
      name: MTEB DBPedia
      type: dbpedia-entity
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 0.04
    - type: map_at_10
      value: 0.089
    - type: map_at_100
      value: 0.133
    - type: map_at_1000
      value: 0.165
    - type: map_at_3
      value: 0.054
    - type: map_at_5
      value: 0.056999999999999995
    - type: mrr_at_1
      value: 0.75
    - type: mrr_at_10
      value: 1.4749999999999999
    - type: mrr_at_100
      value: 1.8010000000000002
    - type: mrr_at_1000
      value: 1.847
    - type: mrr_at_3
      value: 1.208
    - type: mrr_at_5
      value: 1.333
    - type: ndcg_at_1
      value: 0.625
    - type: ndcg_at_10
      value: 0.428
    - type: ndcg_at_100
      value: 0.705
    - type: ndcg_at_1000
      value: 1.564
    - type: ndcg_at_3
      value: 0.5369999999999999
    - type: ndcg_at_5
      value: 0.468
    - type: precision_at_1
      value: 0.75
    - type: precision_at_10
      value: 0.375
    - type: precision_at_100
      value: 0.27499999999999997
    - type: precision_at_1000
      value: 0.10300000000000001
    - type: precision_at_3
      value: 0.583
    - type: precision_at_5
      value: 0.5
    - type: recall_at_1
      value: 0.04
    - type: recall_at_10
      value: 0.385
    - type: recall_at_100
      value: 1.2670000000000001
    - type: recall_at_1000
      value: 4.522
    - type: recall_at_3
      value: 0.07100000000000001
    - type: recall_at_5
      value: 0.08099999999999999
  - task:
      type: Classification
    dataset:
      name: MTEB EmotionClassification
      type: mteb/emotion
      config: default
      split: test
      revision: 4f58c6b202a23cf9a4da393831edf4f9183cad37
    metrics:
    - type: accuracy
      value: 22.749999999999996
    - type: f1
      value: 19.335020165482693
  - task:
      type: Retrieval
    dataset:
      name: MTEB FEVER
      type: fever
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 0.257
    - type: map_at_10
      value: 0.416
    - type: map_at_100
      value: 0.451
    - type: map_at_1000
      value: 0.46499999999999997
    - type: map_at_3
      value: 0.37
    - type: map_at_5
      value: 0.386
    - type: mrr_at_1
      value: 0.27
    - type: mrr_at_10
      value: 0.44200000000000006
    - type: mrr_at_100
      value: 0.48
    - type: mrr_at_1000
      value: 0.49500000000000005
    - type: mrr_at_3
      value: 0.38999999999999996
    - type: mrr_at_5
      value: 0.411
    - type: ndcg_at_1
      value: 0.27
    - type: ndcg_at_10
      value: 0.51
    - type: ndcg_at_100
      value: 0.738
    - type: ndcg_at_1000
      value: 1.2630000000000001
    - type: ndcg_at_3
      value: 0.41000000000000003
    - type: ndcg_at_5
      value: 0.439
    - type: precision_at_1
      value: 0.27
    - type: precision_at_10
      value: 0.084
    - type: precision_at_100
      value: 0.021
    - type: precision_at_1000
      value: 0.006999999999999999
    - type: precision_at_3
      value: 0.17500000000000002
    - type: precision_at_5
      value: 0.123
    - type: recall_at_1
      value: 0.257
    - type: recall_at_10
      value: 0.786
    - type: recall_at_100
      value: 1.959
    - type: recall_at_1000
      value: 6.334
    - type: recall_at_3
      value: 0.49699999999999994
    - type: recall_at_5
      value: 0.5680000000000001
  - task:
      type: Retrieval
    dataset:
      name: MTEB FiQA2018
      type: fiqa
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 0.28900000000000003
    - type: map_at_10
      value: 0.475
    - type: map_at_100
      value: 0.559
    - type: map_at_1000
      value: 0.5930000000000001
    - type: map_at_3
      value: 0.38999999999999996
    - type: map_at_5
      value: 0.41700000000000004
    - type: mrr_at_1
      value: 0.772
    - type: mrr_at_10
      value: 1.107
    - type: mrr_at_100
      value: 1.269
    - type: mrr_at_1000
      value: 1.323
    - type: mrr_at_3
      value: 0.9520000000000001
    - type: mrr_at_5
      value: 1.0290000000000001
    - type: ndcg_at_1
      value: 0.772
    - type: ndcg_at_10
      value: 0.755
    - type: ndcg_at_100
      value: 1.256
    - type: ndcg_at_1000
      value: 2.55
    - type: ndcg_at_3
      value: 0.633
    - type: ndcg_at_5
      value: 0.639
    - type: precision_at_1
      value: 0.772
    - type: precision_at_10
      value: 0.262
    - type: precision_at_100
      value: 0.082
    - type: precision_at_1000
      value: 0.03
    - type: precision_at_3
      value: 0.46299999999999997
    - type: precision_at_5
      value: 0.33999999999999997
    - type: recall_at_1
      value: 0.28900000000000003
    - type: recall_at_10
      value: 0.976
    - type: recall_at_100
      value: 2.802
    - type: recall_at_1000
      value: 11.466
    - type: recall_at_3
      value: 0.54
    - type: recall_at_5
      value: 0.6479999999999999
  - task:
      type: Retrieval
    dataset:
      name: MTEB HotpotQA
      type: hotpotqa
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 0.257
    - type: map_at_10
      value: 0.395
    - type: map_at_100
      value: 0.436
    - type: map_at_1000
      value: 0.447
    - type: map_at_3
      value: 0.347
    - type: map_at_5
      value: 0.369
    - type: mrr_at_1
      value: 0.513
    - type: mrr_at_10
      value: 0.787
    - type: mrr_at_100
      value: 0.865
    - type: mrr_at_1000
      value: 0.8840000000000001
    - type: mrr_at_3
      value: 0.6930000000000001
    - type: mrr_at_5
      value: 0.738
    - type: ndcg_at_1
      value: 0.513
    - type: ndcg_at_10
      value: 0.587
    - type: ndcg_at_100
      value: 0.881
    - type: ndcg_at_1000
      value: 1.336
    - type: ndcg_at_3
      value: 0.46299999999999997
    - type: ndcg_at_5
      value: 0.511
    - type: precision_at_1
      value: 0.513
    - type: precision_at_10
      value: 0.151
    - type: precision_at_100
      value: 0.04
    - type: precision_at_1000
      value: 0.01
    - type: precision_at_3
      value: 0.311
    - type: precision_at_5
      value: 0.22399999999999998
    - type: recall_at_1
      value: 0.257
    - type: recall_at_10
      value: 0.756
    - type: recall_at_100
      value: 1.9849999999999999
    - type: recall_at_1000
      value: 5.111000000000001
    - type: recall_at_3
      value: 0.466
    - type: recall_at_5
      value: 0.5599999999999999
  - task:
      type: Classification
    dataset:
      name: MTEB ImdbClassification
      type: mteb/imdb
      config: default
      split: test
      revision: 3d86128a09e091d6018b6d26cad27f2739fc2db7
    metrics:
    - type: accuracy
      value: 50.76400000000001
    - type: ap
      value: 50.41569411130455
    - type: f1
      value: 50.14266303576945
  - task:
      type: Retrieval
    dataset:
      name: MTEB MSMARCO
      type: msmarco
      config: default
      split: dev
      revision: None
    metrics:
    - type: map_at_1
      value: 0.14300000000000002
    - type: map_at_10
      value: 0.23700000000000002
    - type: map_at_100
      value: 0.27799999999999997
    - type: map_at_1000
      value: 0.291
    - type: map_at_3
      value: 0.197
    - type: map_at_5
      value: 0.215
    - type: mrr_at_1
      value: 0.14300000000000002
    - type: mrr_at_10
      value: 0.247
    - type: mrr_at_100
      value: 0.29
    - type: mrr_at_1000
      value: 0.303
    - type: mrr_at_3
      value: 0.201
    - type: mrr_at_5
      value: 0.219
    - type: ndcg_at_1
      value: 0.14300000000000002
    - type: ndcg_at_10
      value: 0.307
    - type: ndcg_at_100
      value: 0.5720000000000001
    - type: ndcg_at_1000
      value: 1.053
    - type: ndcg_at_3
      value: 0.215
    - type: ndcg_at_5
      value: 0.248
    - type: precision_at_1
      value: 0.14300000000000002
    - type: precision_at_10
      value: 0.056999999999999995
    - type: precision_at_100
      value: 0.02
    - type: precision_at_1000
      value: 0.006
    - type: precision_at_3
      value: 0.091
    - type: precision_at_5
      value: 0.07200000000000001
    - type: recall_at_1
      value: 0.14300000000000002
    - type: recall_at_10
      value: 0.522
    - type: recall_at_100
      value: 1.9009999999999998
    - type: recall_at_1000
      value: 5.893000000000001
    - type: recall_at_3
      value: 0.263
    - type: recall_at_5
      value: 0.34099999999999997
  - task:
      type: Classification
    dataset:
      name: MTEB MTOPDomainClassification (en)
      type: mteb/mtop_domain
      config: en
      split: test
      revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
    metrics:
    - type: accuracy
      value: 61.03283173734611
    - type: f1
      value: 61.24012492746259
  - task:
      type: Classification
    dataset:
      name: MTEB MTOPIntentClassification (en)
      type: mteb/mtop_intent
      config: en
      split: test
      revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
    metrics:
    - type: accuracy
      value: 29.68308253533972
    - type: f1
      value: 16.243459114946905
  - task:
      type: Classification
    dataset:
      name: MTEB MassiveIntentClassification (en)
      type: mteb/amazon_massive_intent
      config: en
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 34.330867518493605
    - type: f1
      value: 33.176158044175935
  - task:
      type: Classification
    dataset:
      name: MTEB MassiveScenarioClassification (en)
      type: mteb/amazon_massive_scenario
      config: en
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 44.13248150638871
    - type: f1
      value: 44.24904249078732
  - task:
      type: Clustering
    dataset:
      name: MTEB MedrxivClusteringP2P
      type: mteb/medrxiv-clustering-p2p
      config: default
      split: test
      revision: e7a26af6f3ae46b30dde8737f02c07b1505bcc73
    metrics:
    - type: v_measure
      value: 15.698400177259078
  - task:
      type: Clustering
    dataset:
      name: MTEB MedrxivClusteringS2S
      type: mteb/medrxiv-clustering-s2s
      config: default
      split: test
      revision: 35191c8c0dca72d8ff3efcd72aa802307d469663
    metrics:
    - type: v_measure
      value: 14.888797785310235
  - task:
      type: Reranking
    dataset:
      name: MTEB MindSmallReranking
      type: mteb/mind_small
      config: default
      split: test
      revision: 3bdac13927fdc888b903db93b2ffdbd90b295a69
    metrics:
    - type: map
      value: 25.652445385382126
    - type: mrr
      value: 25.891573325600227
  - task:
      type: Retrieval
    dataset:
      name: MTEB NFCorpus
      type: nfcorpus
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 0.322
    - type: map_at_10
      value: 0.7230000000000001
    - type: map_at_100
      value: 1.248
    - type: map_at_1000
      value: 1.873
    - type: map_at_3
      value: 0.479
    - type: map_at_5
      value: 0.5700000000000001
    - type: mrr_at_1
      value: 6.502
    - type: mrr_at_10
      value: 10.735
    - type: mrr_at_100
      value: 11.848
    - type: mrr_at_1000
      value: 11.995000000000001
    - type: mrr_at_3
      value: 9.391
    - type: mrr_at_5
      value: 9.732000000000001
    - type: ndcg_at_1
      value: 6.037
    - type: ndcg_at_10
      value: 4.873
    - type: ndcg_at_100
      value: 5.959
    - type: ndcg_at_1000
      value: 14.424000000000001
    - type: ndcg_at_3
      value: 5.4559999999999995
    - type: ndcg_at_5
      value: 5.074
    - type: precision_at_1
      value: 6.192
    - type: precision_at_10
      value: 4.458
    - type: precision_at_100
      value: 2.5700000000000003
    - type: precision_at_1000
      value: 1.3679999999999999
    - type: precision_at_3
      value: 5.676
    - type: precision_at_5
      value: 4.954
    - type: recall_at_1
      value: 0.322
    - type: recall_at_10
      value: 1.545
    - type: recall_at_100
      value: 8.301
    - type: recall_at_1000
      value: 37.294
    - type: recall_at_3
      value: 0.623
    - type: recall_at_5
      value: 0.865
  - task:
      type: Retrieval
    dataset:
      name: MTEB NQ
      type: nq
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 0.188
    - type: map_at_10
      value: 0.27
    - type: map_at_100
      value: 0.322
    - type: map_at_1000
      value: 0.335
    - type: map_at_3
      value: 0.246
    - type: map_at_5
      value: 0.246
    - type: mrr_at_1
      value: 0.203
    - type: mrr_at_10
      value: 0.28300000000000003
    - type: mrr_at_100
      value: 0.344
    - type: mrr_at_1000
      value: 0.357
    - type: mrr_at_3
      value: 0.261
    - type: mrr_at_5
      value: 0.261
    - type: ndcg_at_1
      value: 0.203
    - type: ndcg_at_10
      value: 0.329
    - type: ndcg_at_100
      value: 0.628
    - type: ndcg_at_1000
      value: 1.0959999999999999
    - type: ndcg_at_3
      value: 0.272
    - type: ndcg_at_5
      value: 0.272
    - type: precision_at_1
      value: 0.203
    - type: precision_at_10
      value: 0.055
    - type: precision_at_100
      value: 0.024
    - type: precision_at_1000
      value: 0.006999999999999999
    - type: precision_at_3
      value: 0.116
    - type: precision_at_5
      value: 0.06999999999999999
    - type: recall_at_1
      value: 0.188
    - type: recall_at_10
      value: 0.507
    - type: recall_at_100
      value: 1.883
    - type: recall_at_1000
      value: 5.609999999999999
    - type: recall_at_3
      value: 0.333
    - type: recall_at_5
      value: 0.333
  - task:
      type: Retrieval
    dataset:
      name: MTEB QuoraRetrieval
      type: quora
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 24.016000000000002
    - type: map_at_10
      value: 28.977999999999998
    - type: map_at_100
      value: 29.579
    - type: map_at_1000
      value: 29.648999999999997
    - type: map_at_3
      value: 27.673
    - type: map_at_5
      value: 28.427000000000003
    - type: mrr_at_1
      value: 27.93
    - type: mrr_at_10
      value: 32.462999999999994
    - type: mrr_at_100
      value: 32.993
    - type: mrr_at_1000
      value: 33.044000000000004
    - type: mrr_at_3
      value: 31.252000000000002
    - type: mrr_at_5
      value: 31.968999999999998
    - type: ndcg_at_1
      value: 27.96
    - type: ndcg_at_10
      value: 31.954
    - type: ndcg_at_100
      value: 34.882000000000005
    - type: ndcg_at_1000
      value: 36.751
    - type: ndcg_at_3
      value: 29.767
    - type: ndcg_at_5
      value: 30.816
    - type: precision_at_1
      value: 27.96
    - type: precision_at_10
      value: 4.826
    - type: precision_at_100
      value: 0.697
    - type: precision_at_1000
      value: 0.093
    - type: precision_at_3
      value: 12.837000000000002
    - type: precision_at_5
      value: 8.559999999999999
    - type: recall_at_1
      value: 24.016000000000002
    - type: recall_at_10
      value: 37.574999999999996
    - type: recall_at_100
      value: 50.843
    - type: recall_at_1000
      value: 64.654
    - type: recall_at_3
      value: 31.182
    - type: recall_at_5
      value: 34.055
  - task:
      type: Clustering
    dataset:
      name: MTEB RedditClustering
      type: mteb/reddit-clustering
      config: default
      split: test
      revision: 24640382cdbf8abc73003fb0fa6d111a705499eb
    metrics:
    - type: v_measure
      value: 18.38048892083281
  - task:
      type: Clustering
    dataset:
      name: MTEB RedditClusteringP2P
      type: mteb/reddit-clustering-p2p
      config: default
      split: test
      revision: 282350215ef01743dc01b456c7f5241fa8937f16
    metrics:
    - type: v_measure
      value: 27.103011764141478
  - task:
      type: Retrieval
    dataset:
      name: MTEB SCIDOCS
      type: scidocs
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 0.18
    - type: map_at_10
      value: 0.457
    - type: map_at_100
      value: 0.634
    - type: map_at_1000
      value: 0.7000000000000001
    - type: map_at_3
      value: 0.333
    - type: map_at_5
      value: 0.387
    - type: mrr_at_1
      value: 0.8999999999999999
    - type: mrr_at_10
      value: 1.967
    - type: mrr_at_100
      value: 2.396
    - type: mrr_at_1000
      value: 2.495
    - type: mrr_at_3
      value: 1.567
    - type: mrr_at_5
      value: 1.7670000000000001
    - type: ndcg_at_1
      value: 0.8999999999999999
    - type: ndcg_at_10
      value: 1.022
    - type: ndcg_at_100
      value: 2.366
    - type: ndcg_at_1000
      value: 4.689
    - type: ndcg_at_3
      value: 0.882
    - type: ndcg_at_5
      value: 0.7929999999999999
    - type: precision_at_1
      value: 0.8999999999999999
    - type: precision_at_10
      value: 0.58
    - type: precision_at_100
      value: 0.263
    - type: precision_at_1000
      value: 0.084
    - type: precision_at_3
      value: 0.8999999999999999
    - type: precision_at_5
      value: 0.74
    - type: recall_at_1
      value: 0.18
    - type: recall_at_10
      value: 1.208
    - type: recall_at_100
      value: 5.373
    - type: recall_at_1000
      value: 17.112
    - type: recall_at_3
      value: 0.5579999999999999
    - type: recall_at_5
      value: 0.7779999999999999
  - task:
      type: STS
    dataset:
      name: MTEB SICK-R
      type: mteb/sickr-sts
      config: default
      split: test
      revision: a6ea5a8cab320b040a23452cc28066d9beae2cee
    metrics:
    - type: cos_sim_pearson
      value: 55.229896309578905
    - type: cos_sim_spearman
      value: 48.54616726085393
    - type: euclidean_pearson
      value: 53.828130644322
    - type: euclidean_spearman
      value: 48.2907441223958
    - type: manhattan_pearson
      value: 53.72684612327582
    - type: manhattan_spearman
      value: 48.228319721712744
  - task:
      type: STS
    dataset:
      name: MTEB STS12
      type: mteb/sts12-sts
      config: default
      split: test
      revision: a0d554a64d88156834ff5ae9920b964011b16384
    metrics:
    - type: cos_sim_pearson
      value: 57.73555535277214
    - type: cos_sim_spearman
      value: 55.58790083939622
    - type: euclidean_pearson
      value: 61.009463373795384
    - type: euclidean_spearman
      value: 56.696846101196044
    - type: manhattan_pearson
      value: 60.875111392597894
    - type: manhattan_spearman
      value: 56.63100766160946
  - task:
      type: STS
    dataset:
      name: MTEB STS13
      type: mteb/sts13-sts
      config: default
      split: test
      revision: 7e90230a92c190f1bf69ae9002b8cea547a64cca
    metrics:
    - type: cos_sim_pearson
      value: 19.47269635955134
    - type: cos_sim_spearman
      value: 18.35951746300603
    - type: euclidean_pearson
      value: 23.130707248318714
    - type: euclidean_spearman
      value: 22.92241668287248
    - type: manhattan_pearson
      value: 22.99371642148021
    - type: manhattan_spearman
      value: 22.770233678121897
  - task:
      type: STS
    dataset:
      name: MTEB STS14
      type: mteb/sts14-sts
      config: default
      split: test
      revision: 6031580fec1f6af667f0bd2da0a551cf4f0b2375
    metrics:
    - type: cos_sim_pearson
      value: 31.78346805351368
    - type: cos_sim_spearman
      value: 28.84281669682782
    - type: euclidean_pearson
      value: 34.508176962091156
    - type: euclidean_spearman
      value: 32.269242265609975
    - type: manhattan_pearson
      value: 34.41366600914297
    - type: manhattan_spearman
      value: 32.15352239729175
  - task:
      type: STS
    dataset:
      name: MTEB STS15
      type: mteb/sts15-sts
      config: default
      split: test
      revision: ae752c7c21bf194d8b67fd573edf7ae58183cbe3
    metrics:
    - type: cos_sim_pearson
      value: 29.550332218260465
    - type: cos_sim_spearman
      value: 29.188654452524528
    - type: euclidean_pearson
      value: 33.80339596511417
    - type: euclidean_spearman
      value: 33.49607278843874
    - type: manhattan_pearson
      value: 33.589427741967334
    - type: manhattan_spearman
      value: 33.288312003652884
  - task:
      type: STS
    dataset:
      name: MTEB STS16
      type: mteb/sts16-sts
      config: default
      split: test
      revision: 4d8694f8f0e0100860b497b999b3dbed754a0513
    metrics:
    - type: cos_sim_pearson
      value: 27.163752699585885
    - type: cos_sim_spearman
      value: 39.0544187582685
    - type: euclidean_pearson
      value: 38.93841642732113
    - type: euclidean_spearman
      value: 42.861814968921294
    - type: manhattan_pearson
      value: 38.78821319739337
    - type: manhattan_spearman
      value: 42.757121435678954
  - task:
      type: STS
    dataset:
      name: MTEB STS17 (en-en)
      type: mteb/sts17-crosslingual-sts
      config: en-en
      split: test
      revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
    metrics:
    - type: cos_sim_pearson
      value: 57.15429605615292
    - type: cos_sim_spearman
      value: 61.21576579300284
    - type: euclidean_pearson
      value: 59.2835939062064
    - type: euclidean_spearman
      value: 60.902713241808236
    - type: manhattan_pearson
      value: 59.510770285546364
    - type: manhattan_spearman
      value: 61.02979474159327
  - task:
      type: STS
    dataset:
      name: MTEB STS22 (en)
      type: mteb/sts22-crosslingual-sts
      config: en
      split: test
      revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
    metrics:
    - type: cos_sim_pearson
      value: 41.81726547830133
    - type: cos_sim_spearman
      value: 44.45123398124273
    - type: euclidean_pearson
      value: 46.44144033159064
    - type: euclidean_spearman
      value: 46.61348337508052
    - type: manhattan_pearson
      value: 46.48092744041165
    - type: manhattan_spearman
      value: 46.78049599791891
  - task:
      type: STS
    dataset:
      name: MTEB STSBenchmark
      type: mteb/stsbenchmark-sts
      config: default
      split: test
      revision: b0fddb56ed78048fa8b90373c8a3cfc37b684831
    metrics:
    - type: cos_sim_pearson
      value: 46.085942179295465
    - type: cos_sim_spearman
      value: 44.394736992467365
    - type: euclidean_pearson
      value: 47.06981069147408
    - type: euclidean_spearman
      value: 45.40499474054004
    - type: manhattan_pearson
      value: 46.96497631950794
    - type: manhattan_spearman
      value: 45.31936619298336
  - task:
      type: Reranking
    dataset:
      name: MTEB SciDocsRR
      type: mteb/scidocs-reranking
      config: default
      split: test
      revision: d3c5e1fc0b855ab6097bf1cda04dd73947d7caab
    metrics:
    - type: map
      value: 43.89526517578129
    - type: mrr
      value: 64.30753070458954
  - task:
      type: Retrieval
    dataset:
      name: MTEB SciFact
      type: scifact
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 1.417
    - type: map_at_10
      value: 2.189
    - type: map_at_100
      value: 2.5669999999999997
    - type: map_at_1000
      value: 2.662
    - type: map_at_3
      value: 1.694
    - type: map_at_5
      value: 1.928
    - type: mrr_at_1
      value: 1.667
    - type: mrr_at_10
      value: 2.4899999999999998
    - type: mrr_at_100
      value: 2.8400000000000003
    - type: mrr_at_1000
      value: 2.928
    - type: mrr_at_3
      value: 1.944
    - type: mrr_at_5
      value: 2.178
    - type: ndcg_at_1
      value: 1.667
    - type: ndcg_at_10
      value: 2.913
    - type: ndcg_at_100
      value: 5.482
    - type: ndcg_at_1000
      value: 8.731
    - type: ndcg_at_3
      value: 1.867
    - type: ndcg_at_5
      value: 2.257
    - type: precision_at_1
      value: 1.667
    - type: precision_at_10
      value: 0.567
    - type: precision_at_100
      value: 0.213
    - type: precision_at_1000
      value: 0.053
    - type: precision_at_3
      value: 0.7779999999999999
    - type: precision_at_5
      value: 0.6669999999999999
    - type: recall_at_1
      value: 1.417
    - type: recall_at_10
      value: 5.028
    - type: recall_at_100
      value: 18.5
    - type: recall_at_1000
      value: 45.072
    - type: recall_at_3
      value: 2.083
    - type: recall_at_5
      value: 3.083
  - task:
      type: PairClassification
    dataset:
      name: MTEB SprintDuplicateQuestions
      type: mteb/sprintduplicatequestions-pairclassification
      config: default
      split: test
      revision: d66bd1f72af766a5cc4b0ca5e00c162f89e8cc46
    metrics:
    - type: cos_sim_accuracy
      value: 99.02871287128713
    - type: cos_sim_ap
      value: 17.404404071912694
    - type: cos_sim_f1
      value: 25.89285714285714
    - type: cos_sim_precision
      value: 29.292929292929294
    - type: cos_sim_recall
      value: 23.200000000000003
    - type: dot_accuracy
      value: 99.0118811881188
    - type: dot_ap
      value: 5.4739000785007335
    - type: dot_f1
      value: 12.178702570379436
    - type: dot_precision
      value: 8.774250440917108
    - type: dot_recall
      value: 19.900000000000002
    - type: euclidean_accuracy
      value: 99.03663366336633
    - type: euclidean_ap
      value: 19.20851069839796
    - type: euclidean_f1
      value: 27.16555612506407
    - type: euclidean_precision
      value: 27.865404837013667
    - type: euclidean_recall
      value: 26.5
    - type: manhattan_accuracy
      value: 99.03663366336633
    - type: manhattan_ap
      value: 19.12862913626528
    - type: manhattan_f1
      value: 26.96629213483146
    - type: manhattan_precision
      value: 28.99884925201381
    - type: manhattan_recall
      value: 25.2
    - type: max_accuracy
      value: 99.03663366336633
    - type: max_ap
      value: 19.20851069839796
    - type: max_f1
      value: 27.16555612506407
  - task:
      type: Clustering
    dataset:
      name: MTEB StackExchangeClustering
      type: mteb/stackexchange-clustering
      config: default
      split: test
      revision: 6cbc1f7b2bc0622f2e39d2c77fa502909748c259
    metrics:
    - type: v_measure
      value: 23.657118721775905
  - task:
      type: Clustering
    dataset:
      name: MTEB StackExchangeClusteringP2P
      type: mteb/stackexchange-clustering-p2p
      config: default
      split: test
      revision: 815ca46b2622cec33ccafc3735d572c266efdb44
    metrics:
    - type: v_measure
      value: 27.343558395037043
  - task:
      type: Reranking
    dataset:
      name: MTEB StackOverflowDupQuestions
      type: mteb/stackoverflowdupquestions-reranking
      config: default
      split: test
      revision: e185fbe320c72810689fc5848eb6114e1ef5ec69
    metrics:
    - type: map
      value: 23.346327148080043
    - type: mrr
      value: 21.99097063067651
  - task:
      type: Retrieval
    dataset:
      name: MTEB TRECCOVID
      type: trec-covid
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 0.032
    - type: map_at_10
      value: 0.157
    - type: map_at_100
      value: 0.583
    - type: map_at_1000
      value: 1.48
    - type: map_at_3
      value: 0.066
    - type: map_at_5
      value: 0.105
    - type: mrr_at_1
      value: 10
    - type: mrr_at_10
      value: 16.99
    - type: mrr_at_100
      value: 18.284
    - type: mrr_at_1000
      value: 18.394
    - type: mrr_at_3
      value: 14.000000000000002
    - type: mrr_at_5
      value: 15.8
    - type: ndcg_at_1
      value: 8
    - type: ndcg_at_10
      value: 7.504
    - type: ndcg_at_100
      value: 5.339
    - type: ndcg_at_1000
      value: 6.046
    - type: ndcg_at_3
      value: 8.358
    - type: ndcg_at_5
      value: 8.142000000000001
    - type: precision_at_1
      value: 10
    - type: precision_at_10
      value: 8.6
    - type: precision_at_100
      value: 5.9799999999999995
    - type: precision_at_1000
      value: 2.976
    - type: precision_at_3
      value: 9.333
    - type: precision_at_5
      value: 9.2
    - type: recall_at_1
      value: 0.032
    - type: recall_at_10
      value: 0.252
    - type: recall_at_100
      value: 1.529
    - type: recall_at_1000
      value: 6.364
    - type: recall_at_3
      value: 0.08499999999999999
    - type: recall_at_5
      value: 0.154
  - task:
      type: Retrieval
    dataset:
      name: MTEB Touche2020
      type: webis-touche2020
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 0.44200000000000006
    - type: map_at_10
      value: 0.996
    - type: map_at_100
      value: 1.317
    - type: map_at_1000
      value: 1.624
    - type: map_at_3
      value: 0.736
    - type: map_at_5
      value: 0.951
    - type: mrr_at_1
      value: 4.082
    - type: mrr_at_10
      value: 10.102
    - type: mrr_at_100
      value: 10.978
    - type: mrr_at_1000
      value: 11.1
    - type: mrr_at_3
      value: 7.8229999999999995
    - type: mrr_at_5
      value: 9.252
    - type: ndcg_at_1
      value: 4.082
    - type: ndcg_at_10
      value: 3.821
    - type: ndcg_at_100
      value: 5.682
    - type: ndcg_at_1000
      value: 10.96
    - type: ndcg_at_3
      value: 4.813
    - type: ndcg_at_5
      value: 4.757
    - type: precision_at_1
      value: 4.082
    - type: precision_at_10
      value: 3.061
    - type: precision_at_100
      value: 1.367
    - type: precision_at_1000
      value: 0.46299999999999997
    - type: precision_at_3
      value: 4.7620000000000005
    - type: precision_at_5
      value: 4.898000000000001
    - type: recall_at_1
      value: 0.44200000000000006
    - type: recall_at_10
      value: 2.059
    - type: recall_at_100
      value: 7.439
    - type: recall_at_1000
      value: 25.191000000000003
    - type: recall_at_3
      value: 1.095
    - type: recall_at_5
      value: 1.725
  - task:
      type: Classification
    dataset:
      name: MTEB ToxicConversationsClassification
      type: mteb/toxic_conversations_50k
      config: default
      split: test
      revision: d7c0de2777da35d6aae2200a62c6e0e5af397c4c
    metrics:
    - type: accuracy
      value: 54.925999999999995
    - type: ap
      value: 9.658236434063275
    - type: f1
      value: 43.469829154993064
  - task:
      type: Classification
    dataset:
      name: MTEB TweetSentimentExtractionClassification
      type: mteb/tweet_sentiment_extraction
      config: default
      split: test
      revision: d604517c81ca91fe16a244d1248fc021f9ecee7a
    metrics:
    - type: accuracy
      value: 40.7498585172609
    - type: f1
      value: 40.720120106546574
  - task:
      type: Clustering
    dataset:
      name: MTEB TwentyNewsgroupsClustering
      type: mteb/twentynewsgroups-clustering
      config: default
      split: test
      revision: 6125ec4e24fa026cec8a478383ee943acfbd5449
    metrics:
    - type: v_measure
      value: 20.165152514024733
  - task:
      type: PairClassification
    dataset:
      name: MTEB TwitterSemEval2015
      type: mteb/twittersemeval2015-pairclassification
      config: default
      split: test
      revision: 70970daeab8776df92f5ea462b6173c0b46fd2d1
    metrics:
    - type: cos_sim_accuracy
      value: 77.59432556476128
    - type: cos_sim_ap
      value: 30.37846072188074
    - type: cos_sim_f1
      value: 37.9231242656521
    - type: cos_sim_precision
      value: 24.064474898814172
    - type: cos_sim_recall
      value: 89.41952506596306
    - type: dot_accuracy
      value: 77.42146986946415
    - type: dot_ap
      value: 24.073476661930034
    - type: dot_f1
      value: 37.710580857735025
    - type: dot_precision
      value: 23.61083383243495
    - type: dot_recall
      value: 93.61477572559367
    - type: euclidean_accuracy
      value: 77.64797043571556
    - type: euclidean_ap
      value: 31.892152386237594
    - type: euclidean_f1
      value: 38.21154759481647
    - type: euclidean_precision
      value: 25.719243766554023
    - type: euclidean_recall
      value: 74.30079155672823
    - type: manhattan_accuracy
      value: 77.6539309769327
    - type: manhattan_ap
      value: 31.89545356309865
    - type: manhattan_f1
      value: 38.16428166172855
    - type: manhattan_precision
      value: 25.07247577238466
    - type: manhattan_recall
      value: 79.86807387862797
    - type: max_accuracy
      value: 77.6539309769327
    - type: max_ap
      value: 31.89545356309865
    - type: max_f1
      value: 38.21154759481647
  - task:
      type: PairClassification
    dataset:
      name: MTEB TwitterURLCorpus
      type: mteb/twitterurlcorpus-pairclassification
      config: default
      split: test
      revision: 8b6510b0b1fa4e4c4f879467980e9be563ec1cdf
    metrics:
    - type: cos_sim_accuracy
      value: 76.56886715566422
    - type: cos_sim_ap
      value: 44.04480929059786
    - type: cos_sim_f1
      value: 43.73100054674686
    - type: cos_sim_precision
      value: 30.540367168647098
    - type: cos_sim_recall
      value: 76.97874961502926
    - type: dot_accuracy
      value: 74.80110218496526
    - type: dot_ap
      value: 26.487746384962758
    - type: dot_f1
      value: 40.91940608182585
    - type: dot_precision
      value: 25.9157358738502
    - type: dot_recall
      value: 97.18201416692331
    - type: euclidean_accuracy
      value: 76.97054371870998
    - type: euclidean_ap
      value: 47.079120397438416
    - type: euclidean_f1
      value: 45.866182572614115
    - type: euclidean_precision
      value: 34.580791490692945
    - type: euclidean_recall
      value: 68.0859254696643
    - type: manhattan_accuracy
      value: 76.96084138626927
    - type: manhattan_ap
      value: 47.168701873575976
    - type: manhattan_f1
      value: 45.985439966237614
    - type: manhattan_precision
      value: 34.974321938693635
    - type: manhattan_recall
      value: 67.11579919926086
    - type: max_accuracy
      value: 76.97054371870998
    - type: max_ap
      value: 47.168701873575976
    - type: max_f1
      value: 45.985439966237614
  - task:
      type: STS
    dataset:
      name: MTEB AFQMC
      type: C-MTEB/AFQMC
      config: default
      split: validation
      revision: None
    metrics:
    - type: cos_sim_pearson
      value: 3.322530620021471
    - type: cos_sim_spearman
      value: 3.7583567993545195
    - type: euclidean_pearson
      value: 3.743782192206081
    - type: euclidean_spearman
      value: 3.758336694921531
    - type: manhattan_pearson
      value: 3.845233721819267
    - type: manhattan_spearman
      value: 3.8542743797718026
  - task:
      type: STS
    dataset:
      name: MTEB ATEC
      type: C-MTEB/ATEC
      config: default
      split: test
      revision: None
    metrics:
    - type: cos_sim_pearson
      value: 8.552640773272078
    - type: cos_sim_spearman
      value: 10.086360519713061
    - type: euclidean_pearson
      value: 9.902099049347935
    - type: euclidean_spearman
      value: 10.086351512635042
    - type: manhattan_pearson
      value: 9.898006826713932
    - type: manhattan_spearman
      value: 10.076531690161783
  - task:
      type: Classification
    dataset:
      name: MTEB AmazonReviewsClassification (zh)
      type: mteb/amazon_reviews_multi
      config: zh
      split: test
      revision: 1399c76144fd37290681b995c656ef9b2e06e26d
    metrics:
    - type: accuracy
      value: 21.955999999999996
    - type: f1
      value: 20.596128116112816
  - task:
      type: STS
    dataset:
      name: MTEB BQ
      type: C-MTEB/BQ
      config: default
      split: test
      revision: None
    metrics:
    - type: cos_sim_pearson
      value: 17.6945509937099
    - type: cos_sim_spearman
      value: 19.312286927022825
    - type: euclidean_pearson
      value: 19.259393744977515
    - type: euclidean_spearman
      value: 19.312290390892713
    - type: manhattan_pearson
      value: 19.223527109645772
    - type: manhattan_spearman
      value: 19.32655209742963
  - task:
      type: Clustering
    dataset:
      name: MTEB CLSClusteringP2P
      type: C-MTEB/CLSClusteringP2P
      config: default
      split: test
      revision: None
    metrics:
    - type: v_measure
      value: 18.657841790313405
  - task:
      type: Clustering
    dataset:
      name: MTEB CLSClusteringS2S
      type: C-MTEB/CLSClusteringS2S
      config: default
      split: test
      revision: None
    metrics:
    - type: v_measure
      value: 16.82483158478091
  - task:
      type: Reranking
    dataset:
      name: MTEB CMedQAv1
      type: C-MTEB/CMedQAv1-reranking
      config: default
      split: test
      revision: None
    metrics:
    - type: map
      value: 19.71658789133091
    - type: mrr
      value: 23.480595238095237
  - task:
      type: Reranking
    dataset:
      name: MTEB CMedQAv2
      type: C-MTEB/CMedQAv2-reranking
      config: default
      split: test
      revision: None
    metrics:
    - type: map
      value: 22.475972401039495
    - type: mrr
      value: 25.993650793650797
  - task:
      type: Retrieval
    dataset:
      name: MTEB CmedqaRetrieval
      type: C-MTEB/CmedqaRetrieval
      config: default
      split: dev
      revision: None
    metrics:
    - type: map_at_1
      value: 1.026
    - type: map_at_10
      value: 1.6389999999999998
    - type: map_at_100
      value: 1.875
    - type: map_at_1000
      value: 1.9529999999999998
    - type: map_at_3
      value: 1.417
    - type: map_at_5
      value: 1.5110000000000001
    - type: mrr_at_1
      value: 1.525
    - type: mrr_at_10
      value: 2.478
    - type: mrr_at_100
      value: 2.779
    - type: mrr_at_1000
      value: 2.861
    - type: mrr_at_3
      value: 2.105
    - type: mrr_at_5
      value: 2.283
    - type: ndcg_at_1
      value: 1.525
    - type: ndcg_at_10
      value: 2.222
    - type: ndcg_at_100
      value: 3.81
    - type: ndcg_at_1000
      value: 6.465999999999999
    - type: ndcg_at_3
      value: 1.7489999999999999
    - type: ndcg_at_5
      value: 1.8980000000000001
    - type: precision_at_1
      value: 1.525
    - type: precision_at_10
      value: 0.543
    - type: precision_at_100
      value: 0.187
    - type: precision_at_1000
      value: 0.055
    - type: precision_at_3
      value: 0.992
    - type: precision_at_5
      value: 0.76
    - type: recall_at_1
      value: 1.026
    - type: recall_at_10
      value: 3.1780000000000004
    - type: recall_at_100
      value: 10.481
    - type: recall_at_1000
      value: 29.735
    - type: recall_at_3
      value: 1.8849999999999998
    - type: recall_at_5
      value: 2.2560000000000002
  - task:
      type: PairClassification
    dataset:
      name: MTEB Cmnli
      type: C-MTEB/CMNLI
      config: default
      split: validation
      revision: None
    metrics:
    - type: cos_sim_accuracy
      value: 54.99699338544799
    - type: cos_sim_ap
      value: 57.78007274332544
    - type: cos_sim_f1
      value: 67.95391338895512
    - type: cos_sim_precision
      value: 51.46846413095811
    - type: cos_sim_recall
      value: 99.9766191255553
    - type: dot_accuracy
      value: 54.99699338544799
    - type: dot_ap
      value: 57.7791056074979
    - type: dot_f1
      value: 67.95391338895512
    - type: dot_precision
      value: 51.46846413095811
    - type: dot_recall
      value: 99.9766191255553
    - type: euclidean_accuracy
      value: 54.99699338544799
    - type: euclidean_ap
      value: 57.7800760462191
    - type: euclidean_f1
      value: 67.95391338895512
    - type: euclidean_precision
      value: 51.46846413095811
    - type: euclidean_recall
      value: 99.9766191255553
    - type: manhattan_accuracy
      value: 55.05712567648827
    - type: manhattan_ap
      value: 57.8146828916844
    - type: manhattan_f1
      value: 67.95900532295227
    - type: manhattan_precision
      value: 51.46811070998797
    - type: manhattan_recall
      value: 100
    - type: max_accuracy
      value: 55.05712567648827
    - type: max_ap
      value: 57.8146828916844
    - type: max_f1
      value: 67.95900532295227
  - task:
      type: Retrieval
    dataset:
      name: MTEB CovidRetrieval
      type: C-MTEB/CovidRetrieval
      config: default
      split: dev
      revision: None
    metrics:
    - type: map_at_1
      value: 0.632
    - type: map_at_10
      value: 1.7510000000000001
    - type: map_at_100
      value: 2.004
    - type: map_at_1000
      value: 2.0660000000000003
    - type: map_at_3
      value: 1.493
    - type: map_at_5
      value: 1.635
    - type: mrr_at_1
      value: 0.632
    - type: mrr_at_10
      value: 1.7670000000000001
    - type: mrr_at_100
      value: 2.02
    - type: mrr_at_1000
      value: 2.081
    - type: mrr_at_3
      value: 1.528
    - type: mrr_at_5
      value: 1.649
    - type: ndcg_at_1
      value: 0.632
    - type: ndcg_at_10
      value: 2.32
    - type: ndcg_at_100
      value: 3.758
    - type: ndcg_at_1000
      value: 5.894
    - type: ndcg_at_3
      value: 1.7850000000000001
    - type: ndcg_at_5
      value: 2.044
    - type: precision_at_1
      value: 0.632
    - type: precision_at_10
      value: 0.411
    - type: precision_at_100
      value: 0.11399999999999999
    - type: precision_at_1000
      value: 0.03
    - type: precision_at_3
      value: 0.878
    - type: precision_at_5
      value: 0.653
    - type: recall_at_1
      value: 0.632
    - type: recall_at_10
      value: 4.109999999999999
    - type: recall_at_100
      value: 11.222
    - type: recall_at_1000
      value: 29.083
    - type: recall_at_3
      value: 2.634
    - type: recall_at_5
      value: 3.267
  - task:
      type: Retrieval
    dataset:
      name: MTEB DuRetrieval
      type: C-MTEB/DuRetrieval
      config: default
      split: dev
      revision: None
    metrics:
    - type: map_at_1
      value: 1.436
    - type: map_at_10
      value: 3.4099999999999997
    - type: map_at_100
      value: 4.128
    - type: map_at_1000
      value: 4.282
    - type: map_at_3
      value: 2.423
    - type: map_at_5
      value: 2.927
    - type: mrr_at_1
      value: 6
    - type: mrr_at_10
      value: 9.701
    - type: mrr_at_100
      value: 10.347000000000001
    - type: mrr_at_1000
      value: 10.427999999999999
    - type: mrr_at_3
      value: 8.267
    - type: mrr_at_5
      value: 9.004
    - type: ndcg_at_1
      value: 6
    - type: ndcg_at_10
      value: 5.856
    - type: ndcg_at_100
      value: 9.063
    - type: ndcg_at_1000
      value: 12.475999999999999
    - type: ndcg_at_3
      value: 5.253
    - type: ndcg_at_5
      value: 5.223
    - type: precision_at_1
      value: 6
    - type: precision_at_10
      value: 3.125
    - type: precision_at_100
      value: 0.812
    - type: precision_at_1000
      value: 0.169
    - type: precision_at_3
      value: 4.7669999999999995
    - type: precision_at_5
      value: 4.15
    - type: recall_at_1
      value: 1.436
    - type: recall_at_10
      value: 6.544999999999999
    - type: recall_at_100
      value: 16.634999999999998
    - type: recall_at_1000
      value: 33.987
    - type: recall_at_3
      value: 3.144
    - type: recall_at_5
      value: 4.519
  - task:
      type: Retrieval
    dataset:
      name: MTEB EcomRetrieval
      type: C-MTEB/EcomRetrieval
      config: default
      split: dev
      revision: None
    metrics:
    - type: map_at_1
      value: 4.1000000000000005
    - type: map_at_10
      value: 7.911
    - type: map_at_100
      value: 8.92
    - type: map_at_1000
      value: 9.033
    - type: map_at_3
      value: 6.4
    - type: map_at_5
      value: 7.23
    - type: mrr_at_1
      value: 4.1000000000000005
    - type: mrr_at_10
      value: 7.911
    - type: mrr_at_100
      value: 8.92
    - type: mrr_at_1000
      value: 9.033
    - type: mrr_at_3
      value: 6.4
    - type: mrr_at_5
      value: 7.23
    - type: ndcg_at_1
      value: 4.1000000000000005
    - type: ndcg_at_10
      value: 10.374
    - type: ndcg_at_100
      value: 15.879999999999999
    - type: ndcg_at_1000
      value: 19.246
    - type: ndcg_at_3
      value: 7.217
    - type: ndcg_at_5
      value: 8.706
    - type: precision_at_1
      value: 4.1000000000000005
    - type: precision_at_10
      value: 1.8399999999999999
    - type: precision_at_100
      value: 0.45599999999999996
    - type: precision_at_1000
      value: 0.073
    - type: precision_at_3
      value: 3.2
    - type: precision_at_5
      value: 2.64
    - type: recall_at_1
      value: 4.1000000000000005
    - type: recall_at_10
      value: 18.4
    - type: recall_at_100
      value: 45.6
    - type: recall_at_1000
      value: 72.89999999999999
    - type: recall_at_3
      value: 9.6
    - type: recall_at_5
      value: 13.200000000000001
  - task:
      type: Classification
    dataset:
      name: MTEB IFlyTek
      type: C-MTEB/IFlyTek-classification
      config: default
      split: validation
      revision: None
    metrics:
    - type: accuracy
      value: 20.353982300884958
    - type: f1
      value: 12.69588085868714
  - task:
      type: Classification
    dataset:
      name: MTEB JDReview
      type: C-MTEB/JDReview-classification
      config: default
      split: test
      revision: None
    metrics:
    - type: accuracy
      value: 55.497185741088174
    - type: ap
      value: 20.43046737602198
    - type: f1
      value: 48.93980371558734
  - task:
      type: STS
    dataset:
      name: MTEB LCQMC
      type: C-MTEB/LCQMC
      config: default
      split: test
      revision: None
    metrics:
    - type: cos_sim_pearson
      value: 32.588967426128654
    - type: cos_sim_spearman
      value: 42.14900040682406
    - type: euclidean_pearson
      value: 39.568373451615685
    - type: euclidean_spearman
      value: 42.14899152396297
    - type: manhattan_pearson
      value: 39.5220710244444
    - type: manhattan_spearman
      value: 42.14787636056146
  - task:
      type: Reranking
    dataset:
      name: MTEB MMarcoReranking
      type: C-MTEB/Mmarco-reranking
      config: default
      split: dev
      revision: None
    metrics:
    - type: map
      value: 1.1655156335725807
    - type: mrr
      value: 0.2361111111111111
  - task:
      type: Retrieval
    dataset:
      name: MTEB MMarcoRetrieval
      type: C-MTEB/MMarcoRetrieval
      config: default
      split: dev
      revision: None
    metrics:
    - type: map_at_1
      value: 1.9029999999999998
    - type: map_at_10
      value: 2.9139999999999997
    - type: map_at_100
      value: 3.2259999999999995
    - type: map_at_1000
      value: 3.2870000000000004
    - type: map_at_3
      value: 2.483
    - type: map_at_5
      value: 2.71
    - type: mrr_at_1
      value: 2.02
    - type: mrr_at_10
      value: 3.064
    - type: mrr_at_100
      value: 3.382
    - type: mrr_at_1000
      value: 3.4419999999999997
    - type: mrr_at_3
      value: 2.622
    - type: mrr_at_5
      value: 2.855
    - type: ndcg_at_1
      value: 2.02
    - type: ndcg_at_10
      value: 3.639
    - type: ndcg_at_100
      value: 5.431
    - type: ndcg_at_1000
      value: 7.404
    - type: ndcg_at_3
      value: 2.723
    - type: ndcg_at_5
      value: 3.1350000000000002
    - type: precision_at_1
      value: 2.02
    - type: precision_at_10
      value: 0.626
    - type: precision_at_100
      value: 0.159
    - type: precision_at_1000
      value: 0.033
    - type: precision_at_3
      value: 1.17
    - type: precision_at_5
      value: 0.9199999999999999
    - type: recall_at_1
      value: 1.9029999999999998
    - type: recall_at_10
      value: 5.831
    - type: recall_at_100
      value: 14.737
    - type: recall_at_1000
      value: 30.84
    - type: recall_at_3
      value: 3.2870000000000004
    - type: recall_at_5
      value: 4.282
  - task:
      type: Classification
    dataset:
      name: MTEB MassiveIntentClassification (zh-CN)
      type: mteb/amazon_massive_intent
      config: zh-CN
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 25.3866845998655
    - type: f1
      value: 23.404809615998495
  - task:
      type: Classification
    dataset:
      name: MTEB MassiveScenarioClassification (zh-CN)
      type: mteb/amazon_massive_scenario
      config: zh-CN
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 40.34969737726966
    - type: f1
      value: 37.88244646590394
  - task:
      type: Retrieval
    dataset:
      name: MTEB MedicalRetrieval
      type: C-MTEB/MedicalRetrieval
      config: default
      split: dev
      revision: None
    metrics:
    - type: map_at_1
      value: 1.5
    - type: map_at_10
      value: 2.0740000000000003
    - type: map_at_100
      value: 2.2079999999999997
    - type: map_at_1000
      value: 2.241
    - type: map_at_3
      value: 1.933
    - type: map_at_5
      value: 2.023
    - type: mrr_at_1
      value: 1.5
    - type: mrr_at_10
      value: 2.0740000000000003
    - type: mrr_at_100
      value: 2.2079999999999997
    - type: mrr_at_1000
      value: 2.241
    - type: mrr_at_3
      value: 1.933
    - type: mrr_at_5
      value: 2.023
    - type: ndcg_at_1
      value: 1.5
    - type: ndcg_at_10
      value: 2.368
    - type: ndcg_at_100
      value: 3.309
    - type: ndcg_at_1000
      value: 4.593
    - type: ndcg_at_3
      value: 2.0789999999999997
    - type: ndcg_at_5
      value: 2.242
    - type: precision_at_1
      value: 1.5
    - type: precision_at_10
      value: 0.33
    - type: precision_at_100
      value: 0.084
    - type: precision_at_1000
      value: 0.019
    - type: precision_at_3
      value: 0.8330000000000001
    - type: precision_at_5
      value: 0.58
    - type: recall_at_1
      value: 1.5
    - type: recall_at_10
      value: 3.3000000000000003
    - type: recall_at_100
      value: 8.4
    - type: recall_at_1000
      value: 19.400000000000002
    - type: recall_at_3
      value: 2.5
    - type: recall_at_5
      value: 2.9000000000000004
  - task:
      type: Classification
    dataset:
      name: MTEB MultilingualSentiment
      type: C-MTEB/MultilingualSentiment-classification
      config: default
      split: validation
      revision: None
    metrics:
    - type: accuracy
      value: 38.94
    - type: f1
      value: 38.4171730136538
  - task:
      type: PairClassification
    dataset:
      name: MTEB Ocnli
      type: C-MTEB/OCNLI
      config: default
      split: validation
      revision: None
    metrics:
    - type: cos_sim_accuracy
      value: 54.141851651326476
    - type: cos_sim_ap
      value: 55.63298007661861
    - type: cos_sim_f1
      value: 67.85195936139333
    - type: cos_sim_precision
      value: 51.68601437258153
    - type: cos_sim_recall
      value: 98.73284054910243
    - type: dot_accuracy
      value: 54.141851651326476
    - type: dot_ap
      value: 55.63298007661861
    - type: dot_f1
      value: 67.85195936139333
    - type: dot_precision
      value: 51.68601437258153
    - type: dot_recall
      value: 98.73284054910243
    - type: euclidean_accuracy
      value: 54.141851651326476
    - type: euclidean_ap
      value: 55.63298007661861
    - type: euclidean_f1
      value: 67.85195936139333
    - type: euclidean_precision
      value: 51.68601437258153
    - type: euclidean_recall
      value: 98.73284054910243
    - type: manhattan_accuracy
      value: 54.03356794802382
    - type: manhattan_ap
      value: 55.650247173847944
    - type: manhattan_f1
      value: 67.83667621776503
    - type: manhattan_precision
      value: 51.32791327913279
    - type: manhattan_recall
      value: 100
    - type: max_accuracy
      value: 54.141851651326476
    - type: max_ap
      value: 55.650247173847944
    - type: max_f1
      value: 67.85195936139333
  - task:
      type: Classification
    dataset:
      name: MTEB OnlineShopping
      type: C-MTEB/OnlineShopping-classification
      config: default
      split: test
      revision: None
    metrics:
    - type: accuracy
      value: 56.88999999999999
    - type: ap
      value: 56.075855594697835
    - type: f1
      value: 56.31094564241924
  - task:
      type: STS
    dataset:
      name: MTEB PAWSX
      type: C-MTEB/PAWSX
      config: default
      split: test
      revision: None
    metrics:
    - type: cos_sim_pearson
      value: 10.023575042969506
    - type: cos_sim_spearman
      value: 6.135169971774927
    - type: euclidean_pearson
      value: 9.219072035876794
    - type: euclidean_spearman
      value: 6.147945631319713
    - type: manhattan_pearson
      value: 9.208267921398097
    - type: manhattan_spearman
      value: 6.156480815791583
  - task:
      type: STS
    dataset:
      name: MTEB QBQTC
      type: C-MTEB/QBQTC
      config: default
      split: test
      revision: None
    metrics:
    - type: cos_sim_pearson
      value: 5.7230819885069435
    - type: cos_sim_spearman
      value: 6.116111130034651
    - type: euclidean_pearson
      value: 5.9142712292657205
    - type: euclidean_spearman
      value: 6.115732664912588
    - type: manhattan_pearson
      value: 5.892970378623552
    - type: manhattan_spearman
      value: 6.100463075081302
  - task:
      type: STS
    dataset:
      name: MTEB STS22 (zh)
      type: mteb/sts22-crosslingual-sts
      config: zh
      split: test
      revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
    metrics:
    - type: cos_sim_pearson
      value: 18.353401358720397
    - type: cos_sim_spearman
      value: 33.700002511275095
    - type: euclidean_pearson
      value: 27.654605278731136
    - type: euclidean_spearman
      value: 33.700002511275095
    - type: manhattan_pearson
      value: 29.174977260571083
    - type: manhattan_spearman
      value: 33.901862553268366
  - task:
      type: STS
    dataset:
      name: MTEB STSB
      type: C-MTEB/STSB
      config: default
      split: test
      revision: None
    metrics:
    - type: cos_sim_pearson
      value: 44.66287398363386
    - type: cos_sim_spearman
      value: 45.60317964713117
    - type: euclidean_pearson
      value: 47.434263079423
    - type: euclidean_spearman
      value: 45.603111040461606
    - type: manhattan_pearson
      value: 47.3272049502668
    - type: manhattan_spearman
      value: 45.506449459872805
  - task:
      type: Reranking
    dataset:
      name: MTEB T2Reranking
      type: C-MTEB/T2Reranking
      config: default
      split: dev
      revision: None
    metrics:
    - type: map
      value: 60.05480951659048
    - type: mrr
      value: 69.58201013422746
  - task:
      type: Retrieval
    dataset:
      name: MTEB T2Retrieval
      type: C-MTEB/T2Retrieval
      config: default
      split: dev
      revision: None
    metrics:
    - type: map_at_1
      value: 1.159
    - type: map_at_10
      value: 2.624
    - type: map_at_100
      value: 3.259
    - type: map_at_1000
      value: 3.4090000000000003
    - type: map_at_3
      value: 1.9109999999999998
    - type: map_at_5
      value: 2.254
    - type: mrr_at_1
      value: 5.87
    - type: mrr_at_10
      value: 8.530999999999999
    - type: mrr_at_100
      value: 9.142999999999999
    - type: mrr_at_1000
      value: 9.229
    - type: mrr_at_3
      value: 7.498
    - type: mrr_at_5
      value: 8.056000000000001
    - type: ndcg_at_1
      value: 5.87
    - type: ndcg_at_10
      value: 4.641
    - type: ndcg_at_100
      value: 7.507999999999999
    - type: ndcg_at_1000
      value: 10.823
    - type: ndcg_at_3
      value: 4.775
    - type: ndcg_at_5
      value: 4.515000000000001
    - type: precision_at_1
      value: 5.87
    - type: precision_at_10
      value: 2.632
    - type: precision_at_100
      value: 0.762
    - type: precision_at_1000
      value: 0.166
    - type: precision_at_3
      value: 4.2299999999999995
    - type: precision_at_5
      value: 3.5450000000000004
    - type: recall_at_1
      value: 1.159
    - type: recall_at_10
      value: 4.816
    - type: recall_at_100
      value: 13.841999999999999
    - type: recall_at_1000
      value: 30.469
    - type: recall_at_3
      value: 2.413
    - type: recall_at_5
      value: 3.3300000000000005
  - task:
      type: Classification
    dataset:
      name: MTEB TNews
      type: C-MTEB/TNews-classification
      config: default
      split: validation
      revision: None
    metrics:
    - type: accuracy
      value: 26.786000000000005
    - type: f1
      value: 25.70512339530705
  - task:
      type: Clustering
    dataset:
      name: MTEB ThuNewsClusteringP2P
      type: C-MTEB/ThuNewsClusteringP2P
      config: default
      split: test
      revision: None
    metrics:
    - type: v_measure
      value: 20.691386720429243
  - task:
      type: Clustering
    dataset:
      name: MTEB ThuNewsClusteringS2S
      type: C-MTEB/ThuNewsClusteringS2S
      config: default
      split: test
      revision: None
    metrics:
    - type: v_measure
      value: 17.1882521768033
  - task:
      type: Retrieval
    dataset:
      name: MTEB VideoRetrieval
      type: C-MTEB/VideoRetrieval
      config: default
      split: dev
      revision: None
    metrics:
    - type: map_at_1
      value: 2.9000000000000004
    - type: map_at_10
      value: 4.051
    - type: map_at_100
      value: 4.277
    - type: map_at_1000
      value: 4.315
    - type: map_at_3
      value: 3.567
    - type: map_at_5
      value: 3.897
    - type: mrr_at_1
      value: 2.9000000000000004
    - type: mrr_at_10
      value: 4.051
    - type: mrr_at_100
      value: 4.277
    - type: mrr_at_1000
      value: 4.315
    - type: mrr_at_3
      value: 3.567
    - type: mrr_at_5
      value: 3.897
    - type: ndcg_at_1
      value: 2.9000000000000004
    - type: ndcg_at_10
      value: 4.772
    - type: ndcg_at_100
      value: 6.214
    - type: ndcg_at_1000
      value: 7.456
    - type: ndcg_at_3
      value: 3.805
    - type: ndcg_at_5
      value: 4.390000000000001
    - type: precision_at_1
      value: 2.9000000000000004
    - type: precision_at_10
      value: 0.7100000000000001
    - type: precision_at_100
      value: 0.146
    - type: precision_at_1000
      value: 0.025
    - type: precision_at_3
      value: 1.5
    - type: precision_at_5
      value: 1.18
    - type: recall_at_1
      value: 2.9000000000000004
    - type: recall_at_10
      value: 7.1
    - type: recall_at_100
      value: 14.6
    - type: recall_at_1000
      value: 24.9
    - type: recall_at_3
      value: 4.5
    - type: recall_at_5
      value: 5.8999999999999995
  - task:
      type: Classification
    dataset:
      name: MTEB Waimai
      type: C-MTEB/waimai-classification
      config: default
      split: test
      revision: None
    metrics:
    - type: accuracy
      value: 56.21999999999999
    - type: ap
      value: 36.53654363772411
    - type: f1
      value: 54.922396485449674
---

# {MODEL_NAME}

This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 384 dimensional dense vector space and can be used for tasks like clustering or semantic search.

<!--- Describe your model here -->

## Usage (Sentence-Transformers)

Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:

```
pip install -U sentence-transformers
```

Then you can use the model like this:

```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('{MODEL_NAME}')
embeddings = model.encode(sentences)
print(embeddings)
```



## Usage (HuggingFace Transformers)
Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.

```python
from transformers import AutoTokenizer, AutoModel
import torch


#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)


# Sentences we want sentence embeddings for
sentences = ['This is an example sentence', 'Each sentence is converted']

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('{MODEL_NAME}')
model = AutoModel.from_pretrained('{MODEL_NAME}')

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, mean pooling.
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)
```



## Evaluation Results

<!--- Describe how your model was evaluated -->

For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name={MODEL_NAME})


## Training
The model was trained with the parameters:

**DataLoader**:

`torch.utils.data.dataloader.DataLoader` of length 1468721 with parameters:
```
{'batch_size': 160, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
```

**Loss**:

`sentence_transformers.losses.CosineSimilarityLoss.CosineSimilarityLoss` 

Parameters of the fit()-Method:
```
{
    "epochs": 1,
    "evaluation_steps": 0,
    "evaluator": "NoneType",
    "max_grad_norm": 1,
    "optimizer_class": "<class 'torch.optim.adamw.AdamW'>",
    "optimizer_params": {
        "lr": 2e-05
    },
    "scheduler": "WarmupLinear",
    "steps_per_epoch": null,
    "warmup_steps": 100,
    "weight_decay": 0.01
}
```


## Full Model Architecture
```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
```

## Citing & Authors

<!--- Describe where people can find more information -->