sentence-transformers pytorch t5 finetuner feature-extraction sentence-similarity mteb custom_code en dataset:jinaai/negation-dataset arxiv:2307.11224 license:apache-2.0 model-index autotrain_compatible endpoints_compatible region:us
---
datasets:
- jinaai/negation-dataset
language: en
license: apache-2.0
pipeline_tag: sentence-similarity
tags:
- finetuner
- sentence-transformers
- feature-extraction
- sentence-similarity
- mteb
model-index:
- name: jina-embedding-b-en-v1
  results:
  - task:
      type: Classification
    dataset:
      name: MTEB AmazonCounterfactualClassification (en)
      type: mteb/amazon_counterfactual
      config: en
      split: test
      revision: e8379541af4e31359cca9fbcf4b00f2671dba205
    metrics:
    - type: accuracy
      value: 66.73134328358208
    - type: ap
      value: 28.30575908745204
    - type: f1
      value: 60.02420130946191
  - task:
      type: Classification
    dataset:
      name: MTEB AmazonPolarityClassification
      type: mteb/amazon_polarity
      config: default
      split: test
      revision: e2d317d38cd51312af73b3d32a06d1a08b442046
    metrics:
    - type: accuracy
      value: 67.6068
    - type: ap
      value: 63.5899352938589
    - type: f1
      value: 65.64285334357656
  - task:
      type: Classification
    dataset:
      name: MTEB AmazonReviewsClassification (en)
      type: mteb/amazon_reviews_multi
      config: en
      split: test
      revision: 1399c76144fd37290681b995c656ef9b2e06e26d
    metrics:
    - type: accuracy
      value: 31.178
    - type: f1
      value: 29.68460843733487
  - task:
      type: Retrieval
    dataset:
      name: MTEB ArguAna
      type: arguana
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 24.964
    - type: map_at_10
      value: 40.217999999999996
    - type: map_at_100
      value: 41.263
    - type: map_at_1000
      value: 41.277
    - type: map_at_3
      value: 35.183
    - type: map_at_5
      value: 38.045
    - type: mrr_at_1
      value: 25.107000000000003
    - type: mrr_at_10
      value: 40.272999999999996
    - type: mrr_at_100
      value: 41.318
    - type: mrr_at_1000
      value: 41.333
    - type: mrr_at_3
      value: 35.242000000000004
    - type: mrr_at_5
      value: 38.101
    - type: ndcg_at_1
      value: 24.964
    - type: ndcg_at_10
      value: 49.006
    - type: ndcg_at_100
      value: 53.446000000000005
    - type: ndcg_at_1000
      value: 53.813
    - type: ndcg_at_3
      value: 38.598
    - type: ndcg_at_5
      value: 43.74
    - type: precision_at_1
      value: 24.964
    - type: precision_at_10
      value: 7.724
    - type: precision_at_100
      value: 0.966
    - type: precision_at_1000
      value: 0.099
    - type: precision_at_3
      value: 16.169
    - type: precision_at_5
      value: 12.191
    - type: recall_at_1
      value: 24.964
    - type: recall_at_10
      value: 77.24
    - type: recall_at_100
      value: 96.586
    - type: recall_at_1000
      value: 99.431
    - type: recall_at_3
      value: 48.506
    - type: recall_at_5
      value: 60.953
  - task:
      type: Clustering
    dataset:
      name: MTEB ArxivClusteringP2P
      type: mteb/arxiv-clustering-p2p
      config: default
      split: test
      revision: a122ad7f3f0291bf49cc6f4d32aa80929df69d5d
    metrics:
    - type: v_measure
      value: 39.25203906042786
  - task:
      type: Clustering
    dataset:
      name: MTEB ArxivClusteringS2S
      type: mteb/arxiv-clustering-s2s
      config: default
      split: test
      revision: f910caf1a6075f7329cdf8c1a6135696f37dbd53
    metrics:
    - type: v_measure
      value: 29.07648348376354
  - task:
      type: Reranking
    dataset:
      name: MTEB AskUbuntuDupQuestions
      type: mteb/askubuntudupquestions-reranking
      config: default
      split: test
      revision: 2000358ca161889fa9c082cb41daa8dcfb161a54
    metrics:
    - type: map
      value: 62.4029266143623
    - type: mrr
      value: 75.45750340764191
  - task:
      type: STS
    dataset:
      name: MTEB BIOSSES
      type: mteb/biosses-sts
      config: default
      split: test
      revision: d3fb88f8f02e40887cd149695127462bbcf29b4a
    metrics:
    - type: cos_sim_pearson
      value: 85.92280995704714
    - type: cos_sim_spearman
      value: 83.58082010833608
    - type: euclidean_pearson
      value: 48.64744162695948
    - type: euclidean_spearman
      value: 48.817377397301556
    - type: manhattan_pearson
      value: 48.87684776623195
    - type: manhattan_spearman
      value: 48.94268145725884
  - task:
      type: Classification
    dataset:
      name: MTEB Banking77Classification
      type: mteb/banking77
      config: default
      split: test
      revision: 0fd18e25b25c072e09e0d92ab615fda904d66300
    metrics:
    - type: accuracy
      value: 84.05519480519482
    - type: f1
      value: 83.94978356890618
  - task:
      type: Clustering
    dataset:
      name: MTEB BiorxivClusteringP2P
      type: mteb/biorxiv-clustering-p2p
      config: default
      split: test
      revision: 65b79d1d13f80053f67aca9498d9402c2d9f1f40
    metrics:
    - type: v_measure
      value: 32.2033276486685
  - task:
      type: Clustering
    dataset:
      name: MTEB BiorxivClusteringS2S
      type: mteb/biorxiv-clustering-s2s
      config: default
      split: test
      revision: 258694dd0231531bc1fd9de6ceb52a0853c6d908
    metrics:
    - type: v_measure
      value: 26.631954164406014
  - task:
      type: Retrieval
    dataset:
      name: MTEB CQADupstackAndroidRetrieval
      type: BeIR/cqadupstack
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 29.625
    - type: map_at_10
      value: 40.037
    - type: map_at_100
      value: 41.52
    - type: map_at_1000
      value: 41.654
    - type: map_at_3
      value: 36.818
    - type: map_at_5
      value: 38.426
    - type: mrr_at_1
      value: 35.336
    - type: mrr_at_10
      value: 45.395
    - type: mrr_at_100
      value: 46.221000000000004
    - type: mrr_at_1000
      value: 46.264
    - type: mrr_at_3
      value: 42.823
    - type: mrr_at_5
      value: 44.204
    - type: ndcg_at_1
      value: 35.336
    - type: ndcg_at_10
      value: 46.326
    - type: ndcg_at_100
      value: 51.795
    - type: ndcg_at_1000
      value: 53.834
    - type: ndcg_at_3
      value: 41.299
    - type: ndcg_at_5
      value: 43.247
    - type: precision_at_1
      value: 35.336
    - type: precision_at_10
      value: 8.627
    - type: precision_at_100
      value: 1.428
    - type: precision_at_1000
      value: 0.197
    - type: precision_at_3
      value: 19.647000000000002
    - type: precision_at_5
      value: 13.733999999999998
    - type: recall_at_1
      value: 29.625
    - type: recall_at_10
      value: 59.165
    - type: recall_at_100
      value: 81.675
    - type: recall_at_1000
      value: 94.17
    - type: recall_at_3
      value: 44.485
    - type: recall_at_5
      value: 50.198
    - type: map_at_1
      value: 26.687
    - type: map_at_10
      value: 36.062
    - type: map_at_100
      value: 37.263000000000005
    - type: map_at_1000
      value: 37.397999999999996
    - type: map_at_3
      value: 32.967
    - type: map_at_5
      value: 34.75
    - type: mrr_at_1
      value: 33.885
    - type: mrr_at_10
      value: 42.632999999999996
    - type: mrr_at_100
      value: 43.305
    - type: mrr_at_1000
      value: 43.354
    - type: mrr_at_3
      value: 39.958
    - type: mrr_at_5
      value: 41.63
    - type: ndcg_at_1
      value: 33.885
    - type: ndcg_at_10
      value: 42.001
    - type: ndcg_at_100
      value: 46.436
    - type: ndcg_at_1000
      value: 48.774
    - type: ndcg_at_3
      value: 37.183
    - type: ndcg_at_5
      value: 39.605000000000004
    - type: precision_at_1
      value: 33.885
    - type: precision_at_10
      value: 7.962
    - type: precision_at_100
      value: 1.283
    - type: precision_at_1000
      value: 0.18
    - type: precision_at_3
      value: 17.855999999999998
    - type: precision_at_5
      value: 13.083
    - type: recall_at_1
      value: 26.687
    - type: recall_at_10
      value: 52.75
    - type: recall_at_100
      value: 71.324
    - type: recall_at_1000
      value: 86.356
    - type: recall_at_3
      value: 38.83
    - type: recall_at_5
      value: 45.23
    - type: map_at_1
      value: 34.02
    - type: map_at_10
      value: 45.751999999999995
    - type: map_at_100
      value: 46.867
    - type: map_at_1000
      value: 46.93
    - type: map_at_3
      value: 42.409
    - type: map_at_5
      value: 44.464999999999996
    - type: mrr_at_1
      value: 38.307
    - type: mrr_at_10
      value: 48.718
    - type: mrr_at_100
      value: 49.509
    - type: mrr_at_1000
      value: 49.542
    - type: mrr_at_3
      value: 46.007999999999996
    - type: mrr_at_5
      value: 47.766999999999996
    - type: ndcg_at_1
      value: 38.307
    - type: ndcg_at_10
      value: 51.666999999999994
    - type: ndcg_at_100
      value: 56.242000000000004
    - type: ndcg_at_1000
      value: 57.477999999999994
    - type: ndcg_at_3
      value: 45.912
    - type: ndcg_at_5
      value: 49.106
    - type: precision_at_1
      value: 38.307
    - type: precision_at_10
      value: 8.476
    - type: precision_at_100
      value: 1.176
    - type: precision_at_1000
      value: 0.133
    - type: precision_at_3
      value: 20.522000000000002
    - type: precision_at_5
      value: 14.557999999999998
    - type: recall_at_1
      value: 34.02
    - type: recall_at_10
      value: 66.046
    - type: recall_at_100
      value: 85.817
    - type: recall_at_1000
      value: 94.453
    - type: recall_at_3
      value: 51.059
    - type: recall_at_5
      value: 58.667
    - type: map_at_1
      value: 23.939
    - type: map_at_10
      value: 32.627
    - type: map_at_100
      value: 33.617999999999995
    - type: map_at_1000
      value: 33.701
    - type: map_at_3
      value: 30.11
    - type: map_at_5
      value: 31.380000000000003
    - type: mrr_at_1
      value: 25.989
    - type: mrr_at_10
      value: 34.655
    - type: mrr_at_100
      value: 35.502
    - type: mrr_at_1000
      value: 35.563
    - type: mrr_at_3
      value: 32.109
    - type: mrr_at_5
      value: 33.426
    - type: ndcg_at_1
      value: 25.989
    - type: ndcg_at_10
      value: 37.657000000000004
    - type: ndcg_at_100
      value: 42.467
    - type: ndcg_at_1000
      value: 44.677
    - type: ndcg_at_3
      value: 32.543
    - type: ndcg_at_5
      value: 34.74
    - type: precision_at_1
      value: 25.989
    - type: precision_at_10
      value: 5.876
    - type: precision_at_100
      value: 0.8710000000000001
    - type: precision_at_1000
      value: 0.11
    - type: precision_at_3
      value: 13.861
    - type: precision_at_5
      value: 9.626999999999999
    - type: recall_at_1
      value: 23.939
    - type: recall_at_10
      value: 51.28
    - type: recall_at_100
      value: 73.428
    - type: recall_at_1000
      value: 90.309
    - type: recall_at_3
      value: 37.245
    - type: recall_at_5
      value: 42.541000000000004
    - type: map_at_1
      value: 15.082
    - type: map_at_10
      value: 22.486
    - type: map_at_100
      value: 23.687
    - type: map_at_1000
      value: 23.807000000000002
    - type: map_at_3
      value: 20.076
    - type: map_at_5
      value: 21.362000000000002
    - type: mrr_at_1
      value: 18.532
    - type: mrr_at_10
      value: 26.605
    - type: mrr_at_100
      value: 27.628999999999998
    - type: mrr_at_1000
      value: 27.698
    - type: mrr_at_3
      value: 23.964
    - type: mrr_at_5
      value: 25.319000000000003
    - type: ndcg_at_1
      value: 18.532
    - type: ndcg_at_10
      value: 27.474999999999998
    - type: ndcg_at_100
      value: 33.357
    - type: ndcg_at_1000
      value: 36.361
    - type: ndcg_at_3
      value: 22.851
    - type: ndcg_at_5
      value: 24.87
    - type: precision_at_1
      value: 18.532
    - type: precision_at_10
      value: 5.210999999999999
    - type: precision_at_100
      value: 0.9329999999999999
    - type: precision_at_1000
      value: 0.134
    - type: precision_at_3
      value: 11.235000000000001
    - type: precision_at_5
      value: 8.134
    - type: recall_at_1
      value: 15.082
    - type: recall_at_10
      value: 38.759
    - type: recall_at_100
      value: 64.621
    - type: recall_at_1000
      value: 86.162
    - type: recall_at_3
      value: 26.055
    - type: recall_at_5
      value: 31.208999999999996
    - type: map_at_1
      value: 24.759999999999998
    - type: map_at_10
      value: 33.706
    - type: map_at_100
      value: 35.0
    - type: map_at_1000
      value: 35.134
    - type: map_at_3
      value: 30.789
    - type: map_at_5
      value: 32.427
    - type: mrr_at_1
      value: 29.548000000000002
    - type: mrr_at_10
      value: 38.521
    - type: mrr_at_100
      value: 39.432
    - type: mrr_at_1000
      value: 39.494
    - type: mrr_at_3
      value: 35.691
    - type: mrr_at_5
      value: 37.424
    - type: ndcg_at_1
      value: 29.548000000000002
    - type: ndcg_at_10
      value: 39.301
    - type: ndcg_at_100
      value: 44.907000000000004
    - type: ndcg_at_1000
      value: 47.494
    - type: ndcg_at_3
      value: 34.08
    - type: ndcg_at_5
      value: 36.649
    - type: precision_at_1
      value: 29.548000000000002
    - type: precision_at_10
      value: 7.084
    - type: precision_at_100
      value: 1.169
    - type: precision_at_1000
      value: 0.158
    - type: precision_at_3
      value: 15.881
    - type: precision_at_5
      value: 11.53
    - type: recall_at_1
      value: 24.759999999999998
    - type: recall_at_10
      value: 51.202000000000005
    - type: recall_at_100
      value: 74.542
    - type: recall_at_1000
      value: 91.669
    - type: recall_at_3
      value: 36.892
    - type: recall_at_5
      value: 43.333
    - type: map_at_1
      value: 23.247999999999998
    - type: map_at_10
      value: 31.878
    - type: map_at_100
      value: 33.135
    - type: map_at_1000
      value: 33.263999999999996
    - type: map_at_3
      value: 29.406
    - type: map_at_5
      value: 30.602
    - type: mrr_at_1
      value: 28.767
    - type: mrr_at_10
      value: 36.929
    - type: mrr_at_100
      value: 37.844
    - type: mrr_at_1000
      value: 37.913000000000004
    - type: mrr_at_3
      value: 34.589
    - type: mrr_at_5
      value: 35.908
    - type: ndcg_at_1
      value: 28.767
    - type: ndcg_at_10
      value: 37.172
    - type: ndcg_at_100
      value: 42.842
    - type: ndcg_at_1000
      value: 45.534
    - type: ndcg_at_3
      value: 32.981
    - type: ndcg_at_5
      value: 34.628
    - type: precision_at_1
      value: 28.767
    - type: precision_at_10
      value: 6.678000000000001
    - type: precision_at_100
      value: 1.1199999999999999
    - type: precision_at_1000
      value: 0.155
    - type: precision_at_3
      value: 15.715000000000002
    - type: precision_at_5
      value: 10.913
    - type: recall_at_1
      value: 23.247999999999998
    - type: recall_at_10
      value: 48.16
    - type: recall_at_100
      value: 72.753
    - type: recall_at_1000
      value: 90.8
    - type: recall_at_3
      value: 35.961999999999996
    - type: recall_at_5
      value: 40.504
    - type: map_at_1
      value: 23.825583333333334
    - type: map_at_10
      value: 32.2845
    - type: map_at_100
      value: 33.48566666666667
    - type: map_at_1000
      value: 33.60833333333333
    - type: map_at_3
      value: 29.604916666666664
    - type: map_at_5
      value: 31.015333333333334
    - type: mrr_at_1
      value: 27.850916666666663
    - type: mrr_at_10
      value: 36.122416666666666
    - type: mrr_at_100
      value: 37.01275
    - type: mrr_at_1000
      value: 37.07566666666667
    - type: mrr_at_3
      value: 33.665749999999996
    - type: mrr_at_5
      value: 35.00916666666667
    - type: ndcg_at_1
      value: 27.850916666666663
    - type: ndcg_at_10
      value: 37.47625
    - type: ndcg_at_100
      value: 42.74433333333334
    - type: ndcg_at_1000
      value: 45.21991666666667
    - type: ndcg_at_3
      value: 32.70916666666667
    - type: ndcg_at_5
      value: 34.80658333333333
    - type: precision_at_1
      value: 27.850916666666663
    - type: precision_at_10
      value: 6.5761666666666665
    - type: precision_at_100
      value: 1.0879999999999999
    - type: precision_at_1000
      value: 0.15058333333333332
    - type: precision_at_3
      value: 14.933833333333336
    - type: precision_at_5
      value: 10.607249999999999
    - type: recall_at_1
      value: 23.825583333333334
    - type: recall_at_10
      value: 49.100500000000004
    - type: recall_at_100
      value: 72.21133333333334
    - type: recall_at_1000
      value: 89.34791666666666
    - type: recall_at_3
      value: 35.90525
    - type: recall_at_5
      value: 41.24583333333334
    - type: map_at_1
      value: 21.343
    - type: map_at_10
      value: 27.313
    - type: map_at_100
      value: 28.316999999999997
    - type: map_at_1000
      value: 28.406
    - type: map_at_3
      value: 25.06
    - type: map_at_5
      value: 26.409
    - type: mrr_at_1
      value: 23.313
    - type: mrr_at_10
      value: 29.467
    - type: mrr_at_100
      value: 30.348999999999997
    - type: mrr_at_1000
      value: 30.42
    - type: mrr_at_3
      value: 27.173000000000002
    - type: mrr_at_5
      value: 28.461
    - type: ndcg_at_1
      value: 23.313
    - type: ndcg_at_10
      value: 31.183
    - type: ndcg_at_100
      value: 36.252
    - type: ndcg_at_1000
      value: 38.582
    - type: ndcg_at_3
      value: 26.838
    - type: ndcg_at_5
      value: 29.042
    - type: precision_at_1
      value: 23.313
    - type: precision_at_10
      value: 4.9079999999999995
    - type: precision_at_100
      value: 0.808
    - type: precision_at_1000
      value: 0.109
    - type: precision_at_3
      value: 11.299
    - type: precision_at_5
      value: 8.097999999999999
    - type: recall_at_1
      value: 21.343
    - type: recall_at_10
      value: 41.047
    - type: recall_at_100
      value: 64.372
    - type: recall_at_1000
      value: 81.499
    - type: recall_at_3
      value: 29.337000000000003
    - type: recall_at_5
      value: 34.756
    - type: map_at_1
      value: 16.595
    - type: map_at_10
      value: 23.433
    - type: map_at_100
      value: 24.578
    - type: map_at_1000
      value: 24.709999999999997
    - type: map_at_3
      value: 21.268
    - type: map_at_5
      value: 22.393
    - type: mrr_at_1
      value: 20.131
    - type: mrr_at_10
      value: 27.026
    - type: mrr_at_100
      value: 28.003
    - type: mrr_at_1000
      value: 28.083999999999996
    - type: mrr_at_3
      value: 24.966
    - type: mrr_at_5
      value: 26.064999999999998
    - type: ndcg_at_1
      value: 20.131
    - type: ndcg_at_10
      value: 27.846
    - type: ndcg_at_100
      value: 33.318999999999996
    - type: ndcg_at_1000
      value: 36.403
    - type: ndcg_at_3
      value: 23.883
    - type: ndcg_at_5
      value: 25.595000000000002
    - type: precision_at_1
      value: 20.131
    - type: precision_at_10
      value: 5.034000000000001
    - type: precision_at_100
      value: 0.9079999999999999
    - type: precision_at_1000
      value: 0.13699999999999998
    - type: precision_at_3
      value: 11.23
    - type: precision_at_5
      value: 8.032
    - type: recall_at_1
      value: 16.595
    - type: recall_at_10
      value: 37.576
    - type: recall_at_100
      value: 62.044
    - type: recall_at_1000
      value: 83.97
    - type: recall_at_3
      value: 26.631
    - type: recall_at_5
      value: 31.002000000000002
    - type: map_at_1
      value: 24.85
    - type: map_at_10
      value: 32.762
    - type: map_at_100
      value: 33.896
    - type: map_at_1000
      value: 34.006
    - type: map_at_3
      value: 29.965000000000003
    - type: map_at_5
      value: 31.485999999999997
    - type: mrr_at_1
      value: 28.731
    - type: mrr_at_10
      value: 36.504999999999995
    - type: mrr_at_100
      value: 37.364999999999995
    - type: mrr_at_1000
      value: 37.431
    - type: mrr_at_3
      value: 34.033
    - type: mrr_at_5
      value: 35.4
    - type: ndcg_at_1
      value: 28.731
    - type: ndcg_at_10
      value: 37.788
    - type: ndcg_at_100
      value: 43.1
    - type: ndcg_at_1000
      value: 45.623999999999995
    - type: ndcg_at_3
      value: 32.717
    - type: ndcg_at_5
      value: 35.024
    - type: precision_at_1
      value: 28.731
    - type: precision_at_10
      value: 6.371
    - type: precision_at_100
      value: 1.02
    - type: precision_at_1000
      value: 0.135
    - type: precision_at_3
      value: 14.521
    - type: precision_at_5
      value: 10.41
    - type: recall_at_1
      value: 24.85
    - type: recall_at_10
      value: 49.335
    - type: recall_at_100
      value: 72.792
    - type: recall_at_1000
      value: 90.525
    - type: recall_at_3
      value: 35.698
    - type: recall_at_5
      value: 41.385
    - type: map_at_1
      value: 23.016000000000002
    - type: map_at_10
      value: 32.126
    - type: map_at_100
      value: 33.786
    - type: map_at_1000
      value: 34.012
    - type: map_at_3
      value: 29.256
    - type: map_at_5
      value: 30.552
    - type: mrr_at_1
      value: 27.272999999999996
    - type: mrr_at_10
      value: 35.967
    - type: mrr_at_100
      value: 37.082
    - type: mrr_at_1000
      value: 37.146
    - type: mrr_at_3
      value: 33.531
    - type: mrr_at_5
      value: 34.697
    - type: ndcg_at_1
      value: 27.272999999999996
    - type: ndcg_at_10
      value: 37.945
    - type: ndcg_at_100
      value: 43.928
    - type: ndcg_at_1000
      value: 46.772999999999996
    - type: ndcg_at_3
      value: 33.111000000000004
    - type: ndcg_at_5
      value: 34.794000000000004
    - type: precision_at_1
      value: 27.272999999999996
    - type: precision_at_10
      value: 7.53
    - type: precision_at_100
      value: 1.512
    - type: precision_at_1000
      value: 0.241
    - type: precision_at_3
      value: 15.547
    - type: precision_at_5
      value: 11.146
    - type: recall_at_1
      value: 23.016000000000002
    - type: recall_at_10
      value: 49.576
    - type: recall_at_100
      value: 75.74600000000001
    - type: recall_at_1000
      value: 94.069
    - type: recall_at_3
      value: 35.964
    - type: recall_at_5
      value: 40.455999999999996
    - type: map_at_1
      value: 22.742
    - type: map_at_10
      value: 29.232000000000003
    - type: map_at_100
      value: 30.160999999999998
    - type: map_at_1000
      value: 30.278
    - type: map_at_3
      value: 27.134999999999998
    - type: map_at_5
      value: 27.932000000000002
    - type: mrr_at_1
      value: 24.399
    - type: mrr_at_10
      value: 31.048
    - type: mrr_at_100
      value: 31.912000000000003
    - type: mrr_at_1000
      value: 31.999
    - type: mrr_at_3
      value: 29.144
    - type: mrr_at_5
      value: 29.809
    - type: ndcg_at_1
      value: 24.399
    - type: ndcg_at_10
      value: 33.354
    - type: ndcg_at_100
      value: 38.287
    - type: ndcg_at_1000
      value: 41.105000000000004
    - type: ndcg_at_3
      value: 29.112
    - type: ndcg_at_5
      value: 30.379
    - type: precision_at_1
      value: 24.399
    - type: precision_at_10
      value: 5.157
    - type: precision_at_100
      value: 0.828
    - type: precision_at_1000
      value: 0.11800000000000001
    - type: precision_at_3
      value: 11.892
    - type: precision_at_5
      value: 8.022
    - type: recall_at_1
      value: 22.742
    - type: recall_at_10
      value: 44.31
    - type: recall_at_100
      value: 67.422
    - type: recall_at_1000
      value: 88.193
    - type: recall_at_3
      value: 32.705
    - type: recall_at_5
      value: 35.669000000000004
  - task:
      type: Retrieval
    dataset:
      name: MTEB ClimateFEVER
      type: climate-fever
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 9.067
    - type: map_at_10
      value: 14.821000000000002
    - type: map_at_100
      value: 16.195
    - type: map_at_1000
      value: 16.359
    - type: map_at_3
      value: 12.666
    - type: map_at_5
      value: 13.675999999999998
    - type: mrr_at_1
      value: 20.326
    - type: mrr_at_10
      value: 29.798000000000002
    - type: mrr_at_100
      value: 30.875000000000004
    - type: mrr_at_1000
      value: 30.928
    - type: mrr_at_3
      value: 26.678
    - type: mrr_at_5
      value: 28.433000000000003
    - type: ndcg_at_1
      value: 20.326
    - type: ndcg_at_10
      value: 21.477
    - type: ndcg_at_100
      value: 27.637
    - type: ndcg_at_1000
      value: 30.953000000000003
    - type: ndcg_at_3
      value: 17.456
    - type: ndcg_at_5
      value: 18.789
    - type: precision_at_1
      value: 20.326
    - type: precision_at_10
      value: 6.482
    - type: precision_at_100
      value: 1.302
    - type: precision_at_1000
      value: 0.191
    - type: precision_at_3
      value: 12.53
    - type: precision_at_5
      value: 9.603
    - type: recall_at_1
      value: 9.067
    - type: recall_at_10
      value: 26.246000000000002
    - type: recall_at_100
      value: 47.837
    - type: recall_at_1000
      value: 66.637
    - type: recall_at_3
      value: 16.468
    - type: recall_at_5
      value: 20.088
  - task:
      type: Retrieval
    dataset:
      name: MTEB DBPedia
      type: dbpedia-entity
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 7.563000000000001
    - type: map_at_10
      value: 15.22
    - type: map_at_100
      value: 20.048
    - type: map_at_1000
      value: 21.17
    - type: map_at_3
      value: 11.627
    - type: map_at_5
      value: 13.239
    - type: mrr_at_1
      value: 56.25
    - type: mrr_at_10
      value: 64.846
    - type: mrr_at_100
      value: 65.405
    - type: mrr_at_1000
      value: 65.41799999999999
    - type: mrr_at_3
      value: 63.125
    - type: mrr_at_5
      value: 64.1
    - type: ndcg_at_1
      value: 45.0
    - type: ndcg_at_10
      value: 32.437
    - type: ndcg_at_100
      value: 35.483
    - type: ndcg_at_1000
      value: 42.186
    - type: ndcg_at_3
      value: 37.297000000000004
    - type: ndcg_at_5
      value: 34.697
    - type: precision_at_1
      value: 56.25
    - type: precision_at_10
      value: 25.15
    - type: precision_at_100
      value: 7.539999999999999
    - type: precision_at_1000
      value: 1.678
    - type: precision_at_3
      value: 40.666999999999994
    - type: precision_at_5
      value: 33.45
    - type: recall_at_1
      value: 7.563000000000001
    - type: recall_at_10
      value: 19.969
    - type: recall_at_100
      value: 40.113
    - type: recall_at_1000
      value: 61.72299999999999
    - type: recall_at_3
      value: 12.950999999999999
    - type: recall_at_5
      value: 15.690999999999999
  - task:
      type: Classification
    dataset:
      name: MTEB EmotionClassification
      type: mteb/emotion
      config: default
      split: test
      revision: 4f58c6b202a23cf9a4da393831edf4f9183cad37
    metrics:
    - type: accuracy
      value: 44.675000000000004
    - type: f1
      value: 40.779372586075105
  - task:
      type: Retrieval
    dataset:
      name: MTEB FEVER
      type: fever
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 57.406
    - type: map_at_10
      value: 67.69500000000001
    - type: map_at_100
      value: 68.08
    - type: map_at_1000
      value: 68.095
    - type: map_at_3
      value: 65.688
    - type: map_at_5
      value: 66.93
    - type: mrr_at_1
      value: 61.941
    - type: mrr_at_10
      value: 72.513
    - type: mrr_at_100
      value: 72.83699999999999
    - type: mrr_at_1000
      value: 72.844
    - type: mrr_at_3
      value: 70.60499999999999
    - type: mrr_at_5
      value: 71.807
    - type: ndcg_at_1
      value: 61.941
    - type: ndcg_at_10
      value: 73.29
    - type: ndcg_at_100
      value: 74.96300000000001
    - type: ndcg_at_1000
      value: 75.28200000000001
    - type: ndcg_at_3
      value: 69.491
    - type: ndcg_at_5
      value: 71.573
    - type: precision_at_1
      value: 61.941
    - type: precision_at_10
      value: 9.388
    - type: precision_at_100
      value: 1.0290000000000001
    - type: precision_at_1000
      value: 0.107
    - type: precision_at_3
      value: 27.423
    - type: precision_at_5
      value: 17.627000000000002
    - type: recall_at_1
      value: 57.406
    - type: recall_at_10
      value: 85.975
    - type: recall_at_100
      value: 93.29899999999999
    - type: recall_at_1000
      value: 95.531
    - type: recall_at_3
      value: 75.624
    - type: recall_at_5
      value: 80.78999999999999
  - task:
      type: Retrieval
    dataset:
      name: MTEB FiQA2018
      type: fiqa
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 16.314999999999998
    - type: map_at_10
      value: 26.678
    - type: map_at_100
      value: 28.322000000000003
    - type: map_at_1000
      value: 28.519
    - type: map_at_3
      value: 23.105
    - type: map_at_5
      value: 24.808
    - type: mrr_at_1
      value: 33.333
    - type: mrr_at_10
      value: 41.453
    - type: mrr_at_100
      value: 42.339
    - type: mrr_at_1000
      value: 42.39
    - type: mrr_at_3
      value: 38.863
    - type: mrr_at_5
      value: 40.159
    - type: ndcg_at_1
      value: 33.333
    - type: ndcg_at_10
      value: 34.062
    - type: ndcg_at_100
      value: 40.595
    - type: ndcg_at_1000
      value: 44.124
    - type: ndcg_at_3
      value: 30.689
    - type: ndcg_at_5
      value: 31.255
    - type: precision_at_1
      value: 33.333
    - type: precision_at_10
      value: 9.722
    - type: precision_at_100
      value: 1.6480000000000001
    - type: precision_at_1000
      value: 0.22699999999999998
    - type: precision_at_3
      value: 20.936
    - type: precision_at_5
      value: 15.154
    - type: recall_at_1
      value: 16.314999999999998
    - type: recall_at_10
      value: 41.221000000000004
    - type: recall_at_100
      value: 65.857
    - type: recall_at_1000
      value: 87.327
    - type: recall_at_3
      value: 27.435
    - type: recall_at_5
      value: 32.242
  - task:
      type: Retrieval
    dataset:
      name: MTEB HotpotQA
      type: hotpotqa
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 31.978
    - type: map_at_10
      value: 43.784
    - type: map_at_100
      value: 44.547
    - type: map_at_1000
      value: 44.614
    - type: map_at_3
      value: 41.317
    - type: map_at_5
      value: 42.812
    - type: mrr_at_1
      value: 63.956999999999994
    - type: mrr_at_10
      value: 70.502
    - type: mrr_at_100
      value: 70.845
    - type: mrr_at_1000
      value: 70.865
    - type: mrr_at_3
      value: 69.192
    - type: mrr_at_5
      value: 69.994
    - type: ndcg_at_1
      value: 63.956999999999994
    - type: ndcg_at_10
      value: 52.782
    - type: ndcg_at_100
      value: 55.78999999999999
    - type: ndcg_at_1000
      value: 57.289
    - type: ndcg_at_3
      value: 48.864000000000004
    - type: ndcg_at_5
      value: 50.964
    - type: precision_at_1
      value: 63.956999999999994
    - type: precision_at_10
      value: 10.809000000000001
    - type: precision_at_100
      value: 1.319
    - type: precision_at_1000
      value: 0.152
    - type: precision_at_3
      value: 30.2
    - type: precision_at_5
      value: 19.787
    - type: recall_at_1
      value: 31.978
    - type: recall_at_10
      value: 54.045
    - type: recall_at_100
      value: 65.928
    - type: recall_at_1000
      value: 75.976
    - type: recall_at_3
      value: 45.300000000000004
    - type: recall_at_5
      value: 49.467
  - task:
      type: Classification
    dataset:
      name: MTEB ImdbClassification
      type: mteb/imdb
      config: default
      split: test
      revision: 3d86128a09e091d6018b6d26cad27f2739fc2db7
    metrics:
    - type: accuracy
      value: 63.8708
    - type: ap
      value: 59.02002684158838
    - type: f1
      value: 63.650055896985315
  - task:
      type: Retrieval
    dataset:
      name: MTEB MSMARCO
      type: msmarco
      config: default
      split: dev
      revision: None
    metrics:
    - type: map_at_1
      value: 19.834
    - type: map_at_10
      value: 31.317
    - type: map_at_100
      value: 32.576
    - type: map_at_1000
      value: 32.631
    - type: map_at_3
      value: 27.728
    - type: map_at_5
      value: 29.720000000000002
    - type: mrr_at_1
      value: 20.43
    - type: mrr_at_10
      value: 31.868999999999996
    - type: mrr_at_100
      value: 33.074999999999996
    - type: mrr_at_1000
      value: 33.123999999999995
    - type: mrr_at_3
      value: 28.333000000000002
    - type: mrr_at_5
      value: 30.305
    - type: ndcg_at_1
      value: 20.43
    - type: ndcg_at_10
      value: 37.769000000000005
    - type: ndcg_at_100
      value: 43.924
    - type: ndcg_at_1000
      value: 45.323
    - type: ndcg_at_3
      value: 30.422
    - type: ndcg_at_5
      value: 33.98
    - type: precision_at_1
      value: 20.43
    - type: precision_at_10
      value: 6.027
    - type: precision_at_100
      value: 0.9119999999999999
    - type: precision_at_1000
      value: 0.10300000000000001
    - type: precision_at_3
      value: 12.985
    - type: precision_at_5
      value: 9.593
    - type: recall_at_1
      value: 19.834
    - type: recall_at_10
      value: 57.647000000000006
    - type: recall_at_100
      value: 86.276
    - type: recall_at_1000
      value: 97.065
    - type: recall_at_3
      value: 37.616
    - type: recall_at_5
      value: 46.171
  - task:
      type: Classification
    dataset:
      name: MTEB MTOPDomainClassification (en)
      type: mteb/mtop_domain
      config: en
      split: test
      revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
    metrics:
    - type: accuracy
      value: 91.52530779753762
    - type: f1
      value: 91.4004687820246
  - task:
      type: Classification
    dataset:
      name: MTEB MTOPIntentClassification (en)
      type: mteb/mtop_intent
      config: en
      split: test
      revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
    metrics:
    - type: accuracy
      value: 72.82717738258093
    - type: f1
      value: 56.791387113030346
  - task:
      type: Classification
    dataset:
      name: MTEB MassiveIntentClassification (en)
      type: mteb/amazon_massive_intent
      config: en
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 71.09280430396772
    - type: f1
      value: 68.92843467363518
  - task:
      type: Classification
    dataset:
      name: MTEB MassiveScenarioClassification (en)
      type: mteb/amazon_massive_scenario
      config: en
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 76.2542030934768
    - type: f1
      value: 76.22211319699834
  - task:
      type: Clustering
    dataset:
      name: MTEB MedrxivClusteringP2P
      type: mteb/medrxiv-clustering-p2p
      config: default
      split: test
      revision: e7a26af6f3ae46b30dde8737f02c07b1505bcc73
    metrics:
    - type: v_measure
      value: 29.604407852989457
  - task:
      type: Clustering
    dataset:
      name: MTEB MedrxivClusteringS2S
      type: mteb/medrxiv-clustering-s2s
      config: default
      split: test
      revision: 35191c8c0dca72d8ff3efcd72aa802307d469663
    metrics:
    - type: v_measure
      value: 25.011863718751183
  - task:
      type: Reranking
    dataset:
      name: MTEB MindSmallReranking
      type: mteb/mind_small
      config: default
      split: test
      revision: 3bdac13927fdc888b903db93b2ffdbd90b295a69
    metrics:
    - type: map
      value: 31.55552172383111
    - type: mrr
      value: 32.65475731770242
  - task:
      type: Retrieval
    dataset:
      name: MTEB NFCorpus
      type: nfcorpus
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 4.968
    - type: map_at_10
      value: 10.703999999999999
    - type: map_at_100
      value: 13.316
    - type: map_at_1000
      value: 14.674000000000001
    - type: map_at_3
      value: 7.809000000000001
    - type: map_at_5
      value: 9.268
    - type: mrr_at_1
      value: 41.796
    - type: mrr_at_10
      value: 50.558
    - type: mrr_at_100
      value: 51.125
    - type: mrr_at_1000
      value: 51.184
    - type: mrr_at_3
      value: 48.349
    - type: mrr_at_5
      value: 49.572
    - type: ndcg_at_1
      value: 39.783
    - type: ndcg_at_10
      value: 30.375999999999998
    - type: ndcg_at_100
      value: 27.648
    - type: ndcg_at_1000
      value: 36.711
    - type: ndcg_at_3
      value: 35.053
    - type: ndcg_at_5
      value: 33.278999999999996
    - type: precision_at_1
      value: 41.796
    - type: precision_at_10
      value: 22.663
    - type: precision_at_100
      value: 7.210999999999999
    - type: precision_at_1000
      value: 1.984
    - type: precision_at_3
      value: 33.127
    - type: precision_at_5
      value: 29.102
    - type: recall_at_1
      value: 4.968
    - type: recall_at_10
      value: 14.469999999999999
    - type: recall_at_100
      value: 28.188000000000002
    - type: recall_at_1000
      value: 60.769
    - type: recall_at_3
      value: 8.737
    - type: recall_at_5
      value: 11.539000000000001
  - task:
      type: Retrieval
    dataset:
      name: MTEB NQ
      type: nq
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 26.958
    - type: map_at_10
      value: 40.6
    - type: map_at_100
      value: 41.754000000000005
    - type: map_at_1000
      value: 41.792
    - type: map_at_3
      value: 36.521
    - type: map_at_5
      value: 38.866
    - type: mrr_at_1
      value: 30.330000000000002
    - type: mrr_at_10
      value: 43.013
    - type: mrr_at_100
      value: 43.89
    - type: mrr_at_1000
      value: 43.917
    - type: mrr_at_3
      value: 39.489000000000004
    - type: mrr_at_5
      value: 41.504999999999995
    - type: ndcg_at_1
      value: 30.330000000000002
    - type: ndcg_at_10
      value: 47.878
    - type: ndcg_at_100
      value: 52.761
    - type: ndcg_at_1000
      value: 53.69500000000001
    - type: ndcg_at_3
      value: 40.061
    - type: ndcg_at_5
      value: 43.980000000000004
    - type: precision_at_1
      value: 30.330000000000002
    - type: precision_at_10
      value: 8.048
    - type: precision_at_100
      value: 1.076
    - type: precision_at_1000
      value: 0.117
    - type: precision_at_3
      value: 18.299000000000003
    - type: precision_at_5
      value: 13.25
    - type: recall_at_1
      value: 26.958
    - type: recall_at_10
      value: 67.72399999999999
    - type: recall_at_100
      value: 89.02600000000001
    - type: recall_at_1000
      value: 96.029
    - type: recall_at_3
      value: 47.332
    - type: recall_at_5
      value: 56.36600000000001
  - task:
      type: Retrieval
    dataset:
      name: MTEB QuoraRetrieval
      type: quora
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 69.926
    - type: map_at_10
      value: 83.797
    - type: map_at_100
      value: 84.42699999999999
    - type: map_at_1000
      value: 84.446
    - type: map_at_3
      value: 80.78
    - type: map_at_5
      value: 82.669
    - type: mrr_at_1
      value: 80.44
    - type: mrr_at_10
      value: 86.79
    - type: mrr_at_100
      value: 86.90299999999999
    - type: mrr_at_1000
      value: 86.904
    - type: mrr_at_3
      value: 85.753
    - type: mrr_at_5
      value: 86.478
    - type: ndcg_at_1
      value: 80.44
    - type: ndcg_at_10
      value: 87.634
    - type: ndcg_at_100
      value: 88.9
    - type: ndcg_at_1000
      value: 89.03
    - type: ndcg_at_3
      value: 84.622
    - type: ndcg_at_5
      value: 86.29
    - type: precision_at_1
      value: 80.44
    - type: precision_at_10
      value: 13.305
    - type: precision_at_100
      value: 1.524
    - type: precision_at_1000
      value: 0.157
    - type: precision_at_3
      value: 36.957
    - type: precision_at_5
      value: 24.328
    - type: recall_at_1
      value: 69.926
    - type: recall_at_10
      value: 94.99300000000001
    - type: recall_at_100
      value: 99.345
    - type: recall_at_1000
      value: 99.97
    - type: recall_at_3
      value: 86.465
    - type: recall_at_5
      value: 91.121
  - task:
      type: Clustering
    dataset:
      name: MTEB RedditClustering
      type: mteb/reddit-clustering
      config: default
      split: test
      revision: 24640382cdbf8abc73003fb0fa6d111a705499eb
    metrics:
    - type: v_measure
      value: 42.850644235471144
  - task:
      type: Clustering
    dataset:
      name: MTEB RedditClusteringP2P
      type: mteb/reddit-clustering-p2p
      config: default
      split: test
      revision: 282350215ef01743dc01b456c7f5241fa8937f16
    metrics:
    - type: v_measure
      value: 52.547875398320734
  - task:
      type: Retrieval
    dataset:
      name: MTEB SCIDOCS
      type: scidocs
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 4.328
    - type: map_at_10
      value: 10.479
    - type: map_at_100
      value: 12.25
    - type: map_at_1000
      value: 12.522
    - type: map_at_3
      value: 7.548000000000001
    - type: map_at_5
      value: 9.039
    - type: mrr_at_1
      value: 21.3
    - type: mrr_at_10
      value: 30.678
    - type: mrr_at_100
      value: 31.77
    - type: mrr_at_1000
      value: 31.831
    - type: mrr_at_3
      value: 27.500000000000004
    - type: mrr_at_5
      value: 29.375
    - type: ndcg_at_1
      value: 21.3
    - type: ndcg_at_10
      value: 17.626
    - type: ndcg_at_100
      value: 25.03
    - type: ndcg_at_1000
      value: 30.055
    - type: ndcg_at_3
      value: 16.744999999999997
    - type: ndcg_at_5
      value: 14.729999999999999
    - type: precision_at_1
      value: 21.3
    - type: precision_at_10
      value: 9.09
    - type: precision_at_100
      value: 1.989
    - type: precision_at_1000
      value: 0.32
    - type: precision_at_3
      value: 15.467
    - type: precision_at_5
      value: 12.879999999999999
    - type: recall_at_1
      value: 4.328
    - type: recall_at_10
      value: 18.412
    - type: recall_at_100
      value: 40.363
    - type: recall_at_1000
      value: 64.997
    - type: recall_at_3
      value: 9.408
    - type: recall_at_5
      value: 13.048000000000002
  - task:
      type: STS
    dataset:
      name: MTEB SICK-R
      type: mteb/sickr-sts
      config: default
      split: test
      revision: a6ea5a8cab320b040a23452cc28066d9beae2cee
    metrics:
    - type: cos_sim_pearson
      value: 84.1338589503896
    - type: cos_sim_spearman
      value: 79.1378154534123
    - type: euclidean_pearson
      value: 73.17857462509251
    - type: euclidean_spearman
      value: 70.79268955610539
    - type: manhattan_pearson
      value: 72.8280251705823
    - type: manhattan_spearman
      value: 70.60323787229834
  - task:
      type: STS
    dataset:
      name: MTEB STS12
      type: mteb/sts12-sts
      config: default
      split: test
      revision: a0d554a64d88156834ff5ae9920b964011b16384
    metrics:
    - type: cos_sim_pearson
      value: 84.21604641858598
    - type: cos_sim_spearman
      value: 75.06080146054282
    - type: euclidean_pearson
      value: 69.44429285856924
    - type: euclidean_spearman
      value: 58.240130690046456
    - type: manhattan_pearson
      value: 69.07597314234852
    - type: manhattan_spearman
      value: 58.08224335836159
  - task:
      type: STS
    dataset:
      name: MTEB STS13
      type: mteb/sts13-sts
      config: default
      split: test
      revision: 7e90230a92c190f1bf69ae9002b8cea547a64cca
    metrics:
    - type: cos_sim_pearson
      value: 80.2252849321165
    - type: cos_sim_spearman
      value: 80.85907200101076
    - type: euclidean_pearson
      value: 70.85619832878055
    - type: euclidean_spearman
      value: 71.59417341887324
    - type: manhattan_pearson
      value: 70.55842192345895
    - type: manhattan_spearman
      value: 71.30332994715893
  - task:
      type: STS
    dataset:
      name: MTEB STS14
      type: mteb/sts14-sts
      config: default
      split: test
      revision: 6031580fec1f6af667f0bd2da0a551cf4f0b2375
    metrics:
    - type: cos_sim_pearson
      value: 80.50469360654135
    - type: cos_sim_spearman
      value: 76.12917164308409
    - type: euclidean_pearson
      value: 70.4070213910491
    - type: euclidean_spearman
      value: 66.97320451942113
    - type: manhattan_pearson
      value: 70.24834290119863
    - type: manhattan_spearman
      value: 66.9047074173091
  - task:
      type: STS
    dataset:
      name: MTEB STS15
      type: mteb/sts15-sts
      config: default
      split: test
      revision: ae752c7c21bf194d8b67fd573edf7ae58183cbe3
    metrics:
    - type: cos_sim_pearson
      value: 84.70140350059746
    - type: cos_sim_spearman
      value: 85.55427877110485
    - type: euclidean_pearson
      value: 63.4780453371435
    - type: euclidean_spearman
      value: 64.65485395077273
    - type: manhattan_pearson
      value: 63.64869846572011
    - type: manhattan_spearman
      value: 64.87219311596813
  - task:
      type: STS
    dataset:
      name: MTEB STS16
      type: mteb/sts16-sts
      config: default
      split: test
      revision: 4d8694f8f0e0100860b497b999b3dbed754a0513
    metrics:
    - type: cos_sim_pearson
      value: 79.4416477676503
    - type: cos_sim_spearman
      value: 81.2094925260351
    - type: euclidean_pearson
      value: 68.372257553367
    - type: euclidean_spearman
      value: 69.47792807911692
    - type: manhattan_pearson
      value: 68.17773583183664
    - type: manhattan_spearman
      value: 69.31505452732998
  - task:
      type: STS
    dataset:
      name: MTEB STS17 (en-en)
      type: mteb/sts17-crosslingual-sts
      config: en-en
      split: test
      revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
    metrics:
    - type: cos_sim_pearson
      value: 88.94688403351994
    - type: cos_sim_spearman
      value: 88.97626967707933
    - type: euclidean_pearson
      value: 74.09942728422159
    - type: euclidean_spearman
      value: 72.91022362666948
    - type: manhattan_pearson
      value: 74.11262432880199
    - type: manhattan_spearman
      value: 72.82115894578564
  - task:
      type: STS
    dataset:
      name: MTEB STS22 (en)
      type: mteb/sts22-crosslingual-sts
      config: en
      split: test
      revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
    metrics:
    - type: cos_sim_pearson
      value: 67.42605802805606
    - type: cos_sim_spearman
      value: 66.22330559222408
    - type: euclidean_pearson
      value: 50.15272876367891
    - type: euclidean_spearman
      value: 60.695400782452715
    - type: manhattan_pearson
      value: 50.17076569264417
    - type: manhattan_spearman
      value: 60.3761281869747
  - task:
      type: STS
    dataset:
      name: MTEB STSBenchmark
      type: mteb/stsbenchmark-sts
      config: default
      split: test
      revision: b0fddb56ed78048fa8b90373c8a3cfc37b684831
    metrics:
    - type: cos_sim_pearson
      value: 82.85939227596093
    - type: cos_sim_spearman
      value: 82.57071649593358
    - type: euclidean_pearson
      value: 72.18291316100125
    - type: euclidean_spearman
      value: 70.70702024402348
    - type: manhattan_pearson
      value: 72.36789718833687
    - type: manhattan_spearman
      value: 70.92789721402387
  - task:
      type: Reranking
    dataset:
      name: MTEB SciDocsRR
      type: mteb/scidocs-reranking
      config: default
      split: test
      revision: d3c5e1fc0b855ab6097bf1cda04dd73947d7caab
    metrics:
    - type: map
      value: 79.31107201598611
    - type: mrr
      value: 93.66321314850727
  - task:
      type: Retrieval
    dataset:
      name: MTEB SciFact
      type: scifact
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 45.428000000000004
    - type: map_at_10
      value: 54.730000000000004
    - type: map_at_100
      value: 55.421
    - type: map_at_1000
      value: 55.47299999999999
    - type: map_at_3
      value: 52.333
    - type: map_at_5
      value: 53.72
    - type: mrr_at_1
      value: 48.333
    - type: mrr_at_10
      value: 56.601
    - type: mrr_at_100
      value: 57.106
    - type: mrr_at_1000
      value: 57.154
    - type: mrr_at_3
      value: 54.611
    - type: mrr_at_5
      value: 55.87800000000001
    - type: ndcg_at_1
      value: 48.333
    - type: ndcg_at_10
      value: 59.394999999999996
    - type: ndcg_at_100
      value: 62.549
    - type: ndcg_at_1000
      value: 63.941
    - type: ndcg_at_3
      value: 55.096000000000004
    - type: ndcg_at_5
      value: 57.325
    - type: precision_at_1
      value: 48.333
    - type: precision_at_10
      value: 8.1
    - type: precision_at_100
      value: 0.983
    - type: precision_at_1000
      value: 0.11
    - type: precision_at_3
      value: 21.889
    - type: precision_at_5
      value: 14.533
    - type: recall_at_1
      value: 45.428000000000004
    - type: recall_at_10
      value: 71.806
    - type: recall_at_100
      value: 86.533
    - type: recall_at_1000
      value: 97.5
    - type: recall_at_3
      value: 60.228
    - type: recall_at_5
      value: 65.90599999999999
  - task:
      type: PairClassification
    dataset:
      name: MTEB SprintDuplicateQuestions
      type: mteb/sprintduplicatequestions-pairclassification
      config: default
      split: test
      revision: d66bd1f72af766a5cc4b0ca5e00c162f89e8cc46
    metrics:
    - type: cos_sim_accuracy
      value: 99.8029702970297
    - type: cos_sim_ap
      value: 95.48085242816634
    - type: cos_sim_f1
      value: 89.86653484923382
    - type: cos_sim_precision
      value: 88.85630498533725
    - type: cos_sim_recall
      value: 90.9
    - type: dot_accuracy
      value: 99.21881188118812
    - type: dot_ap
      value: 55.14126603018576
    - type: dot_f1
      value: 55.22458628841608
    - type: dot_precision
      value: 52.37668161434977
    - type: dot_recall
      value: 58.4
    - type: euclidean_accuracy
      value: 99.64356435643565
    - type: euclidean_ap
      value: 84.52487064474103
    - type: euclidean_f1
      value: 80.53908355795149
    - type: euclidean_precision
      value: 87.36842105263159
    - type: euclidean_recall
      value: 74.7
    - type: manhattan_accuracy
      value: 99.63861386138613
    - type: manhattan_ap
      value: 84.1994288662172
    - type: manhattan_f1
      value: 80.38482095136291
    - type: manhattan_precision
      value: 86.33754305396096
    - type: manhattan_recall
      value: 75.2
    - type: max_accuracy
      value: 99.8029702970297
    - type: max_ap
      value: 95.48085242816634
    - type: max_f1
      value: 89.86653484923382
  - task:
      type: Clustering
    dataset:
      name: MTEB StackExchangeClustering
      type: mteb/stackexchange-clustering
      config: default
      split: test
      revision: 6cbc1f7b2bc0622f2e39d2c77fa502909748c259
    metrics:
    - type: v_measure
      value: 48.06508273111389
  - task:
      type: Clustering
    dataset:
      name: MTEB StackExchangeClusteringP2P
      type: mteb/stackexchange-clustering-p2p
      config: default
      split: test
      revision: 815ca46b2622cec33ccafc3735d572c266efdb44
    metrics:
    - type: v_measure
      value: 31.36169910951664
  - task:
      type: Reranking
    dataset:
      name: MTEB StackOverflowDupQuestions
      type: mteb/stackoverflowdupquestions-reranking
      config: default
      split: test
      revision: e185fbe320c72810689fc5848eb6114e1ef5ec69
    metrics:
    - type: map
      value: 50.110601218420356
    - type: mrr
      value: 50.90277777777777
  - task:
      type: Summarization
    dataset:
      name: MTEB SummEval
      type: mteb/summeval
      config: default
      split: test
      revision: cda12ad7615edc362dbf25a00fdd61d3b1eaf93c
    metrics:
    - type: cos_sim_pearson
      value: 29.63669555287747
    - type: cos_sim_spearman
      value: 30.708042454053853
    - type: dot_pearson
      value: 20.309025749838924
    - type: dot_spearman
      value: 21.511758746817165
  - task:
      type: Retrieval
    dataset:
      name: MTEB TRECCOVID
      type: trec-covid
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 0.201
    - type: map_at_10
      value: 1.405
    - type: map_at_100
      value: 7.359999999999999
    - type: map_at_1000
      value: 17.858
    - type: map_at_3
      value: 0.494
    - type: map_at_5
      value: 0.757
    - type: mrr_at_1
      value: 74.0
    - type: mrr_at_10
      value: 84.89999999999999
    - type: mrr_at_100
      value: 84.89999999999999
    - type: mrr_at_1000
      value: 84.89999999999999
    - type: mrr_at_3
      value: 84.0
    - type: mrr_at_5
      value: 84.89999999999999
    - type: ndcg_at_1
      value: 68.0
    - type: ndcg_at_10
      value: 60.571
    - type: ndcg_at_100
      value: 46.016
    - type: ndcg_at_1000
      value: 41.277
    - type: ndcg_at_3
      value: 63.989
    - type: ndcg_at_5
      value: 61.41
    - type: precision_at_1
      value: 74.0
    - type: precision_at_10
      value: 65.2
    - type: precision_at_100
      value: 47.04
    - type: precision_at_1000
      value: 18.416
    - type: precision_at_3
      value: 68.0
    - type: precision_at_5
      value: 66.4
    - type: recall_at_1
      value: 0.201
    - type: recall_at_10
      value: 1.763
    - type: recall_at_100
      value: 11.008999999999999
    - type: recall_at_1000
      value: 38.509
    - type: recall_at_3
      value: 0.551
    - type: recall_at_5
      value: 0.881
  - task:
      type: Retrieval
    dataset:
      name: MTEB Touche2020
      type: webis-touche2020
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 1.4040000000000001
    - type: map_at_10
      value: 7.847999999999999
    - type: map_at_100
      value: 12.908
    - type: map_at_1000
      value: 14.37
    - type: map_at_3
      value: 3.6450000000000005
    - type: map_at_5
      value: 4.93
    - type: mrr_at_1
      value: 18.367
    - type: mrr_at_10
      value: 32.576
    - type: mrr_at_100
      value: 34.163
    - type: mrr_at_1000
      value: 34.18
    - type: mrr_at_3
      value: 28.571
    - type: mrr_at_5
      value: 30.918
    - type: ndcg_at_1
      value: 15.306000000000001
    - type: ndcg_at_10
      value: 18.59
    - type: ndcg_at_100
      value: 30.394
    - type: ndcg_at_1000
      value: 42.198
    - type: ndcg_at_3
      value: 18.099
    - type: ndcg_at_5
      value: 16.955000000000002
    - type: precision_at_1
      value: 16.326999999999998
    - type: precision_at_10
      value: 17.959
    - type: precision_at_100
      value: 6.755
    - type: precision_at_1000
      value: 1.4529999999999998
    - type: precision_at_3
      value: 20.408
    - type: precision_at_5
      value: 18.367
    - type: recall_at_1
      value: 1.4040000000000001
    - type: recall_at_10
      value: 14.048
    - type: recall_at_100
      value: 42.150999999999996
    - type: recall_at_1000
      value: 77.85600000000001
    - type: recall_at_3
      value: 4.819
    - type: recall_at_5
      value: 7.13
  - task:
      type: Classification
    dataset:
      name: MTEB ToxicConversationsClassification
      type: mteb/toxic_conversations_50k
      config: default
      split: test
      revision: d7c0de2777da35d6aae2200a62c6e0e5af397c4c
    metrics:
    - type: accuracy
      value: 66.1456
    - type: ap
      value: 11.631023858569064
    - type: f1
      value: 50.128196455722254
  - task:
      type: Classification
    dataset:
      name: MTEB TweetSentimentExtractionClassification
      type: mteb/tweet_sentiment_extraction
      config: default
      split: test
      revision: d604517c81ca91fe16a244d1248fc021f9ecee7a
    metrics:
    - type: accuracy
      value: 56.850594227504246
    - type: f1
      value: 56.82313689360827
  - task:
      type: Clustering
    dataset:
      name: MTEB TwentyNewsgroupsClustering
      type: mteb/twentynewsgroups-clustering
      config: default
      split: test
      revision: 6125ec4e24fa026cec8a478383ee943acfbd5449
    metrics:
    - type: v_measure
      value: 38.060423744064764
  - task:
      type: PairClassification
    dataset:
      name: MTEB TwitterSemEval2015
      type: mteb/twittersemeval2015-pairclassification
      config: default
      split: test
      revision: 70970daeab8776df92f5ea462b6173c0b46fd2d1
    metrics:
    - type: cos_sim_accuracy
      value: 84.43702688204088
    - type: cos_sim_ap
      value: 68.30176948820142
    - type: cos_sim_f1
      value: 64.25430330443524
    - type: cos_sim_precision
      value: 61.33365315423362
    - type: cos_sim_recall
      value: 67.46701846965699
    - type: dot_accuracy
      value: 77.76718126005842
    - type: dot_ap
      value: 37.510516716176305
    - type: dot_f1
      value: 43.53859496964441
    - type: dot_precision
      value: 32.428940568475454
    - type: dot_recall
      value: 66.2269129287599
    - type: euclidean_accuracy
      value: 82.10049472492102
    - type: euclidean_ap
      value: 61.64354520687271
    - type: euclidean_f1
      value: 59.804144841721694
    - type: euclidean_precision
      value: 52.604166666666664
    - type: euclidean_recall
      value: 69.28759894459104
    - type: manhattan_accuracy
      value: 82.22566609048101
    - type: manhattan_ap
      value: 61.753431124879974
    - type: manhattan_f1
      value: 59.77735297424941
    - type: manhattan_precision
      value: 52.0870076425632
    - type: manhattan_recall
      value: 70.13192612137203
    - type: max_accuracy
      value: 84.43702688204088
    - type: max_ap
      value: 68.30176948820142
    - type: max_f1
      value: 64.25430330443524
  - task:
      type: PairClassification
    dataset:
      name: MTEB TwitterURLCorpus
      type: mteb/twitterurlcorpus-pairclassification
      config: default
      split: test
      revision: 8b6510b0b1fa4e4c4f879467980e9be563ec1cdf
    metrics:
    - type: cos_sim_accuracy
      value: 88.81515116233942
    - type: cos_sim_ap
      value: 85.33305785100573
    - type: cos_sim_f1
      value: 78.11202938475667
    - type: cos_sim_precision
      value: 74.68567816253424
    - type: cos_sim_recall
      value: 81.86787804126887
    - type: dot_accuracy
      value: 82.50475414289595
    - type: dot_ap
      value: 69.87015340174045
    - type: dot_f1
      value: 65.94174480373633
    - type: dot_precision
      value: 61.40362525728703
    - type: dot_recall
      value: 71.20418848167539
    - type: euclidean_accuracy
      value: 83.05778709201692
    - type: euclidean_ap
      value: 70.54206653977498
    - type: euclidean_f1
      value: 62.98969847356943
    - type: euclidean_precision
      value: 61.55033063923585
    - type: euclidean_recall
      value: 64.49799815214044
    - type: manhattan_accuracy
      value: 83.0034540303489
    - type: manhattan_ap
      value: 70.53997987198404
    - type: manhattan_f1
      value: 62.95875898600075
    - type: manhattan_precision
      value: 61.89555125725339
    - type: manhattan_recall
      value: 64.05913150600554
    - type: max_accuracy
      value: 88.81515116233942
    - type: max_ap
      value: 85.33305785100573
    - type: max_f1
      value: 78.11202938475667
---
---

<br><br>

<p align="center">
<img src="https://huggingface.co/datasets/jinaai/documentation-images/resolve/main/logo.webp" alt="Jina AI: Your Search Foundation, Supercharged!" width="150px">
</p>


<p align="center">
<b>The text embedding set trained by <a href="https://jina.ai/"><b>Jina AI</b></a></b>
</p>


## Intented Usage & Model Info

`jina-embedding-b-en-v1` is a language model that has been trained using Jina AI's Linnaeus-Clean dataset.
This dataset consists of 380 million pairs of sentences, which include both query-document pairs.
These pairs were obtained from various domains and were carefully selected through a thorough cleaning process.
The Linnaeus-Full dataset, from which the Linnaeus-Clean dataset is derived, originally contained 1.6 billion sentence pairs.

The model has a range of use cases, including information retrieval, semantic textual similarity, text reranking, and more.

With a standard size of 110 million parameters,
the model enables fast inference while delivering better performance than our small model.
It is recommended to use a single GPU for inference.
Additionally, we provide the following options:

- [`jina-embedding-t-en-v1`](https://huggingface.co/jinaai/jina-embedding-t-en-v1): 14 million parameters.
- [`jina-embedding-s-en-v1`](https://huggingface.co/jinaai/jina-embedding-s-en-v1): 35 million parameters
- [`jina-embedding-b-en-v1`](https://huggingface.co/jinaai/jina-embedding-b-en-v1): 110 million parameters **(you are here)**.
- [`jina-embedding-l-en-v1`](https://huggingface.co/jinaai/jina-embedding-l-en-v1): 330 million parameters.
- `jina-embedding-1b-en-v1`: 1.2 billion parameters, 10 times bert-base (soon).
- `jina-embedding-6b-en-v1`: 6 billion parameters, 30 times bert-base (soon).

## Data & Parameters

Please checkout our [technical blog](https://arxiv.org/abs/2307.11224).

## Metrics

We compared the model against `all-minilm-l6-v2`/`all-mpnet-base-v2` from sbert and `text-embeddings-ada-002` from OpenAI:

|Name|param    |dimension|
|------------------------------|-----|------|
|all-minilm-l6-v2|23m      |384|
|all-mpnet-base-v2 |110m     |768|
|ada-embedding-002|Unknown/OpenAI API  |1536|
|jina-embedding-t-en-v1|14m      |312|
|jina-embedding-s-en-v1|35m      |512|
|jina-embedding-b-en-v1|110m      |768|
|jina-embedding-l-en-v1|330m      |1024|


|Name|STS12|STS13|STS14|STS15|STS16|STS17|TRECOVID|Quora|SciFact|
|------------------------------|-----|-----|-----|-----|-----|-----|--------|-----|-----|
|all-minilm-l6-v2|0.724|0.806|0.756|0.854|0.79 |0.876|0.473   |0.876|0.645  |
|all-mpnet-base-v2|0.726|**0.835**|0.78 |0.857|0.8  |**0.906**|0.513   |0.875|0.656  |
|ada-embedding-002|0.698|0.833|0.761|0.861|**0.86** |0.903|**0.685**   |0.876|**0.726**  |
|jina-embedding-t-en-v1|0.717|0.773|0.731|0.829|0.777|0.860|0.482   |0.840|0.522  |
|jina-embedding-s-en-v1|0.743|0.786|0.738|0.837|0.80|0.875|0.523   |0.857|0.524  |
|jina-embedding-b-en-v1|**0.751**|0.809|0.761|0.856|0.812|0.890|0.606   |0.876|0.594  |
|jina-embedding-l-en-v1|0.745|0.832|**0.781**|**0.869**|0.837|0.902|0.573   |**0.881**|0.598  |

## Usage

Usage with Jina AI Finetuner:

```python
!pip install finetuner
import finetuner

model = finetuner.build_model('jinaai/jina-embedding-b-en-v1')
embeddings = finetuner.encode(
    model=model,
    data=['how is the weather today', 'What is the current weather like today?']
)
print(finetuner.cos_sim(embeddings[0], embeddings[1]))
```

Use with sentence-transformers:

```python
from sentence_transformers import SentenceTransformer
from sentence_transformers.util import cos_sim

sentences = ['how is the weather today', 'What is the current weather like today?']

model = SentenceTransformer('jinaai/jina-embedding-b-en-v1')
embeddings = model.encode(sentences)
print(cos_sim(embeddings[0], embeddings[1]))
```


## Fine-tuning

Please consider [Finetuner](https://github.com/jina-ai/finetuner).

## Plans

1. The development of `jina-embedding-s-en-v2` is currently underway with two main objectives: improving performance and increasing the maximum sequence length.
2. We are currently working on a bilingual embedding model that combines English and X language. The upcoming model will be called `jina-embedding-s/b/l-de-v1`.

## Contact

Join our [Discord community](https://discord.jina.ai) and chat with other community members about ideas.

## Citation

If you find Jina Embeddings useful in your research, please cite the following paper:

``` latex
@misc{günther2023jina,
      title={Jina Embeddings: A Novel Set of High-Performance Sentence Embedding Models},
      author={Michael Günther and Louis Milliken and Jonathan Geuter and Georgios Mastrapas and Bo Wang and Han Xiao},
      year={2023},
      eprint={2307.11224},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
```
