metadata
language:
- en
license: apache-2.0
tags:
- sentence-transformers
- sparse-encoder
- sparse
- csr
- generated_from_trainer
- dataset_size:99000
- loss:CSRLoss
- loss:SparseMultipleNegativesRankingLoss
base_model: mixedbread-ai/mxbai-embed-large-v1
widget:
- text: >-
Saudi Arabia–United Arab Emirates relations However, the UAE and Saudi
Arabia continue to take somewhat differing stances on regional conflicts
such the Yemeni Civil War, where the UAE opposes Al-Islah, and supports
the Southern Movement, which has fought against Saudi-backed forces, and
the Syrian Civil War, where the UAE has disagreed with Saudi support for
Islamist movements.[4]
- text: >-
Economy of New Zealand New Zealand's diverse market economy has a sizable
service sector, accounting for 63% of all GDP activity in 2013.[17] Large
scale manufacturing industries include aluminium production, food
processing, metal fabrication, wood and paper products. Mining,
manufacturing, electricity, gas, water, and waste services accounted for
16.5% of GDP in 2013.[17] The primary sector continues to dominate New
Zealand's exports, despite accounting for 6.5% of GDP in 2013.[17]
- text: >-
who was the first president of indian science congress meeting held in
kolkata in 1914
- text: >-
Get Over It (Eagles song) "Get Over It" is a song by the Eagles released
as a single after a fourteen-year breakup. It was also the first song
written by bandmates Don Henley and Glenn Frey when the band reunited.
"Get Over It" was played live for the first time during their Hell Freezes
Over tour in 1994. It returned the band to the U.S. Top 40 after a
fourteen-year absence, peaking at No. 31 on the Billboard Hot 100 chart.
It also hit No. 4 on the Billboard Mainstream Rock Tracks chart. The song
was not played live by the Eagles after the "Hell Freezes Over" tour in
1994. It remains the group's last Top 40 hit in the U.S.
- text: >-
Cornelius the Centurion Cornelius (Greek: Κορνήλιος) was a Roman centurion
who is considered by Christians to be one of the first Gentiles to convert
to the faith, as related in Acts of the Apostles.
datasets:
- sentence-transformers/natural-questions
pipeline_tag: feature-extraction
library_name: sentence-transformers
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
- query_active_dims
- query_sparsity_ratio
- corpus_active_dims
- corpus_sparsity_ratio
co2_eq_emissions:
emissions: 105.88726450328866
energy_consumed: 0.27241245093487726
source: codecarbon
training_type: fine-tuning
on_cloud: false
cpu_model: 13th Gen Intel(R) Core(TM) i7-13700K
ram_total_size: 31.777088165283203
hours_used: 0.75
hardware_used: 1 x NVIDIA GeForce RTX 3090
model-index:
- name: Sparse CSR model trained on Natural Questions
results:
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoMSMARCO 4
type: NanoMSMARCO_4
metrics:
- type: cosine_accuracy@1
value: 0.12
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.16
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.26
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.34
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.12
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.05333333333333333
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.052000000000000005
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.034
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.12
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.16
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.26
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.34
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.20848075322384305
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.16888095238095235
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.18291408151127517
name: Cosine Map@100
- type: query_active_dims
value: 4
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9990234375
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 4
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9990234375
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoNFCorpus 4
type: NanoNFCorpus_4
metrics:
- type: cosine_accuracy@1
value: 0.06
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.14
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.24
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.3
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.06
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.06000000000000001
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.068
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.064
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.0009459743220542356
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.003449821160051155
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.007601209053812086
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.014969691928058278
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.06420092741811712
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.12744444444444444
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.014642071302654571
name: Cosine Map@100
- type: query_active_dims
value: 4
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9990234375
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 4
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9990234375
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoNQ 4
type: NanoNQ_4
metrics:
- type: cosine_accuracy@1
value: 0.04
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.08
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.16
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.26
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.04
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.026666666666666665
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.032
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.026000000000000002
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.04
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.08
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.16
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.25
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.12446577906212845
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.08757936507936508
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.09936029341073244
name: Cosine Map@100
- type: query_active_dims
value: 4
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9990234375
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 4
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9990234375
name: Corpus Sparsity Ratio
- task:
type: sparse-nano-beir
name: Sparse Nano BEIR
dataset:
name: NanoBEIR mean 4
type: NanoBEIR_mean_4
metrics:
- type: cosine_accuracy@1
value: 0.07333333333333333
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.12666666666666668
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.22
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.3
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.07333333333333333
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.04666666666666667
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.05066666666666667
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.04133333333333333
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.053648658107351414
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.08114994038668372
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.14253373635127067
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.20165656397601942
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.13238248656802956
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.12796825396825398
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.09897214874155406
name: Cosine Map@100
- type: query_active_dims
value: 4
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9990234375
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 4
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9990234375
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoMSMARCO 8
type: NanoMSMARCO_8
metrics:
- type: cosine_accuracy@1
value: 0.14
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.24
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.32
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.42
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.14
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.08
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.064
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.042
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.14
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.24
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.32
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.42
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.2597698452054917
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.21088888888888888
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.22093158927995368
name: Cosine Map@100
- type: query_active_dims
value: 8
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.998046875
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 8
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.998046875
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoNFCorpus 8
type: NanoNFCorpus_8
metrics:
- type: cosine_accuracy@1
value: 0.08
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.24
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.32
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.4
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.08
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.10666666666666666
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.1
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.08199999999999999
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.003534803921568628
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.01332319047951684
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.018603958472557434
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.027472535276451802
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.08639423970883567
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.16755555555555557
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.02734093319516609
name: Cosine Map@100
- type: query_active_dims
value: 8
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.998046875
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 8
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.998046875
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoNQ 8
type: NanoNQ_8
metrics:
- type: cosine_accuracy@1
value: 0.08
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.24
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.3
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.44
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.08
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.07999999999999999
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.06000000000000001
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.046000000000000006
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.08
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.22
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.28
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.42
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.2376977753947817
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.1862142857142857
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.18701815429415725
name: Cosine Map@100
- type: query_active_dims
value: 8
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.998046875
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 8
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.998046875
name: Corpus Sparsity Ratio
- task:
type: sparse-nano-beir
name: Sparse Nano BEIR
dataset:
name: NanoBEIR mean 8
type: NanoBEIR_mean_8
metrics:
- type: cosine_accuracy@1
value: 0.10000000000000002
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.24
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.3133333333333333
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.42
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.10000000000000002
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.08888888888888886
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.07466666666666667
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.05666666666666667
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.07451160130718955
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.15777439682650563
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.2062013194908525
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.28915751175881726
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.19462062010303635
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.1882195767195767
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.14509689225642566
name: Cosine Map@100
- type: query_active_dims
value: 8
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.998046875
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 8
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.998046875
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoMSMARCO 16
type: NanoMSMARCO_16
metrics:
- type: cosine_accuracy@1
value: 0.22
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.48
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.58
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.7
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.22
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.15999999999999998
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.11599999999999999
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.07
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.22
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.48
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.58
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.7
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.45695469767923136
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.37962698412698415
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.3855020346571363
name: Cosine Map@100
- type: query_active_dims
value: 16
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.99609375
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 16
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.99609375
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoNFCorpus 16
type: NanoNFCorpus_16
metrics:
- type: cosine_accuracy@1
value: 0.18
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.34
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.44
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.54
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.18
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.15333333333333332
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.14400000000000002
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.12399999999999999
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.006630871390546997
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.015107785892825198
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.023769342657046163
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.03915909301380926
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.134487928424105
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.28135714285714286
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.04034378873464851
name: Cosine Map@100
- type: query_active_dims
value: 16
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.99609375
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 16
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.99609375
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoNQ 16
type: NanoNQ_16
metrics:
- type: cosine_accuracy@1
value: 0.22
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.32
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.36
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.5
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.22
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.10666666666666665
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.07200000000000001
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.052000000000000005
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.22
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.3
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.33
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.48
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.33052122676463463
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.2881904761904762
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.2997157011386181
name: Cosine Map@100
- type: query_active_dims
value: 16
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.99609375
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 16
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.99609375
name: Corpus Sparsity Ratio
- task:
type: sparse-nano-beir
name: Sparse Nano BEIR
dataset:
name: NanoBEIR mean 16
type: NanoBEIR_mean_16
metrics:
- type: cosine_accuracy@1
value: 0.20666666666666667
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.38000000000000006
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.45999999999999996
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.58
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.20666666666666667
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.13999999999999999
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.11066666666666668
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.082
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.14887695713018234
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.2650359286309417
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.3112564475523487
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.4063863643379364
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.30732128428932365
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.3163915343915344
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.2418538415101343
name: Cosine Map@100
- type: query_active_dims
value: 16
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.99609375
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 16
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.99609375
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoMSMARCO 32
type: NanoMSMARCO_32
metrics:
- type: cosine_accuracy@1
value: 0.36
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.58
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.62
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.7
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.36
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.19333333333333333
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.124
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.07
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.36
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.58
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.62
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.7
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.5319469082007623
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.47833333333333333
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.4895239579497892
name: Cosine Map@100
- type: query_active_dims
value: 32
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9921875
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 32
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9921875
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoNFCorpus 32
type: NanoNFCorpus_32
metrics:
- type: cosine_accuracy@1
value: 0.36
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.48
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.52
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.64
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.36
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.24
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.21999999999999997
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.172
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.015057828440744998
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.03195263461978998
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.051589014542877495
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.07035182595749563
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.21044771940181314
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.4425476190476191
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.07089713098470313
name: Cosine Map@100
- type: query_active_dims
value: 32
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9921875
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 32
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9921875
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoNQ 32
type: NanoNQ_32
metrics:
- type: cosine_accuracy@1
value: 0.28
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.4
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.52
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.62
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.28
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.13333333333333333
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.10400000000000002
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.066
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.26
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.37
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.47
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.59
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.4160684104470306
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.3762142857142857
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.3702476731998177
name: Cosine Map@100
- type: query_active_dims
value: 32
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9921875
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 32
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9921875
name: Corpus Sparsity Ratio
- task:
type: sparse-nano-beir
name: Sparse Nano BEIR
dataset:
name: NanoBEIR mean 32
type: NanoBEIR_mean_32
metrics:
- type: cosine_accuracy@1
value: 0.3333333333333333
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.48666666666666664
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.5533333333333333
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.6533333333333333
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.3333333333333333
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.18888888888888888
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.14933333333333335
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.10266666666666667
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.21168594281358166
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.3273175448732633
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.38052967151429246
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.45345060865249853
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.3861543460165353
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.43236507936507945
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.31022292071143664
name: Cosine Map@100
- type: query_active_dims
value: 32
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9921875
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 32
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9921875
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoMSMARCO 64
type: NanoMSMARCO_64
metrics:
- type: cosine_accuracy@1
value: 0.42
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.6
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.66
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.8
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.42
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.2
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.13200000000000003
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.08
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.42
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.6
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.66
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.8
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.591232993639232
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.5273015873015873
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.5351182048005023
name: Cosine Map@100
- type: query_active_dims
value: 64
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.984375
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 64
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.984375
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoNFCorpus 64
type: NanoNFCorpus_64
metrics:
- type: cosine_accuracy@1
value: 0.32
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.5
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.52
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.6
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.32
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.2933333333333333
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.252
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.236
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.020044789335191174
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.04526010398813148
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.05627084683228478
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.08933472256987589
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.2628775829193256
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.4175000000000001
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.10566929023749187
name: Cosine Map@100
- type: query_active_dims
value: 64
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.984375
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 64
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.984375
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoNQ 64
type: NanoNQ_64
metrics:
- type: cosine_accuracy@1
value: 0.4
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.62
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.66
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.72
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.4
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.20666666666666667
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.136
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.07600000000000001
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.38
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.58
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.61
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.67
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.5342140484753161
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.5065555555555555
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.49683164821698605
name: Cosine Map@100
- type: query_active_dims
value: 64
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.984375
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 64
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.984375
name: Corpus Sparsity Ratio
- task:
type: sparse-nano-beir
name: Sparse Nano BEIR
dataset:
name: NanoBEIR mean 64
type: NanoBEIR_mean_64
metrics:
- type: cosine_accuracy@1
value: 0.38000000000000006
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.5733333333333334
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.6133333333333334
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.7066666666666667
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.38000000000000006
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.2333333333333333
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.17333333333333334
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.13066666666666668
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.2733482631117304
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.4084200346627105
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.44209028227742825
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.5197782408566253
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.4627748750112912
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.48378571428571426
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.37920638108499344
name: Cosine Map@100
- type: query_active_dims
value: 64
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.984375
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 64
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.984375
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoMSMARCO 128
type: NanoMSMARCO_128
metrics:
- type: cosine_accuracy@1
value: 0.34
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.64
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.68
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.8
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.34
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.21333333333333332
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.136
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.08
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.34
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.64
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.68
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.8
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.5730777373893381
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.5008015873015873
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.5127463554963555
name: Cosine Map@100
- type: query_active_dims
value: 128
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.96875
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 128
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.96875
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoNFCorpus 128
type: NanoNFCorpus_128
metrics:
- type: cosine_accuracy@1
value: 0.4
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.54
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.58
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.72
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.4
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.36666666666666664
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.308
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.276
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.04265347253746901
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.08086072465767052
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.0941496797136197
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.13775131432237744
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.33199374875578674
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.48576984126984124
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.15090058991053457
name: Cosine Map@100
- type: query_active_dims
value: 128
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.96875
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 128
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.96875
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoNQ 128
type: NanoNQ_128
metrics:
- type: cosine_accuracy@1
value: 0.44
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.64
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.68
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.78
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.44
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.22
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.14400000000000002
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.08199999999999999
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.41
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.6
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.64
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.73
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.5797743501932063
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.5520714285714285
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.533573558140474
name: Cosine Map@100
- type: query_active_dims
value: 128
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.96875
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 128
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.96875
name: Corpus Sparsity Ratio
- task:
type: sparse-nano-beir
name: Sparse Nano BEIR
dataset:
name: NanoBEIR mean 128
type: NanoBEIR_mean_128
metrics:
- type: cosine_accuracy@1
value: 0.3933333333333333
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.6066666666666668
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.6466666666666666
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.7666666666666666
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.3933333333333333
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.26666666666666666
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.19600000000000004
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.14600000000000002
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.26421782417915635
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.4402869082192235
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.47138322657120657
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.5559171047741258
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.494948612112777
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.5128809523809523
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.39907350118245466
name: Cosine Map@100
- type: query_active_dims
value: 128
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.96875
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 128
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.96875
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoMSMARCO 256
type: NanoMSMARCO_256
metrics:
- type: cosine_accuracy@1
value: 0.36
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.62
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.7
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.84
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.36
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.20666666666666667
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.14
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.08399999999999999
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.36
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.62
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.7
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.84
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.5934641617159162
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.5158809523809523
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.5228335563036209
name: Cosine Map@100
- type: query_active_dims
value: 256
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9375
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 256
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9375
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoNFCorpus 256
type: NanoNFCorpus_256
metrics:
- type: cosine_accuracy@1
value: 0.46
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.62
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.74
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.78
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.46
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.38666666666666666
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.364
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.29800000000000004
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.04750699466385613
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.08527169237328079
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.11543452383164411
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.1526866044864678
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.3632299338880757
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.5608571428571427
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.16765768014542204
name: Cosine Map@100
- type: query_active_dims
value: 256
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9375
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 256
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9375
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoNQ 256
type: NanoNQ_256
metrics:
- type: cosine_accuracy@1
value: 0.56
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.7
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.76
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.82
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.56
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.24666666666666665
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.16
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.08799999999999997
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.52
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.67
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.72
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.78
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.664653961269068
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.6482222222222223
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.6252713508893053
name: Cosine Map@100
- type: query_active_dims
value: 256
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9375
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 256
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9375
name: Corpus Sparsity Ratio
- task:
type: sparse-nano-beir
name: Sparse Nano BEIR
dataset:
name: NanoBEIR mean 256
type: NanoBEIR_mean_256
metrics:
- type: cosine_accuracy@1
value: 0.46
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.6466666666666666
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.7333333333333334
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.8133333333333334
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.46
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.27999999999999997
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.22133333333333335
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.15666666666666665
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.30916899822128535
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.4584238974577603
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.5118115079438813
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.5908955348288226
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.54044935229102
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.5749867724867724
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.43858752911278276
name: Cosine Map@100
- type: query_active_dims
value: 256
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9375
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 256
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9375
name: Corpus Sparsity Ratio
Sparse CSR model trained on Natural Questions
This is a CSR Sparse Encoder model finetuned from mixedbread-ai/mxbai-embed-large-v1 on the natural-questions dataset using the sentence-transformers library. It maps sentences & paragraphs to a 4096-dimensional sparse vector space with 256 maximum active dimensions and can be used for semantic search and sparse retrieval.
Model Details
Model Description
- Model Type: CSR Sparse Encoder
- Base model: mixedbread-ai/mxbai-embed-large-v1
- Maximum Sequence Length: 512 tokens
- Output Dimensionality: 4096 dimensions (trained with 256 maximum active dimensions)
- Similarity Function: Cosine Similarity
- Training Dataset:
- Language: en
- License: apache-2.0
Model Sources
Full Model Architecture
SparseEncoder(
(0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel
(1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): CSRSparsity({'input_dim': 1024, 'hidden_dim': 4096, 'k': 256, 'k_aux': 512, 'normalize': False, 'dead_threshold': 30})
)
Usage
Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import SparseEncoder
model = SparseEncoder("tomaarsen/csr-mxbai-embed-large-v1-nq")
queries = [
"who is cornelius in the book of acts",
]
documents = [
'Cornelius the Centurion Cornelius (Greek: Κορνήλιος) was a Roman centurion who is considered by Christians to be one of the first Gentiles to convert to the faith, as related in Acts of the Apostles.',
"Joe Ranft Ranft reunited with Lasseter when he was hired by Pixar in 1991 as their head of story.[1] There he worked on all of their films produced up to 2006; this included Toy Story (for which he received an Academy Award nomination) and A Bug's Life, as the co-story writer and others as story supervisor. His final film was Cars. He also voiced characters in many of the films, including Heimlich the caterpillar in A Bug's Life, Wheezy the penguin in Toy Story 2, and Jacques the shrimp in Finding Nemo.[1]",
'Wonderful Tonight "Wonderful Tonight" is a ballad written by Eric Clapton. It was included on Clapton\'s 1977 album Slowhand. Clapton wrote the song about Pattie Boyd.[1] The female vocal harmonies on the song are provided by Marcella Detroit (then Marcy Levy) and Yvonne Elliman.',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
Evaluation
Metrics
Sparse Information Retrieval
| Metric |
NanoMSMARCO_4 |
NanoNFCorpus_4 |
NanoNQ_4 |
| cosine_accuracy@1 |
0.12 |
0.06 |
0.04 |
| cosine_accuracy@3 |
0.16 |
0.14 |
0.08 |
| cosine_accuracy@5 |
0.26 |
0.24 |
0.16 |
| cosine_accuracy@10 |
0.34 |
0.3 |
0.26 |
| cosine_precision@1 |
0.12 |
0.06 |
0.04 |
| cosine_precision@3 |
0.0533 |
0.06 |
0.0267 |
| cosine_precision@5 |
0.052 |
0.068 |
0.032 |
| cosine_precision@10 |
0.034 |
0.064 |
0.026 |
| cosine_recall@1 |
0.12 |
0.0009 |
0.04 |
| cosine_recall@3 |
0.16 |
0.0034 |
0.08 |
| cosine_recall@5 |
0.26 |
0.0076 |
0.16 |
| cosine_recall@10 |
0.34 |
0.015 |
0.25 |
| cosine_ndcg@10 |
0.2085 |
0.0642 |
0.1245 |
| cosine_mrr@10 |
0.1689 |
0.1274 |
0.0876 |
| cosine_map@100 |
0.1829 |
0.0146 |
0.0994 |
| query_active_dims |
4.0 |
4.0 |
4.0 |
| query_sparsity_ratio |
0.999 |
0.999 |
0.999 |
| corpus_active_dims |
4.0 |
4.0 |
4.0 |
| corpus_sparsity_ratio |
0.999 |
0.999 |
0.999 |
Sparse Nano BEIR
| Metric |
Value |
| cosine_accuracy@1 |
0.0733 |
| cosine_accuracy@3 |
0.1267 |
| cosine_accuracy@5 |
0.22 |
| cosine_accuracy@10 |
0.3 |
| cosine_precision@1 |
0.0733 |
| cosine_precision@3 |
0.0467 |
| cosine_precision@5 |
0.0507 |
| cosine_precision@10 |
0.0413 |
| cosine_recall@1 |
0.0536 |
| cosine_recall@3 |
0.0811 |
| cosine_recall@5 |
0.1425 |
| cosine_recall@10 |
0.2017 |
| cosine_ndcg@10 |
0.1324 |
| cosine_mrr@10 |
0.128 |
| cosine_map@100 |
0.099 |
| query_active_dims |
4.0 |
| query_sparsity_ratio |
0.999 |
| corpus_active_dims |
4.0 |
| corpus_sparsity_ratio |
0.999 |
Sparse Information Retrieval
| Metric |
NanoMSMARCO_8 |
NanoNFCorpus_8 |
NanoNQ_8 |
| cosine_accuracy@1 |
0.14 |
0.08 |
0.08 |
| cosine_accuracy@3 |
0.24 |
0.24 |
0.24 |
| cosine_accuracy@5 |
0.32 |
0.32 |
0.3 |
| cosine_accuracy@10 |
0.42 |
0.4 |
0.44 |
| cosine_precision@1 |
0.14 |
0.08 |
0.08 |
| cosine_precision@3 |
0.08 |
0.1067 |
0.08 |
| cosine_precision@5 |
0.064 |
0.1 |
0.06 |
| cosine_precision@10 |
0.042 |
0.082 |
0.046 |
| cosine_recall@1 |
0.14 |
0.0035 |
0.08 |
| cosine_recall@3 |
0.24 |
0.0133 |
0.22 |
| cosine_recall@5 |
0.32 |
0.0186 |
0.28 |
| cosine_recall@10 |
0.42 |
0.0275 |
0.42 |
| cosine_ndcg@10 |
0.2598 |
0.0864 |
0.2377 |
| cosine_mrr@10 |
0.2109 |
0.1676 |
0.1862 |
| cosine_map@100 |
0.2209 |
0.0273 |
0.187 |
| query_active_dims |
8.0 |
8.0 |
8.0 |
| query_sparsity_ratio |
0.998 |
0.998 |
0.998 |
| corpus_active_dims |
8.0 |
8.0 |
8.0 |
| corpus_sparsity_ratio |
0.998 |
0.998 |
0.998 |
Sparse Nano BEIR
| Metric |
Value |
| cosine_accuracy@1 |
0.1 |
| cosine_accuracy@3 |
0.24 |
| cosine_accuracy@5 |
0.3133 |
| cosine_accuracy@10 |
0.42 |
| cosine_precision@1 |
0.1 |
| cosine_precision@3 |
0.0889 |
| cosine_precision@5 |
0.0747 |
| cosine_precision@10 |
0.0567 |
| cosine_recall@1 |
0.0745 |
| cosine_recall@3 |
0.1578 |
| cosine_recall@5 |
0.2062 |
| cosine_recall@10 |
0.2892 |
| cosine_ndcg@10 |
0.1946 |
| cosine_mrr@10 |
0.1882 |
| cosine_map@100 |
0.1451 |
| query_active_dims |
8.0 |
| query_sparsity_ratio |
0.998 |
| corpus_active_dims |
8.0 |
| corpus_sparsity_ratio |
0.998 |
Sparse Information Retrieval
| Metric |
NanoMSMARCO_16 |
NanoNFCorpus_16 |
NanoNQ_16 |
| cosine_accuracy@1 |
0.22 |
0.18 |
0.22 |
| cosine_accuracy@3 |
0.48 |
0.34 |
0.32 |
| cosine_accuracy@5 |
0.58 |
0.44 |
0.36 |
| cosine_accuracy@10 |
0.7 |
0.54 |
0.5 |
| cosine_precision@1 |
0.22 |
0.18 |
0.22 |
| cosine_precision@3 |
0.16 |
0.1533 |
0.1067 |
| cosine_precision@5 |
0.116 |
0.144 |
0.072 |
| cosine_precision@10 |
0.07 |
0.124 |
0.052 |
| cosine_recall@1 |
0.22 |
0.0066 |
0.22 |
| cosine_recall@3 |
0.48 |
0.0151 |
0.3 |
| cosine_recall@5 |
0.58 |
0.0238 |
0.33 |
| cosine_recall@10 |
0.7 |
0.0392 |
0.48 |
| cosine_ndcg@10 |
0.457 |
0.1345 |
0.3305 |
| cosine_mrr@10 |
0.3796 |
0.2814 |
0.2882 |
| cosine_map@100 |
0.3855 |
0.0403 |
0.2997 |
| query_active_dims |
16.0 |
16.0 |
16.0 |
| query_sparsity_ratio |
0.9961 |
0.9961 |
0.9961 |
| corpus_active_dims |
16.0 |
16.0 |
16.0 |
| corpus_sparsity_ratio |
0.9961 |
0.9961 |
0.9961 |
Sparse Nano BEIR
| Metric |
Value |
| cosine_accuracy@1 |
0.2067 |
| cosine_accuracy@3 |
0.38 |
| cosine_accuracy@5 |
0.46 |
| cosine_accuracy@10 |
0.58 |
| cosine_precision@1 |
0.2067 |
| cosine_precision@3 |
0.14 |
| cosine_precision@5 |
0.1107 |
| cosine_precision@10 |
0.082 |
| cosine_recall@1 |
0.1489 |
| cosine_recall@3 |
0.265 |
| cosine_recall@5 |
0.3113 |
| cosine_recall@10 |
0.4064 |
| cosine_ndcg@10 |
0.3073 |
| cosine_mrr@10 |
0.3164 |
| cosine_map@100 |
0.2419 |
| query_active_dims |
16.0 |
| query_sparsity_ratio |
0.9961 |
| corpus_active_dims |
16.0 |
| corpus_sparsity_ratio |
0.9961 |
Sparse Information Retrieval
| Metric |
NanoMSMARCO_32 |
NanoNFCorpus_32 |
NanoNQ_32 |
| cosine_accuracy@1 |
0.36 |
0.36 |
0.28 |
| cosine_accuracy@3 |
0.58 |
0.48 |
0.4 |
| cosine_accuracy@5 |
0.62 |
0.52 |
0.52 |
| cosine_accuracy@10 |
0.7 |
0.64 |
0.62 |
| cosine_precision@1 |
0.36 |
0.36 |
0.28 |
| cosine_precision@3 |
0.1933 |
0.24 |
0.1333 |
| cosine_precision@5 |
0.124 |
0.22 |
0.104 |
| cosine_precision@10 |
0.07 |
0.172 |
0.066 |
| cosine_recall@1 |
0.36 |
0.0151 |
0.26 |
| cosine_recall@3 |
0.58 |
0.032 |
0.37 |
| cosine_recall@5 |
0.62 |
0.0516 |
0.47 |
| cosine_recall@10 |
0.7 |
0.0704 |
0.59 |
| cosine_ndcg@10 |
0.5319 |
0.2104 |
0.4161 |
| cosine_mrr@10 |
0.4783 |
0.4425 |
0.3762 |
| cosine_map@100 |
0.4895 |
0.0709 |
0.3702 |
| query_active_dims |
32.0 |
32.0 |
32.0 |
| query_sparsity_ratio |
0.9922 |
0.9922 |
0.9922 |
| corpus_active_dims |
32.0 |
32.0 |
32.0 |
| corpus_sparsity_ratio |
0.9922 |
0.9922 |
0.9922 |
Sparse Nano BEIR
| Metric |
Value |
| cosine_accuracy@1 |
0.3333 |
| cosine_accuracy@3 |
0.4867 |
| cosine_accuracy@5 |
0.5533 |
| cosine_accuracy@10 |
0.6533 |
| cosine_precision@1 |
0.3333 |
| cosine_precision@3 |
0.1889 |
| cosine_precision@5 |
0.1493 |
| cosine_precision@10 |
0.1027 |
| cosine_recall@1 |
0.2117 |
| cosine_recall@3 |
0.3273 |
| cosine_recall@5 |
0.3805 |
| cosine_recall@10 |
0.4535 |
| cosine_ndcg@10 |
0.3862 |
| cosine_mrr@10 |
0.4324 |
| cosine_map@100 |
0.3102 |
| query_active_dims |
32.0 |
| query_sparsity_ratio |
0.9922 |
| corpus_active_dims |
32.0 |
| corpus_sparsity_ratio |
0.9922 |
Sparse Information Retrieval
| Metric |
NanoMSMARCO_64 |
NanoNFCorpus_64 |
NanoNQ_64 |
| cosine_accuracy@1 |
0.42 |
0.32 |
0.4 |
| cosine_accuracy@3 |
0.6 |
0.5 |
0.62 |
| cosine_accuracy@5 |
0.66 |
0.52 |
0.66 |
| cosine_accuracy@10 |
0.8 |
0.6 |
0.72 |
| cosine_precision@1 |
0.42 |
0.32 |
0.4 |
| cosine_precision@3 |
0.2 |
0.2933 |
0.2067 |
| cosine_precision@5 |
0.132 |
0.252 |
0.136 |
| cosine_precision@10 |
0.08 |
0.236 |
0.076 |
| cosine_recall@1 |
0.42 |
0.02 |
0.38 |
| cosine_recall@3 |
0.6 |
0.0453 |
0.58 |
| cosine_recall@5 |
0.66 |
0.0563 |
0.61 |
| cosine_recall@10 |
0.8 |
0.0893 |
0.67 |
| cosine_ndcg@10 |
0.5912 |
0.2629 |
0.5342 |
| cosine_mrr@10 |
0.5273 |
0.4175 |
0.5066 |
| cosine_map@100 |
0.5351 |
0.1057 |
0.4968 |
| query_active_dims |
64.0 |
64.0 |
64.0 |
| query_sparsity_ratio |
0.9844 |
0.9844 |
0.9844 |
| corpus_active_dims |
64.0 |
64.0 |
64.0 |
| corpus_sparsity_ratio |
0.9844 |
0.9844 |
0.9844 |
Sparse Nano BEIR
| Metric |
Value |
| cosine_accuracy@1 |
0.38 |
| cosine_accuracy@3 |
0.5733 |
| cosine_accuracy@5 |
0.6133 |
| cosine_accuracy@10 |
0.7067 |
| cosine_precision@1 |
0.38 |
| cosine_precision@3 |
0.2333 |
| cosine_precision@5 |
0.1733 |
| cosine_precision@10 |
0.1307 |
| cosine_recall@1 |
0.2733 |
| cosine_recall@3 |
0.4084 |
| cosine_recall@5 |
0.4421 |
| cosine_recall@10 |
0.5198 |
| cosine_ndcg@10 |
0.4628 |
| cosine_mrr@10 |
0.4838 |
| cosine_map@100 |
0.3792 |
| query_active_dims |
64.0 |
| query_sparsity_ratio |
0.9844 |
| corpus_active_dims |
64.0 |
| corpus_sparsity_ratio |
0.9844 |
Sparse Information Retrieval
| Metric |
NanoMSMARCO_128 |
NanoNFCorpus_128 |
NanoNQ_128 |
| cosine_accuracy@1 |
0.34 |
0.4 |
0.44 |
| cosine_accuracy@3 |
0.64 |
0.54 |
0.64 |
| cosine_accuracy@5 |
0.68 |
0.58 |
0.68 |
| cosine_accuracy@10 |
0.8 |
0.72 |
0.78 |
| cosine_precision@1 |
0.34 |
0.4 |
0.44 |
| cosine_precision@3 |
0.2133 |
0.3667 |
0.22 |
| cosine_precision@5 |
0.136 |
0.308 |
0.144 |
| cosine_precision@10 |
0.08 |
0.276 |
0.082 |
| cosine_recall@1 |
0.34 |
0.0427 |
0.41 |
| cosine_recall@3 |
0.64 |
0.0809 |
0.6 |
| cosine_recall@5 |
0.68 |
0.0941 |
0.64 |
| cosine_recall@10 |
0.8 |
0.1378 |
0.73 |
| cosine_ndcg@10 |
0.5731 |
0.332 |
0.5798 |
| cosine_mrr@10 |
0.5008 |
0.4858 |
0.5521 |
| cosine_map@100 |
0.5127 |
0.1509 |
0.5336 |
| query_active_dims |
128.0 |
128.0 |
128.0 |
| query_sparsity_ratio |
0.9688 |
0.9688 |
0.9688 |
| corpus_active_dims |
128.0 |
128.0 |
128.0 |
| corpus_sparsity_ratio |
0.9688 |
0.9688 |
0.9688 |
Sparse Nano BEIR
| Metric |
Value |
| cosine_accuracy@1 |
0.3933 |
| cosine_accuracy@3 |
0.6067 |
| cosine_accuracy@5 |
0.6467 |
| cosine_accuracy@10 |
0.7667 |
| cosine_precision@1 |
0.3933 |
| cosine_precision@3 |
0.2667 |
| cosine_precision@5 |
0.196 |
| cosine_precision@10 |
0.146 |
| cosine_recall@1 |
0.2642 |
| cosine_recall@3 |
0.4403 |
| cosine_recall@5 |
0.4714 |
| cosine_recall@10 |
0.5559 |
| cosine_ndcg@10 |
0.4949 |
| cosine_mrr@10 |
0.5129 |
| cosine_map@100 |
0.3991 |
| query_active_dims |
128.0 |
| query_sparsity_ratio |
0.9688 |
| corpus_active_dims |
128.0 |
| corpus_sparsity_ratio |
0.9688 |
Sparse Information Retrieval
| Metric |
NanoMSMARCO_256 |
NanoNFCorpus_256 |
NanoNQ_256 |
| cosine_accuracy@1 |
0.36 |
0.46 |
0.56 |
| cosine_accuracy@3 |
0.62 |
0.62 |
0.7 |
| cosine_accuracy@5 |
0.7 |
0.74 |
0.76 |
| cosine_accuracy@10 |
0.84 |
0.78 |
0.82 |
| cosine_precision@1 |
0.36 |
0.46 |
0.56 |
| cosine_precision@3 |
0.2067 |
0.3867 |
0.2467 |
| cosine_precision@5 |
0.14 |
0.364 |
0.16 |
| cosine_precision@10 |
0.084 |
0.298 |
0.088 |
| cosine_recall@1 |
0.36 |
0.0475 |
0.52 |
| cosine_recall@3 |
0.62 |
0.0853 |
0.67 |
| cosine_recall@5 |
0.7 |
0.1154 |
0.72 |
| cosine_recall@10 |
0.84 |
0.1527 |
0.78 |
| cosine_ndcg@10 |
0.5935 |
0.3632 |
0.6647 |
| cosine_mrr@10 |
0.5159 |
0.5609 |
0.6482 |
| cosine_map@100 |
0.5228 |
0.1677 |
0.6253 |
| query_active_dims |
256.0 |
256.0 |
256.0 |
| query_sparsity_ratio |
0.9375 |
0.9375 |
0.9375 |
| corpus_active_dims |
256.0 |
256.0 |
256.0 |
| corpus_sparsity_ratio |
0.9375 |
0.9375 |
0.9375 |
Sparse Nano BEIR
| Metric |
Value |
| cosine_accuracy@1 |
0.46 |
| cosine_accuracy@3 |
0.6467 |
| cosine_accuracy@5 |
0.7333 |
| cosine_accuracy@10 |
0.8133 |
| cosine_precision@1 |
0.46 |
| cosine_precision@3 |
0.28 |
| cosine_precision@5 |
0.2213 |
| cosine_precision@10 |
0.1567 |
| cosine_recall@1 |
0.3092 |
| cosine_recall@3 |
0.4584 |
| cosine_recall@5 |
0.5118 |
| cosine_recall@10 |
0.5909 |
| cosine_ndcg@10 |
0.5404 |
| cosine_mrr@10 |
0.575 |
| cosine_map@100 |
0.4386 |
| query_active_dims |
256.0 |
| query_sparsity_ratio |
0.9375 |
| corpus_active_dims |
256.0 |
| corpus_sparsity_ratio |
0.9375 |
Training Details
Training Dataset
natural-questions
- Dataset: natural-questions at f9e894e
- Size: 99,000 training samples
- Columns:
query and answer
- Approximate statistics based on the first 1000 samples:
|
query |
answer |
| type |
string |
string |
| details |
- min: 10 tokens
- mean: 11.71 tokens
- max: 26 tokens
|
- min: 4 tokens
- mean: 131.81 tokens
- max: 450 tokens
|
- Samples:
| query |
answer |
who played the father in papa don't preach |
Alex McArthur Alex McArthur (born March 6, 1957) is an American actor. |
where was the location of the battle of hastings |
Battle of Hastings The Battle of Hastings[a] was fought on 14 October 1066 between the Norman-French army of William, the Duke of Normandy, and an English army under the Anglo-Saxon King Harold Godwinson, beginning the Norman conquest of England. It took place approximately 7 miles (11 kilometres) northwest of Hastings, close to the present-day town of Battle, East Sussex, and was a decisive Norman victory. |
how many puppies can a dog give birth to |
Canine reproduction The largest litter size to date was set by a Neapolitan Mastiff in Manea, Cambridgeshire, UK on November 29, 2004; the litter was 24 puppies.[22] |
- Loss:
CSRLoss with these parameters:{
"beta": 0.1,
"gamma": 0.1,
"loss": "SparseMultipleNegativesRankingLoss(scale=20.0, similarity_fct='cos_sim')"
}
Evaluation Dataset
natural-questions
- Dataset: natural-questions at f9e894e
- Size: 1,000 evaluation samples
- Columns:
query and answer
- Approximate statistics based on the first 1000 samples:
|
query |
answer |
| type |
string |
string |
| details |
- min: 10 tokens
- mean: 11.69 tokens
- max: 23 tokens
|
- min: 15 tokens
- mean: 134.01 tokens
- max: 512 tokens
|
- Samples:
| query |
answer |
where is the tiber river located in italy |
Tiber The Tiber (/ˈtaɪbər/, Latin: Tiberis,[1] Italian: Tevere [ˈteːvere])[2] is the third-longest river in Italy, rising in the Apennine Mountains in Emilia-Romagna and flowing 406 kilometres (252 mi) through Tuscany, Umbria and Lazio, where it is joined by the river Aniene, to the Tyrrhenian Sea, between Ostia and Fiumicino.[3] It drains a basin estimated at 17,375 square kilometres (6,709 sq mi). The river has achieved lasting fame as the main watercourse of the city of Rome, founded on its eastern banks. |
what kind of car does jay gatsby drive |
Jay Gatsby At the Buchanan home, Jordan Baker, Nick, Jay, and the Buchanans decide to visit New York City. Tom borrows Gatsby's yellow Rolls Royce to drive up to the city. On the way to New York City, Tom makes a detour at a gas station in "the Valley of Ashes", a run-down part of Long Island. The owner, George Wilson, shares his concern that his wife, Myrtle, may be having an affair. This unnerves Tom, who has been having an affair with Myrtle, and he leaves in a hurry. |
who sings if i can dream about you |
I Can Dream About You "I Can Dream About You" is a song performed by American singer Dan Hartman on the soundtrack album of the film Streets of Fire. Released in 1984 as a single from the soundtrack, and included on Hartman's album I Can Dream About You, it reached number 6 on the Billboard Hot 100.[1] |
- Loss:
CSRLoss with these parameters:{
"beta": 0.1,
"gamma": 0.1,
"loss": "SparseMultipleNegativesRankingLoss(scale=20.0, similarity_fct='cos_sim')"
}
Training Hyperparameters
Non-Default Hyperparameters
eval_strategy: stepsper_device_train_batch_size: 64per_device_eval_batch_size: 64learning_rate: 4e-05num_train_epochs: 1bf16: Truebatch_sampler: no_duplicates
All Hyperparameters
Click to expand
overwrite_output_dir: Falsedo_predict: Falseeval_strategy: stepsprediction_loss_only: Trueper_device_train_batch_size: 64per_device_eval_batch_size: 64per_gpu_train_batch_size: Noneper_gpu_eval_batch_size: Nonegradient_accumulation_steps: 1eval_accumulation_steps: Nonetorch_empty_cache_steps: Nonelearning_rate: 4e-05weight_decay: 0.0adam_beta1: 0.9adam_beta2: 0.999adam_epsilon: 1e-08max_grad_norm: 1.0num_train_epochs: 1max_steps: -1lr_scheduler_type: linearlr_scheduler_kwargs: {}warmup_ratio: 0.0warmup_steps: 0log_level: passivelog_level_replica: warninglog_on_each_node: Truelogging_nan_inf_filter: Truesave_safetensors: Truesave_on_each_node: Falsesave_only_model: Falserestore_callback_states_from_checkpoint: Falseno_cuda: Falseuse_cpu: Falseuse_mps_device: Falseseed: 42data_seed: Nonejit_mode_eval: Falseuse_ipex: Falsebf16: Truefp16: Falsefp16_opt_level: O1half_precision_backend: autobf16_full_eval: Falsefp16_full_eval: Falsetf32: Nonelocal_rank: 0ddp_backend: Nonetpu_num_cores: Nonetpu_metrics_debug: Falsedebug: []dataloader_drop_last: Falsedataloader_num_workers: 0dataloader_prefetch_factor: Nonepast_index: -1disable_tqdm: Falseremove_unused_columns: Truelabel_names: Noneload_best_model_at_end: Falseignore_data_skip: Falsefsdp: []fsdp_min_num_params: 0fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}fsdp_transformer_layer_cls_to_wrap: Noneaccelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}deepspeed: Nonelabel_smoothing_factor: 0.0optim: adamw_torchoptim_args: Noneadafactor: Falsegroup_by_length: Falselength_column_name: lengthddp_find_unused_parameters: Noneddp_bucket_cap_mb: Noneddp_broadcast_buffers: Falsedataloader_pin_memory: Truedataloader_persistent_workers: Falseskip_memory_metrics: Trueuse_legacy_prediction_loop: Falsepush_to_hub: Falseresume_from_checkpoint: Nonehub_model_id: Nonehub_strategy: every_savehub_private_repo: Nonehub_always_push: Falsegradient_checkpointing: Falsegradient_checkpointing_kwargs: Noneinclude_inputs_for_metrics: Falseinclude_for_metrics: []eval_do_concat_batches: Truefp16_backend: autopush_to_hub_model_id: Nonepush_to_hub_organization: Nonemp_parameters: auto_find_batch_size: Falsefull_determinism: Falsetorchdynamo: Noneray_scope: lastddp_timeout: 1800torch_compile: Falsetorch_compile_backend: Nonetorch_compile_mode: Noneinclude_tokens_per_second: Falseinclude_num_input_tokens_seen: Falseneftune_noise_alpha: Noneoptim_target_modules: Nonebatch_eval_metrics: Falseeval_on_start: Falseuse_liger_kernel: Falseeval_use_gather_object: Falseaverage_tokens_across_devices: Falseprompts: Nonebatch_sampler: no_duplicatesmulti_dataset_batch_sampler: proportionalrouter_mapping: {}learning_rate_mapping: {}
Training Logs
| Epoch |
Step |
Training Loss |
Validation Loss |
NanoMSMARCO_4_cosine_ndcg@10 |
NanoNFCorpus_4_cosine_ndcg@10 |
NanoNQ_4_cosine_ndcg@10 |
NanoBEIR_mean_4_cosine_ndcg@10 |
NanoMSMARCO_8_cosine_ndcg@10 |
NanoNFCorpus_8_cosine_ndcg@10 |
NanoNQ_8_cosine_ndcg@10 |
NanoBEIR_mean_8_cosine_ndcg@10 |
NanoMSMARCO_16_cosine_ndcg@10 |
NanoNFCorpus_16_cosine_ndcg@10 |
NanoNQ_16_cosine_ndcg@10 |
NanoBEIR_mean_16_cosine_ndcg@10 |
NanoMSMARCO_32_cosine_ndcg@10 |
NanoNFCorpus_32_cosine_ndcg@10 |
NanoNQ_32_cosine_ndcg@10 |
NanoBEIR_mean_32_cosine_ndcg@10 |
NanoMSMARCO_64_cosine_ndcg@10 |
NanoNFCorpus_64_cosine_ndcg@10 |
NanoNQ_64_cosine_ndcg@10 |
NanoBEIR_mean_64_cosine_ndcg@10 |
NanoMSMARCO_128_cosine_ndcg@10 |
NanoNFCorpus_128_cosine_ndcg@10 |
NanoNQ_128_cosine_ndcg@10 |
NanoBEIR_mean_128_cosine_ndcg@10 |
NanoMSMARCO_256_cosine_ndcg@10 |
NanoNFCorpus_256_cosine_ndcg@10 |
NanoNQ_256_cosine_ndcg@10 |
NanoBEIR_mean_256_cosine_ndcg@10 |
| -1 |
-1 |
- |
- |
0.1587 |
0.0673 |
0.0962 |
0.1074 |
0.2787 |
0.0843 |
0.2254 |
0.1962 |
0.4270 |
0.1786 |
0.3601 |
0.3219 |
0.5226 |
0.2079 |
0.4714 |
0.4006 |
0.6018 |
0.2616 |
0.5733 |
0.4789 |
0.6019 |
0.3201 |
0.6425 |
0.5215 |
0.6480 |
0.3496 |
0.6699 |
0.5558 |
| 0.0646 |
100 |
0.3153 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.1293 |
200 |
0.2764 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.1939 |
300 |
0.2646 |
0.2497 |
0.1417 |
0.0671 |
0.1031 |
0.1040 |
0.2714 |
0.1042 |
0.2025 |
0.1927 |
0.3948 |
0.1421 |
0.3478 |
0.2949 |
0.5338 |
0.1954 |
0.4266 |
0.3852 |
0.6107 |
0.2885 |
0.5707 |
0.4900 |
0.5864 |
0.3582 |
0.6326 |
0.5257 |
0.6045 |
0.3607 |
0.6362 |
0.5338 |
| 0.2586 |
400 |
0.2572 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.3232 |
500 |
0.2521 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.3878 |
600 |
0.2485 |
0.2365 |
0.1768 |
0.0722 |
0.1584 |
0.1358 |
0.2110 |
0.0697 |
0.2194 |
0.1667 |
0.3999 |
0.1301 |
0.3274 |
0.2858 |
0.5493 |
0.2184 |
0.4476 |
0.4051 |
0.5867 |
0.2808 |
0.5253 |
0.4643 |
0.5823 |
0.3298 |
0.5948 |
0.5023 |
0.5816 |
0.3532 |
0.6561 |
0.5303 |
| 0.4525 |
700 |
0.2456 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.5171 |
800 |
0.2431 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.5818 |
900 |
0.2412 |
0.2301 |
0.1837 |
0.0763 |
0.1371 |
0.1324 |
0.2875 |
0.0834 |
0.2195 |
0.1968 |
0.4224 |
0.1298 |
0.3448 |
0.2990 |
0.5197 |
0.2075 |
0.4749 |
0.4007 |
0.6067 |
0.2714 |
0.5342 |
0.4708 |
0.6101 |
0.3247 |
0.6003 |
0.5117 |
0.5662 |
0.3652 |
0.6407 |
0.5240 |
| 0.6464 |
1000 |
0.2397 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.7111 |
1100 |
0.2378 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.7757 |
1200 |
0.2375 |
0.2267 |
0.1783 |
0.0569 |
0.1241 |
0.1198 |
0.2543 |
0.1010 |
0.1927 |
0.1827 |
0.4190 |
0.1357 |
0.3332 |
0.2959 |
0.5284 |
0.2205 |
0.4416 |
0.3968 |
0.5786 |
0.2487 |
0.5570 |
0.4614 |
0.5783 |
0.3295 |
0.6148 |
0.5075 |
0.5860 |
0.3670 |
0.6558 |
0.5363 |
| 0.8403 |
1300 |
0.2372 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.9050 |
1400 |
0.2357 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.9696 |
1500 |
0.236 |
0.2255 |
0.2011 |
0.0670 |
0.1246 |
0.1309 |
0.2540 |
0.0858 |
0.2371 |
0.1923 |
0.4558 |
0.1372 |
0.3172 |
0.3034 |
0.5263 |
0.2110 |
0.4061 |
0.3811 |
0.5971 |
0.2639 |
0.5188 |
0.4599 |
0.5752 |
0.3326 |
0.5755 |
0.4945 |
0.5886 |
0.3658 |
0.6536 |
0.5360 |
| -1 |
-1 |
- |
- |
0.2085 |
0.0642 |
0.1245 |
0.1324 |
0.2598 |
0.0864 |
0.2377 |
0.1946 |
0.4570 |
0.1345 |
0.3305 |
0.3073 |
0.5319 |
0.2104 |
0.4161 |
0.3862 |
0.5912 |
0.2629 |
0.5342 |
0.4628 |
0.5731 |
0.3320 |
0.5798 |
0.4949 |
0.5935 |
0.3632 |
0.6647 |
0.5404 |
Environmental Impact
Carbon emissions were measured using CodeCarbon.
- Energy Consumed: 0.272 kWh
- Carbon Emitted: 0.106 kg of CO2
- Hours Used: 0.75 hours
Training Hardware
- On Cloud: No
- GPU Model: 1 x NVIDIA GeForce RTX 3090
- CPU Model: 13th Gen Intel(R) Core(TM) i7-13700K
- RAM Size: 31.78 GB
Framework Versions
- Python: 3.11.6
- Sentence Transformers: 4.2.0.dev0
- Transformers: 4.52.4
- PyTorch: 2.6.0+cu124
- Accelerate: 1.5.1
- Datasets: 2.21.0
- Tokenizers: 0.21.1
Citation
BibTeX
Sentence Transformers
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
CSRLoss
@misc{wen2025matryoshkarevisitingsparsecoding,
title={Beyond Matryoshka: Revisiting Sparse Coding for Adaptive Representation},
author={Tiansheng Wen and Yifei Wang and Zequn Zeng and Zhong Peng and Yudi Su and Xinyang Liu and Bo Chen and Hongwei Liu and Stefanie Jegelka and Chenyu You},
year={2025},
eprint={2503.01776},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2503.01776},
}
SparseMultipleNegativesRankingLoss
@misc{henderson2017efficient,
title={Efficient Natural Language Response Suggestion for Smart Reply},
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
year={2017},
eprint={1705.00652},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
