sentence-transformers safetensors roberta sentence-similarity feature-extraction generated_from_trainer dataset_size:4232571 loss:MaskedCachedMultipleNegativesRankingLoss arxiv:1908.10084 arxiv:2101.06983 base_model:sentence-transformers/all-distilroberta-v1 base_model:finetune:sentence-transformers/all-distilroberta-v1 autotrain_compatible text-embeddings-inference endpoints_compatible region:us
---
base_model: sentence-transformers/all-distilroberta-v1
library_name: sentence-transformers
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:4232571
- loss:MaskedCachedMultipleNegativesRankingLoss
widget:
- source_sentence: Mathlib.Data.ENNReal.Operations#190
  sentences:
  - Finset.Nonempty.cons_induction
  - Set.mem_iUnion
  - exists_prop
- source_sentence: Mathlib.LinearAlgebra.CliffordAlgebra.Basic#46
  sentences:
  - Int.sub_nonneg_of_le
  - SMulCommClass.symm
  - CategoryTheory.Category.id_comp
- source_sentence: Mathlib.AlgebraicGeometry.ProjectiveSpectrum.StructureSheaf#23
  sentences:
  - hasFDerivAt_pi
  - Set.IsPWO.image_of_monotone
  - Pi.mul_apply
- source_sentence: Mathlib.Data.Nat.ModEq#84
  sentences:
  - Nat.gcd_dvd_left
  - neg_neg
  - map_list_sum
- source_sentence: Mathlib.AlgebraicGeometry.Noetherian#22
  sentences:
  - AlgebraicGeometry.of_affine_open_cover
  - pow_lt_pow_right_of_lt_one₀
  - LT.lt.trans_le
---

# SentenceTransformer based on sentence-transformers/all-distilroberta-v1

This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/all-distilroberta-v1](https://huggingface.co/sentence-transformers/all-distilroberta-v1). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [sentence-transformers/all-distilroberta-v1](https://huggingface.co/sentence-transformers/all-distilroberta-v1) <!-- at revision 8d88b92a34345fd6a139aa47768c9881720006ce -->
- **Maximum Sequence Length:** 512 tokens
- **Output Dimensionality:** 768 tokens
- **Similarity Function:** Cosine Similarity
<!-- - **Training Dataset:** Unknown -->
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->

### Model Sources

- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)

### Full Model Architecture

```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: RobertaModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)
```

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("hanwenzhu/all-distilroberta-v1-lr2e-4-bs1024-nneg3-ml-feb04")
# Run inference
sentences = [
    'Mathlib.AlgebraicGeometry.Noetherian#22',
    'AlgebraicGeometry.of_affine_open_cover',
    'pow_lt_pow_right_of_lt_one₀',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```

<!--
### Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details>
-->

<!--
### Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details>
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Dataset

#### Unnamed Dataset


* Size: 4,232,571 training samples
* Columns: <code>state_name</code> and <code>premise_name</code>
* Approximate statistics based on the first 1000 samples:
  |         | state_name                                                                         | premise_name                                                                      |
  |:--------|:-----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|
  | type    | string                                                                             | string                                                                            |
  | details | <ul><li>min: 11 tokens</li><li>mean: 16.91 tokens</li><li>max: 28 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 10.27 tokens</li><li>max: 27 tokens</li></ul> |
* Samples:
  | state_name                                               | premise_name                         |
  |:---------------------------------------------------------|:-------------------------------------|
  | <code>Mathlib.Algebra.Group.Subgroup.Pointwise#27</code> | <code>Set.mul_subgroupClosure</code> |
  | <code>Mathlib.Algebra.Group.Subgroup.Pointwise#27</code> | <code>pow_succ</code>                |
  | <code>Mathlib.Algebra.Group.Subgroup.Pointwise#27</code> | <code>mul_assoc</code>               |
* Loss: <code>loss.MaskedCachedMultipleNegativesRankingLoss</code> with these parameters:
  ```json
  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  ```

### Evaluation Dataset

#### Unnamed Dataset


* Size: 1,648 evaluation samples
* Columns: <code>state_name</code> and <code>premise_name</code>
* Approximate statistics based on the first 1000 samples:
  |         | state_name                                                                         | premise_name                                                                     |
  |:--------|:-----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|
  | type    | string                                                                             | string                                                                           |
  | details | <ul><li>min: 12 tokens</li><li>mean: 17.34 tokens</li><li>max: 26 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 10.9 tokens</li><li>max: 34 tokens</li></ul> |
* Samples:
  | state_name                                             | premise_name                          |
  |:-------------------------------------------------------|:--------------------------------------|
  | <code>Mathlib.Algebra.BigOperators.Associated#0</code> | <code>Prime.dvd_or_dvd</code>         |
  | <code>Mathlib.Algebra.BigOperators.Associated#0</code> | <code>Multiset.induction_on</code>    |
  | <code>Mathlib.Algebra.BigOperators.Associated#0</code> | <code>Multiset.mem_cons_of_mem</code> |
* Loss: <code>loss.MaskedCachedMultipleNegativesRankingLoss</code> with these parameters:
  ```json
  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  ```

### Training Hyperparameters
#### Non-Default Hyperparameters

- `eval_strategy`: steps
- `per_device_train_batch_size`: 1024
- `per_device_eval_batch_size`: 64
- `learning_rate`: 0.0002
- `num_train_epochs`: 1.0
- `lr_scheduler_type`: cosine
- `warmup_ratio`: 0.03
- `bf16`: True
- `dataloader_num_workers`: 4
- `batch_sampler`: no_duplicates

#### All Hyperparameters
<details><summary>Click to expand</summary>

- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: steps
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 1024
- `per_device_eval_batch_size`: 64
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 0.0002
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 1.0
- `max_steps`: -1
- `lr_scheduler_type`: cosine
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.03
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: True
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 4
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: False
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: False
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`: 
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `use_liger_kernel`: False
- `eval_use_gather_object`: False
- `batch_sampler`: no_duplicates
- `multi_dataset_batch_sampler`: proportional

</details>

### Training Logs
<details><summary>Click to expand</summary>

| Epoch  | Step | Training Loss | loss   |
|:------:|:----:|:-------------:|:------:|
| 0.0024 | 10   | 6.4677        | -      |
| 0.0048 | 20   | 5.8525        | -      |
| 0.0073 | 30   | 5.4638        | -      |
| 0.0097 | 40   | 5.1916        | -      |
| 0.0102 | 42   | -             | 1.8024 |
| 0.0121 | 50   | 5.0406        | -      |
| 0.0145 | 60   | 4.9618        | -      |
| 0.0169 | 70   | 4.8628        | -      |
| 0.0194 | 80   | 4.8094        | -      |
| 0.0203 | 84   | -             | 1.6661 |
| 0.0218 | 90   | 4.728         | -      |
| 0.0242 | 100  | 4.6682        | -      |
| 0.0266 | 110  | 4.6147        | -      |
| 0.0290 | 120  | 4.6194        | -      |
| 0.0305 | 126  | -             | 1.4863 |
| 0.0314 | 130  | 4.5386        | -      |
| 0.0339 | 140  | 4.4644        | -      |
| 0.0363 | 150  | 4.4456        | -      |
| 0.0387 | 160  | 4.4112        | -      |
| 0.0406 | 168  | -             | 1.4766 |
| 0.0411 | 170  | 4.3334        | -      |
| 0.0435 | 180  | 4.326         | -      |
| 0.0460 | 190  | 4.2364        | -      |
| 0.0484 | 200  | 4.232         | -      |
| 0.0508 | 210  | 4.211         | 1.2885 |
| 0.0532 | 220  | 4.16          | -      |
| 0.0556 | 230  | 4.1569        | -      |
| 0.0581 | 240  | 4.1202        | -      |
| 0.0605 | 250  | 4.0603        | -      |
| 0.0610 | 252  | -             | 1.3337 |
| 0.0629 | 260  | 4.0629        | -      |
| 0.0653 | 270  | 4.0401        | -      |
| 0.0677 | 280  | 4.0171        | -      |
| 0.0701 | 290  | 4.0034        | -      |
| 0.0711 | 294  | -             | 1.2049 |
| 0.0726 | 300  | 3.9911        | -      |
| 0.0750 | 310  | 3.893         | -      |
| 0.0774 | 320  | 3.8999        | -      |
| 0.0798 | 330  | 3.9395        | -      |
| 0.0813 | 336  | -             | 1.2090 |
| 0.0822 | 340  | 3.9087        | -      |
| 0.0847 | 350  | 3.925         | -      |
| 0.0871 | 360  | 3.8888        | -      |
| 0.0895 | 370  | 3.8282        | -      |
| 0.0914 | 378  | -             | 1.2345 |
| 0.0919 | 380  | 3.8787        | -      |
| 0.0943 | 390  | 3.8517        | -      |
| 0.0968 | 400  | 3.7446        | -      |
| 0.0992 | 410  | 3.7888        | -      |
| 0.1016 | 420  | 3.7833        | 1.2055 |
| 0.1040 | 430  | 3.7522        | -      |
| 0.1064 | 440  | 3.7601        | -      |
| 0.1089 | 450  | 3.7093        | -      |
| 0.1113 | 460  | 3.713         | -      |
| 0.1118 | 462  | -             | 1.2316 |
| 0.1137 | 470  | 3.6461        | -      |
| 0.1161 | 480  | 3.6153        | -      |
| 0.1185 | 490  | 3.6419        | -      |
| 0.1209 | 500  | 3.5929        | -      |
| 0.1219 | 504  | -             | 1.1385 |
| 0.1234 | 510  | 3.5881        | -      |
| 0.1258 | 520  | 3.6296        | -      |
| 0.1282 | 530  | 3.6302        | -      |
| 0.1306 | 540  | 3.59          | -      |
| 0.1321 | 546  | -             | 1.1055 |
| 0.1330 | 550  | 3.5682        | -      |
| 0.1355 | 560  | 3.5808        | -      |
| 0.1379 | 570  | 3.4821        | -      |
| 0.1403 | 580  | 3.5309        | -      |
| 0.1422 | 588  | -             | 1.1142 |
| 0.1427 | 590  | 3.4714        | -      |
| 0.1451 | 600  | 3.4859        | -      |
| 0.1476 | 610  | 3.4835        | -      |
| 0.1500 | 620  | 3.4335        | -      |
| 0.1524 | 630  | 3.459         | 1.0673 |
| 0.1548 | 640  | 3.4735        | -      |
| 0.1572 | 650  | 3.3819        | -      |
| 0.1597 | 660  | 3.3618        | -      |
| 0.1621 | 670  | 3.317         | -      |
| 0.1626 | 672  | -             | 1.0852 |
| 0.1645 | 680  | 3.3512        | -      |
| 0.1669 | 690  | 3.3519        | -      |
| 0.1693 | 700  | 3.3672        | -      |
| 0.1717 | 710  | 3.3021        | -      |
| 0.1727 | 714  | -             | 1.0624 |
| 0.1742 | 720  | 3.3557        | -      |
| 0.1766 | 730  | 3.3778        | -      |
| 0.1790 | 740  | 3.2999        | -      |
| 0.1814 | 750  | 3.2691        | -      |
| 0.1829 | 756  | -             | 1.0505 |
| 0.1838 | 760  | 3.268         | -      |
| 0.1863 | 770  | 3.2883        | -      |
| 0.1887 | 780  | 3.2546        | -      |
| 0.1911 | 790  | 3.2313        | -      |
| 0.1930 | 798  | -             | 1.0156 |
| 0.1935 | 800  | 3.2847        | -      |
| 0.1959 | 810  | 3.2852        | -      |
| 0.1984 | 820  | 3.2001        | -      |
| 0.2008 | 830  | 3.2561        | -      |
| 0.2032 | 840  | 3.1764        | 0.9978 |
| 0.2056 | 850  | 3.1471        | -      |
| 0.2080 | 860  | 3.1506        | -      |
| 0.2104 | 870  | 3.1225        | -      |
| 0.2129 | 880  | 3.2139        | -      |
| 0.2134 | 882  | -             | 0.9681 |
| 0.2153 | 890  | 3.1746        | -      |
| 0.2177 | 900  | 3.0522        | -      |
| 0.2201 | 910  | 3.1581        | -      |
| 0.2225 | 920  | 3.1535        | -      |
| 0.2235 | 924  | -             | 0.9900 |
| 0.2250 | 930  | 3.1659        | -      |
| 0.2274 | 940  | 3.069         | -      |
| 0.2298 | 950  | 3.1406        | -      |
| 0.2322 | 960  | 3.1171        | -      |
| 0.2337 | 966  | -             | 0.9881 |
| 0.2346 | 970  | 3.0983        | -      |
| 0.2371 | 980  | 3.0748        | -      |
| 0.2395 | 990  | 3.0279        | -      |
| 0.2419 | 1000 | 3.0465        | -      |
| 0.2438 | 1008 | -             | 0.9158 |
| 0.2443 | 1010 | 2.9777        | -      |
| 0.2467 | 1020 | 2.9791        | -      |
| 0.2492 | 1030 | 2.9773        | -      |
| 0.2516 | 1040 | 3.0372        | -      |
| 0.2540 | 1050 | 2.998         | 0.9662 |
| 0.2564 | 1060 | 3.0012        | -      |
| 0.2588 | 1070 | 2.9436        | -      |
| 0.2612 | 1080 | 2.9661        | -      |
| 0.2637 | 1090 | 3.0087        | -      |
| 0.2642 | 1092 | -             | 0.9232 |
| 0.2661 | 1100 | 2.9713        | -      |
| 0.2685 | 1110 | 2.9653        | -      |
| 0.2709 | 1120 | 2.9268        | -      |
| 0.2733 | 1130 | 2.9321        | -      |
| 0.2743 | 1134 | -             | 0.9229 |
| 0.2758 | 1140 | 2.884         | -      |
| 0.2782 | 1150 | 2.9282        | -      |
| 0.2806 | 1160 | 2.9197        | -      |
| 0.2830 | 1170 | 2.8831        | -      |
| 0.2845 | 1176 | -             | 0.9419 |
| 0.2854 | 1180 | 2.8617        | -      |
| 0.2879 | 1190 | 2.8926        | -      |
| 0.2903 | 1200 | 2.8259        | -      |
| 0.2927 | 1210 | 2.7944        | -      |
| 0.2946 | 1218 | -             | 0.8967 |
| 0.2951 | 1220 | 2.7952        | -      |
| 0.2975 | 1230 | 2.8103        | -      |
| 0.3000 | 1240 | 2.7346        | -      |
| 0.3024 | 1250 | 2.7674        | -      |
| 0.3048 | 1260 | 2.7845        | 0.9320 |
| 0.3072 | 1270 | 2.8123        | -      |
| 0.3096 | 1280 | 2.7938        | -      |
| 0.3120 | 1290 | 2.8232        | -      |
| 0.3145 | 1300 | 2.7676        | -      |
| 0.3149 | 1302 | -             | 0.8623 |
| 0.3169 | 1310 | 2.756         | -      |
| 0.3193 | 1320 | 2.7683        | -      |
| 0.3217 | 1330 | 2.8046        | -      |
| 0.3241 | 1340 | 2.8106        | -      |
| 0.3251 | 1344 | -             | 0.8718 |
| 0.3266 | 1350 | 2.7349        | -      |
| 0.3290 | 1360 | 2.747         | -      |
| 0.3314 | 1370 | 2.7276        | -      |
| 0.3338 | 1380 | 2.764         | -      |
| 0.3353 | 1386 | -             | 0.8143 |
| 0.3362 | 1390 | 2.7233        | -      |
| 0.3387 | 1400 | 2.694         | -      |
| 0.3411 | 1410 | 2.7002        | -      |
| 0.3435 | 1420 | 2.705         | -      |
| 0.3454 | 1428 | -             | 0.8237 |
| 0.3459 | 1430 | 2.6625        | -      |
| 0.3483 | 1440 | 2.6954        | -      |
| 0.3507 | 1450 | 2.6671        | -      |
| 0.3532 | 1460 | 2.6801        | -      |
| 0.3556 | 1470 | 2.6724        | 0.8144 |
| 0.3580 | 1480 | 2.6525        | -      |
| 0.3604 | 1490 | 2.6905        | -      |
| 0.3628 | 1500 | 2.62          | -      |
| 0.3653 | 1510 | 2.6563        | -      |
| 0.3657 | 1512 | -             | 0.8374 |
| 0.3677 | 1520 | 2.6774        | -      |
| 0.3701 | 1530 | 2.6183        | -      |
| 0.3725 | 1540 | 2.6164        | -      |
| 0.3749 | 1550 | 2.6187        | -      |
| 0.3759 | 1554 | -             | 0.8515 |
| 0.3774 | 1560 | 2.6186        | -      |
| 0.3798 | 1570 | 2.6555        | -      |
| 0.3822 | 1580 | 2.6541        | -      |
| 0.3846 | 1590 | 2.6595        | -      |
| 0.3861 | 1596 | -             | 0.8322 |
| 0.3870 | 1600 | 2.6699        | -      |
| 0.3895 | 1610 | 2.6404        | -      |
| 0.3919 | 1620 | 2.5847        | -      |
| 0.3943 | 1630 | 2.6133        | -      |
| 0.3962 | 1638 | -             | 0.8247 |
| 0.3967 | 1640 | 2.6379        | -      |
| 0.3991 | 1650 | 2.6635        | -      |
| 0.4015 | 1660 | 2.6122        | -      |
| 0.4040 | 1670 | 2.6352        | -      |
| 0.4064 | 1680 | 2.5765        | 0.8084 |
| 0.4088 | 1690 | 2.58          | -      |
| 0.4112 | 1700 | 2.5415        | -      |
| 0.4136 | 1710 | 2.5199        | -      |
| 0.4161 | 1720 | 2.5341        | -      |
| 0.4165 | 1722 | -             | 0.8113 |
| 0.4185 | 1730 | 2.5443        | -      |
| 0.4209 | 1740 | 2.5558        | -      |
| 0.4233 | 1750 | 2.547         | -      |
| 0.4257 | 1760 | 2.5383        | -      |
| 0.4267 | 1764 | -             | 0.7759 |
| 0.4282 | 1770 | 2.5656        | -      |
| 0.4306 | 1780 | 2.5711        | -      |
| 0.4330 | 1790 | 2.4755        | -      |
| 0.4354 | 1800 | 2.4792        | -      |
| 0.4369 | 1806 | -             | 0.7821 |
| 0.4378 | 1810 | 2.489         | -      |
| 0.4403 | 1820 | 2.4752        | -      |
| 0.4427 | 1830 | 2.4708        | -      |
| 0.4451 | 1840 | 2.5073        | -      |
| 0.4470 | 1848 | -             | 0.7817 |
| 0.4475 | 1850 | 2.4888        | -      |
| 0.4499 | 1860 | 2.4498        | -      |
| 0.4523 | 1870 | 2.4089        | -      |
| 0.4548 | 1880 | 2.4877        | -      |
| 0.4572 | 1890 | 2.4617        | 0.7354 |
| 0.4596 | 1900 | 2.4715        | -      |
| 0.4620 | 1910 | 2.4175        | -      |
| 0.4644 | 1920 | 2.4554        | -      |
| 0.4669 | 1930 | 2.4406        | -      |
| 0.4673 | 1932 | -             | 0.7383 |
| 0.4693 | 1940 | 2.4692        | -      |
| 0.4717 | 1950 | 2.4418        | -      |
| 0.4741 | 1960 | 2.5003        | -      |
| 0.4765 | 1970 | 2.4467        | -      |
| 0.4775 | 1974 | -             | 0.7271 |
| 0.4790 | 1980 | 2.4256        | -      |
| 0.4814 | 1990 | 2.4249        | -      |
| 0.4838 | 2000 | 2.3878        | -      |
| 0.4862 | 2010 | 2.3596        | -      |
| 0.4877 | 2016 | -             | 0.7600 |
| 0.4886 | 2020 | 2.3905        | -      |
| 0.4910 | 2030 | 2.4118        | -      |
| 0.4935 | 2040 | 2.3785        | -      |
| 0.4959 | 2050 | 2.4141        | -      |
| 0.4978 | 2058 | -             | 0.7464 |
| 0.4983 | 2060 | 2.3821        | -      |
| 0.5007 | 2070 | 2.3513        | -      |
| 0.5031 | 2080 | 2.3928        | -      |
| 0.5056 | 2090 | 2.3406        | -      |
| 0.5080 | 2100 | 2.3967        | 0.7483 |
| 0.5104 | 2110 | 2.4006        | -      |
| 0.5128 | 2120 | 2.3474        | -      |
| 0.5152 | 2130 | 2.395         | -      |
| 0.5177 | 2140 | 2.3809        | -      |
| 0.5181 | 2142 | -             | 0.7415 |
| 0.5201 | 2150 | 2.3646        | -      |
| 0.5225 | 2160 | 2.3675        | -      |
| 0.5249 | 2170 | 2.3295        | -      |
| 0.5273 | 2180 | 2.3365        | -      |
| 0.5283 | 2184 | -             | 0.7332 |
| 0.5298 | 2190 | 2.3974        | -      |
| 0.5322 | 2200 | 2.2649        | -      |
| 0.5346 | 2210 | 2.3317        | -      |
| 0.5370 | 2220 | 2.3452        | -      |
| 0.5385 | 2226 | -             | 0.7313 |
| 0.5394 | 2230 | 2.3219        | -      |
| 0.5418 | 2240 | 2.3616        | -      |
| 0.5443 | 2250 | 2.3093        | -      |
| 0.5467 | 2260 | 2.2907        | -      |
| 0.5486 | 2268 | -             | 0.7111 |
| 0.5491 | 2270 | 2.3125        | -      |
| 0.5515 | 2280 | 2.3176        | -      |
| 0.5539 | 2290 | 2.3241        | -      |
| 0.5564 | 2300 | 2.3287        | -      |
| 0.5588 | 2310 | 2.2922        | 0.6895 |
| 0.5612 | 2320 | 2.3044        | -      |
| 0.5636 | 2330 | 2.3215        | -      |
| 0.5660 | 2340 | 2.3486        | -      |
| 0.5685 | 2350 | 2.3294        | -      |
| 0.5689 | 2352 | -             | 0.7046 |
| 0.5709 | 2360 | 2.3204        | -      |
| 0.5733 | 2370 | 2.273         | -      |
| 0.5757 | 2380 | 2.2787        | -      |
| 0.5781 | 2390 | 2.2772        | -      |
| 0.5791 | 2394 | -             | 0.6885 |
| 0.5806 | 2400 | 2.3089        | -      |
| 0.5830 | 2410 | 2.3235        | -      |
| 0.5854 | 2420 | 2.251         | -      |
| 0.5878 | 2430 | 2.2445        | -      |
| 0.5893 | 2436 | -             | 0.6755 |
| 0.5902 | 2440 | 2.2201        | -      |
| 0.5926 | 2450 | 2.2468        | -      |
| 0.5951 | 2460 | 2.2094        | -      |
| 0.5975 | 2470 | 2.2455        | -      |
| 0.5994 | 2478 | -             | 0.6602 |
| 0.5999 | 2480 | 2.2538        | -      |
| 0.6023 | 2490 | 2.2125        | -      |
| 0.6047 | 2500 | 2.2468        | -      |
| 0.6072 | 2510 | 2.2107        | -      |
| 0.6096 | 2520 | 2.2291        | 0.6725 |
| 0.6120 | 2530 | 2.2051        | -      |
| 0.6144 | 2540 | 2.2408        | -      |
| 0.6168 | 2550 | 2.1987        | -      |
| 0.6193 | 2560 | 2.1743        | -      |
| 0.6197 | 2562 | -             | 0.6907 |
| 0.6217 | 2570 | 2.2191        | -      |
| 0.6241 | 2580 | 2.1297        | -      |
| 0.6265 | 2590 | 2.1588        | -      |
| 0.6289 | 2600 | 2.171         | -      |
| 0.6299 | 2604 | -             | 0.6736 |
| 0.6313 | 2610 | 2.1956        | -      |
| 0.6338 | 2620 | 2.1818        | -      |
| 0.6362 | 2630 | 2.1472        | -      |
| 0.6386 | 2640 | 2.1413        | -      |
| 0.6401 | 2646 | -             | 0.6672 |
| 0.6410 | 2650 | 2.1891        | -      |
| 0.6434 | 2660 | 2.228         | -      |
| 0.6459 | 2670 | 2.1725        | -      |
| 0.6483 | 2680 | 2.1622        | -      |
| 0.6502 | 2688 | -             | 0.6465 |
| 0.6507 | 2690 | 2.1245        | -      |
| 0.6531 | 2700 | 2.1485        | -      |
| 0.6555 | 2710 | 2.1685        | -      |
| 0.6580 | 2720 | 2.1808        | -      |
| 0.6604 | 2730 | 2.1547        | 0.6508 |
| 0.6628 | 2740 | 2.1459        | -      |
| 0.6652 | 2750 | 2.1755        | -      |
| 0.6676 | 2760 | 2.1741        | -      |
| 0.6701 | 2770 | 2.1441        | -      |
| 0.6705 | 2772 | -             | 0.6543 |
| 0.6725 | 2780 | 2.1351        | -      |
| 0.6749 | 2790 | 2.1795        | -      |
| 0.6773 | 2800 | 2.1234        | -      |
| 0.6797 | 2810 | 2.1839        | -      |
| 0.6807 | 2814 | -             | 0.6338 |
| 0.6821 | 2820 | 2.172         | -      |
| 0.6846 | 2830 | 2.1623        | -      |
| 0.6870 | 2840 | 2.1384        | -      |
| 0.6894 | 2850 | 2.1192        | -      |
| 0.6909 | 2856 | -             | 0.6365 |
| 0.6918 | 2860 | 2.1299        | -      |
| 0.6942 | 2870 | 2.0923        | -      |
| 0.6967 | 2880 | 2.1235        | -      |
| 0.6991 | 2890 | 2.1251        | -      |
| 0.7010 | 2898 | -             | 0.6314 |
| 0.7015 | 2900 | 2.0977        | -      |
| 0.7039 | 2910 | 2.1185        | -      |
| 0.7063 | 2920 | 2.1247        | -      |
| 0.7088 | 2930 | 2.1402        | -      |
| 0.7112 | 2940 | 2.147         | 0.6360 |
| 0.7136 | 2950 | 2.0964        | -      |
| 0.7160 | 2960 | 2.0909        | -      |
| 0.7184 | 2970 | 2.1207        | -      |
| 0.7209 | 2980 | 2.168         | -      |
| 0.7213 | 2982 | -             | 0.6326 |
| 0.7233 | 2990 | 2.0948        | -      |
| 0.7257 | 3000 | 2.0667        | -      |
| 0.7281 | 3010 | 2.1356        | -      |
| 0.7305 | 3020 | 2.1777        | -      |
| 0.7315 | 3024 | -             | 0.6356 |
| 0.7329 | 3030 | 2.1407        | -      |
| 0.7354 | 3040 | 2.1987        | -      |
| 0.7378 | 3050 | 2.1879        | -      |
| 0.7402 | 3060 | 2.1648        | -      |
| 0.7417 | 3066 | -             | 0.6207 |
| 0.7426 | 3070 | 2.16          | -      |
| 0.7450 | 3080 | 2.106         | -      |
| 0.7475 | 3090 | 2.1453        | -      |
| 0.7499 | 3100 | 2.1011        | -      |
| 0.7518 | 3108 | -             | 0.6296 |
| 0.7523 | 3110 | 2.104         | -      |
| 0.7547 | 3120 | 2.0936        | -      |
| 0.7571 | 3130 | 2.0504        | -      |
| 0.7596 | 3140 | 2.0462        | -      |
| 0.7620 | 3150 | 2.0193        | 0.6334 |
| 0.7644 | 3160 | 2.0453        | -      |
| 0.7668 | 3170 | 2.0954        | -      |
| 0.7692 | 3180 | 2.1092        | -      |
| 0.7716 | 3190 | 2.1209        | -      |
| 0.7721 | 3192 | -             | 0.6383 |
| 0.7741 | 3200 | 2.1385        | -      |
| 0.7765 | 3210 | 2.0488        | -      |
| 0.7789 | 3220 | 2.0748        | -      |
| 0.7813 | 3230 | 2.0723        | -      |
| 0.7823 | 3234 | -             | 0.6196 |
| 0.7837 | 3240 | 2.0578        | -      |
| 0.7862 | 3250 | 2.0708        | -      |
| 0.7886 | 3260 | 2.086         | -      |
| 0.7910 | 3270 | 2.0504        | -      |
| 0.7925 | 3276 | -             | 0.6171 |
| 0.7934 | 3280 | 2.0338        | -      |
| 0.7958 | 3290 | 2.0754        | -      |
| 0.7983 | 3300 | 2.0353        | -      |
| 0.8007 | 3310 | 2.0709        | -      |
| 0.8026 | 3318 | -             | 0.6091 |
| 0.8031 | 3320 | 2.0502        | -      |
| 0.8055 | 3330 | 2.0541        | -      |
| 0.8079 | 3340 | 2.0509        | -      |
| 0.8104 | 3350 | 1.9828        | -      |
| 0.8128 | 3360 | 2.0352        | 0.6119 |
| 0.8152 | 3370 | 2.0217        | -      |
| 0.8176 | 3380 | 2.0101        | -      |
| 0.8200 | 3390 | 2.0851        | -      |
| 0.8224 | 3400 | 2.0473        | -      |
| 0.8229 | 3402 | -             | 0.6128 |
| 0.8249 | 3410 | 2.0676        | -      |
| 0.8273 | 3420 | 2.0666        | -      |
| 0.8297 | 3430 | 2.0612        | -      |
| 0.8321 | 3440 | 2.0778        | -      |
| 0.8331 | 3444 | -             | 0.6213 |
| 0.8345 | 3450 | 2.0755        | -      |
| 0.8370 | 3460 | 2.0443        | -      |
| 0.8394 | 3470 | 2.039         | -      |
| 0.8418 | 3480 | 1.9625        | -      |
| 0.8433 | 3486 | -             | 0.6093 |
| 0.8442 | 3490 | 2.0408        | -      |
| 0.8466 | 3500 | 2.0795        | -      |
| 0.8491 | 3510 | 2.0077        | -      |
| 0.8515 | 3520 | 2.0181        | -      |
| 0.8534 | 3528 | -             | 0.6073 |
| 0.8539 | 3530 | 2.0182        | -      |
| 0.8563 | 3540 | 2.0174        | -      |
| 0.8587 | 3550 | 2.0551        | -      |
| 0.8612 | 3560 | 2.0427        | -      |
| 0.8636 | 3570 | 1.9981        | 0.6042 |
| 0.8660 | 3580 | 2.0657        | -      |
| 0.8684 | 3590 | 2.0465        | -      |
| 0.8708 | 3600 | 2.0011        | -      |
| 0.8732 | 3610 | 2.0421        | -      |
| 0.8737 | 3612 | -             | 0.6024 |
| 0.8757 | 3620 | 1.9901        | -      |
| 0.8781 | 3630 | 1.6599        | -      |
| 0.8805 | 3640 | 1.5043        | -      |
| 0.8829 | 3650 | 1.593         | -      |
| 0.8839 | 3654 | -             | 0.5947 |
| 0.8853 | 3660 | 1.7022        | -      |
| 0.8878 | 3670 | 1.8429        | -      |
| 0.8902 | 3680 | 1.8807        | -      |
| 0.8926 | 3690 | 1.9801        | -      |
| 0.8940 | 3696 | -             | 0.6562 |
| 0.8950 | 3700 | 1.9469        | -      |
| 0.8974 | 3710 | 1.9986        | -      |
| 0.8999 | 3720 | 1.9962        | -      |
| 0.9023 | 3730 | 1.9762        | -      |
| 0.9042 | 3738 | -             | 0.7028 |
| 0.9047 | 3740 | 1.9579        | -      |
| 0.9071 | 3750 | 1.9973        | -      |
| 0.9095 | 3760 | 1.9966        | -      |
| 0.9119 | 3770 | 1.8911        | -      |
| 0.9144 | 3780 | 1.967         | 0.7325 |
| 0.9168 | 3790 | 1.9749        | -      |
| 0.9192 | 3800 | 2.0358        | -      |
| 0.9216 | 3810 | 1.9572        | -      |
| 0.9240 | 3820 | 1.9778        | -      |
| 0.9245 | 3822 | -             | 0.7554 |
| 0.9265 | 3830 | 1.9152        | -      |
| 0.9289 | 3840 | 1.9637        | -      |
| 0.9313 | 3850 | 1.9514        | -      |
| 0.9337 | 3860 | 1.986         | -      |
| 0.9347 | 3864 | -             | 0.7732 |
| 0.9361 | 3870 | 2.0145        | -      |
| 0.9386 | 3880 | 1.8994        | -      |
| 0.9410 | 3890 | 1.9492        | -      |
| 0.9434 | 3900 | 1.9374        | -      |
| 0.9448 | 3906 | -             | 0.7844 |
| 0.9458 | 3910 | 1.9583        | -      |
| 0.9482 | 3920 | 1.9123        | -      |
| 0.9507 | 3930 | 1.9629        | -      |
| 0.9531 | 3940 | 1.92          | -      |
| 0.9550 | 3948 | -             | 0.7910 |
| 0.9555 | 3950 | 2.0428        | -      |
| 0.9579 | 3960 | 1.9489        | -      |
| 0.9603 | 3970 | 1.9754        | -      |
| 0.9627 | 3980 | 1.9748        | -      |
| 0.9652 | 3990 | 1.8946        | 0.7957 |
| 0.9676 | 4000 | 1.9238        | -      |
| 0.9700 | 4010 | 1.9004        | -      |
| 0.9724 | 4020 | 1.8867        | -      |
| 0.9748 | 4030 | 1.8482        | -      |
| 0.9753 | 4032 | -             | 0.8006 |
| 0.9773 | 4040 | 1.919         | -      |
| 0.9797 | 4050 | 2.0569        | -      |
| 0.9821 | 4060 | 1.986         | -      |
| 0.9845 | 4070 | 1.9729        | -      |
| 0.9855 | 4074 | -             | 0.8030 |
| 0.9869 | 4080 | 1.8762        | -      |
| 0.9894 | 4090 | 1.9108        | -      |
| 0.9918 | 4100 | 1.9484        | -      |
| 0.9942 | 4110 | 1.9913        | -      |
| 0.9956 | 4116 | -             | 0.8033 |
| 0.9966 | 4120 | 1.8677        | -      |
| 0.9990 | 4130 | 1.963         | -      |

</details>

### Framework Versions
- Python: 3.11.8
- Sentence Transformers: 3.1.1
- Transformers: 4.45.1
- PyTorch: 2.4.0+cu121
- Accelerate: 0.34.2
- Datasets: 3.0.0
- Tokenizers: 0.20.0

## Citation

### BibTeX

#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
```

#### MaskedCachedMultipleNegativesRankingLoss
```bibtex
@misc{gao2021scaling,
    title={Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup},
    author={Luyu Gao and Yunyi Zhang and Jiawei Han and Jamie Callan},
    year={2021},
    eprint={2101.06983},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->