sentence-transformers safetensors deberta-v2 sentence-similarity feature-extraction generated_from_trainer dataset_size:1079040 loss:CoSENTLoss en dataset:agentlans/sentence-paraphrases arxiv:1908.10084 base_model:agentlans/deberta-v3-base-zyda-2 base_model:finetune:agentlans/deberta-v3-base-zyda-2 license:mit autotrain_compatible endpoints_compatible region:us
---
base_model: agentlans/deberta-v3-base-zyda-2
datasets:
- agentlans/sentence-paraphrases
language:
- en
library_name: sentence-transformers
license: mit
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:1079040
- loss:CoSENTLoss
widget:
- source_sentence: Can you describe the steps involved in reconciling a bank statement?
  sentences:
  - How can we reduce delay in wowza live stream?
  - Did India meet Abdul Kalam's 2020 vision so far?
  - Why US people supporting a old woman president despite a lots of the young men?
- source_sentence: He also met Theodoros Kolokotronis , whom he began to admire later
    .
  sentences:
  - How do you make pancakes without the batter?
  - 'The original team was produced by the writer Sal Buscema and artist Roy Thomas
    in `` The Avengers '''' # 71 ( December 1969 ) .'
  - He also met Theodoros Kolokotronis , whom he later came to admire .
- source_sentence: To enable phone networks to monitor usage, Carrier IQ is a common
    pre-installed feature on U.S. handsets.
  sentences:
  - What does the Pope do?
  - Carrier IQ is installed as standard on many U.S. handsets to allow phone networks
    to monitor usage.
  - Is the risk of disease higher with dead rats than with live ones?
- source_sentence: Many other individuals, communities, and countries perceive "black"
    as a derogatory, outdated, reductionist, or otherwise unrepresentative label,
    and as a result, it is neither used nor defined.
  sentences:
  - Have you been friendzoned?
  - According to Sandoval, Raul Jr. fled from the car but was allegedly shot in the
    back.
  - Do you lose nutrients & calories when you have diarrhea?
- source_sentence: 'While holidaying with her partner and his son, Mrs Searle, an
    administrator, stated: ''If I had been informed, I would have instructed her to
    desist.'''
  sentences:
  - Mechanical digestion involves breaking down large food pieces into smaller ones
    that can be acted upon by digestive enzymes.
  - From prehistoric times, the region was home to tribes such as the Koilis, Halis,
    Dagis, Dhaugris, Dasa, Khasas, Kinnars, and Kirats.
  - 'Mrs Searle, an administrator, who was on holiday with her partner and his son,
    added: ''If I had known I would have told her to stop.'
---

# SentenceTransformer based on agentlans/deberta-v3-base-zyda-2

This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [agentlans/deberta-v3-base-zyda-2](https://huggingface.co/agentlans/deberta-v3-base-zyda-2). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

This model has been finetuned on the [agentlans/sentence-paraphrases](https://huggingface.co/datasets/agentlans/sentence-paraphrases) dataset.
- Text were truncated to 500 characters to save VRAM.
- Two negative examples were generated for each row in the dataset.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [agentlans/deberta-v3-base-zyda-2](https://huggingface.co/agentlans/deberta-v3-base-zyda-2) <!-- at revision 717f1d489c48f91ab631177006839a255255724b -->
- **Maximum Sequence Length:** 512 tokens
- **Output Dimensionality:** 768 tokens
- **Similarity Function:** Cosine Similarity
<!-- - **Training Dataset:** Unknown -->
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->

### Model Sources

- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)

### Full Model Architecture

```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: DebertaV2Model 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)
```

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("agentlans/deberta-v3-base-zyda-2-v2")
# Run inference
sentences = [
    "While holidaying with her partner and his son, Mrs Searle, an administrator, stated: 'If I had been informed, I would have instructed her to desist.'",
    "Mrs Searle, an administrator, who was on holiday with her partner and his son, added: 'If I had known I would have told her to stop.",
    'Mechanical digestion involves breaking down large food pieces into smaller ones that can be acted upon by digestive enzymes.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```

<!--
### Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details>
-->

<!--
### Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details>
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Dataset

#### Unnamed Dataset


* Size: 1,079,040 training samples
* Columns: <code>sentence_0</code>, <code>sentence_1</code>, and <code>label</code>
* Approximate statistics based on the first 1000 samples:
  |         | sentence_0                                                                        | sentence_1                                                                        | label                                                          |
  |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------|
  | type    | string                                                                            | string                                                                            | float                                                          |
  | details | <ul><li>min: 6 tokens</li><li>mean: 22.64 tokens</li><li>max: 75 tokens</li></ul> | <ul><li>min: 6 tokens</li><li>mean: 20.55 tokens</li><li>max: 71 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.33</li><li>max: 1.0</li></ul> |
* Samples:
  | sentence_0                                                                                                            | sentence_1                                                                                | label            |
  |:----------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------|:-----------------|
  | <code>Mrs Alper's late spouse, Sam, had an extensive art collection that was also destroyed.</code>                   | <code>The art collection of Mrs Alper's deceased husband, Sam, was also destroyed.</code> | <code>1.0</code> |
  | <code>Is the 1 x 1 Rubik's cube intended to be comical?</code>                                                        | <code>What is the least fuve digit decimal number with five significant figures?</code>   | <code>0.0</code> |
  | <code>What is the Prasoon Joshi song that you enjoy the most and what is the rationale behind your preference?</code> | <code>What are some interesting facts about human brain?</code>                           | <code>0.0</code> |
* Loss: [<code>CoSENTLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosentloss) with these parameters:
  ```json
  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  ```

### Training Hyperparameters
#### Non-Default Hyperparameters

- `per_device_train_batch_size`: 16
- `per_device_eval_batch_size`: 16
- `multi_dataset_batch_sampler`: round_robin

#### All Hyperparameters
<details><summary>Click to expand</summary>

- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: no
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 16
- `per_device_eval_batch_size`: 16
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 5e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1
- `num_train_epochs`: 3
- `max_steps`: -1
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.0
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: False
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: False
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: False
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`: 
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `eval_use_gather_object`: False
- `batch_sampler`: batch_sampler
- `multi_dataset_batch_sampler`: round_robin

</details>

### Training Logs
<details><summary>Click to expand</summary>

| Epoch  | Step   | Training Loss |
|:------:|:------:|:-------------:|
| 0.0074 | 500    | 1.1791        |
| 0.0148 | 1000   | 0.4056        |
| 0.0222 | 1500   | 0.2729        |
| 0.0297 | 2000   | 0.1775        |
| 0.0371 | 2500   | 0.1513        |
| 0.0445 | 3000   | 0.1329        |
| 0.0519 | 3500   | 0.1503        |
| 0.0593 | 4000   | 0.1325        |
| 0.0667 | 4500   | 0.1187        |
| 0.0741 | 5000   | 0.1012        |
| 0.0816 | 5500   | 0.1504        |
| 0.0890 | 6000   | 0.1161        |
| 0.0964 | 6500   | 0.1194        |
| 0.1038 | 7000   | 0.1172        |
| 0.1112 | 7500   | 0.1391        |
| 0.1186 | 8000   | 0.1056        |
| 0.1260 | 8500   | 0.0697        |
| 0.1335 | 9000   | 0.1157        |
| 0.1409 | 9500   | 0.1009        |
| 0.1483 | 10000  | 0.0996        |
| 0.1557 | 10500  | 0.1076        |
| 0.1631 | 11000  | 0.1057        |
| 0.1705 | 11500  | 0.084         |
| 0.1779 | 12000  | 0.0711        |
| 0.1853 | 12500  | 0.1026        |
| 0.1928 | 13000  | 0.0841        |
| 0.2002 | 13500  | 0.1149        |
| 0.2076 | 14000  | 0.0871        |
| 0.2150 | 14500  | 0.1201        |
| 0.2224 | 15000  | 0.0851        |
| 0.2298 | 15500  | 0.073         |
| 0.2372 | 16000  | 0.0893        |
| 0.2447 | 16500  | 0.1083        |
| 0.2521 | 17000  | 0.0824        |
| 0.2595 | 17500  | 0.0721        |
| 0.2669 | 18000  | 0.056         |
| 0.2743 | 18500  | 0.1062        |
| 0.2817 | 19000  | 0.094         |
| 0.2891 | 19500  | 0.0887        |
| 0.2966 | 20000  | 0.0756        |
| 0.3040 | 20500  | 0.0932        |
| 0.3114 | 21000  | 0.0718        |
| 0.3188 | 21500  | 0.067         |
| 0.3262 | 22000  | 0.0792        |
| 0.3336 | 22500  | 0.0639        |
| 0.3410 | 23000  | 0.0987        |
| 0.3485 | 23500  | 0.0682        |
| 0.3559 | 24000  | 0.0769        |
| 0.3633 | 24500  | 0.1255        |
| 0.3707 | 25000  | 0.0929        |
| 0.3781 | 25500  | 0.0948        |
| 0.3855 | 26000  | 0.0983        |
| 0.3929 | 26500  | 0.1228        |
| 0.4004 | 27000  | 0.1028        |
| 0.4078 | 27500  | 0.0856        |
| 0.4152 | 28000  | 0.1173        |
| 0.4226 | 28500  | 0.0718        |
| 0.4300 | 29000  | 0.0964        |
| 0.4374 | 29500  | 0.0844        |
| 0.4448 | 30000  | 0.0871        |
| 0.4523 | 30500  | 0.0943        |
| 0.4597 | 31000  | 0.1353        |
| 0.4671 | 31500  | 0.0634        |
| 0.4745 | 32000  | 0.1263        |
| 0.4819 | 32500  | 0.1098        |
| 0.4893 | 33000  | 0.098         |
| 0.4967 | 33500  | 0.1182        |
| 0.5042 | 34000  | 0.0818        |
| 0.5116 | 34500  | 0.1207        |
| 0.5190 | 35000  | 0.097         |
| 0.5264 | 35500  | 0.0904        |
| 0.5338 | 36000  | 0.1011        |
| 0.5412 | 36500  | 0.1323        |
| 0.5486 | 37000  | 0.0699        |
| 0.5560 | 37500  | 0.0803        |
| 0.5635 | 38000  | 0.0737        |
| 0.5709 | 38500  | 0.0798        |
| 0.5783 | 39000  | 0.1348        |
| 0.5857 | 39500  | 0.0914        |
| 0.5931 | 40000  | 0.0654        |
| 0.6005 | 40500  | 0.0729        |
| 0.6079 | 41000  | 0.0737        |
| 0.6154 | 41500  | 0.1018        |
| 0.6228 | 42000  | 0.0809        |
| 0.6302 | 42500  | 0.0906        |
| 0.6376 | 43000  | 0.0955        |
| 0.6450 | 43500  | 0.0759        |
| 0.6524 | 44000  | 0.1055        |
| 0.6598 | 44500  | 0.0924        |
| 0.6673 | 45000  | 0.1027        |
| 0.6747 | 45500  | 0.0826        |
| 0.6821 | 46000  | 0.0763        |
| 0.6895 | 46500  | 0.1035        |
| 0.6969 | 47000  | 0.0969        |
| 0.7043 | 47500  | 0.0714        |
| 0.7117 | 48000  | 0.0826        |
| 0.7192 | 48500  | 0.0923        |
| 0.7266 | 49000  | 0.0651        |
| 0.7340 | 49500  | 0.0901        |
| 0.7414 | 50000  | 0.1001        |
| 0.7488 | 50500  | 0.0961        |
| 0.7562 | 51000  | 0.085         |
| 0.7636 | 51500  | 0.0633        |
| 0.7711 | 52000  | 0.0879        |
| 0.7785 | 52500  | 0.0717        |
| 0.7859 | 53000  | 0.0589        |
| 0.7933 | 53500  | 0.0822        |
| 0.8007 | 54000  | 0.0857        |
| 0.8081 | 54500  | 0.0994        |
| 0.8155 | 55000  | 0.0752        |
| 0.8230 | 55500  | 0.0965        |
| 0.8304 | 56000  | 0.0776        |
| 0.8378 | 56500  | 0.089         |
| 0.8452 | 57000  | 0.0638        |
| 0.8526 | 57500  | 0.111         |
| 0.8600 | 58000  | 0.072         |
| 0.8674 | 58500  | 0.0755        |
| 0.8749 | 59000  | 0.096         |
| 0.8823 | 59500  | 0.1205        |
| 0.8897 | 60000  | 0.0728        |
| 0.8971 | 60500  | 0.1014        |
| 0.9045 | 61000  | 0.0987        |
| 0.9119 | 61500  | 0.0756        |
| 0.9193 | 62000  | 0.0746        |
| 0.9267 | 62500  | 0.0992        |
| 0.9342 | 63000  | 0.0961        |
| 0.9416 | 63500  | 0.0861        |
| 0.9490 | 64000  | 0.0723        |
| 0.9564 | 64500  | 0.0765        |
| 0.9638 | 65000  | 0.0859        |
| 0.9712 | 65500  | 0.0839        |
| 0.9786 | 66000  | 0.085         |
| 0.9861 | 66500  | 0.1136        |
| 0.9935 | 67000  | 0.0735        |
| 1.0009 | 67500  | 0.0791        |
| 1.0083 | 68000  | 0.0747        |
| 1.0157 | 68500  | 0.1148        |
| 1.0231 | 69000  | 0.1022        |
| 1.0305 | 69500  | 0.0501        |
| 1.0380 | 70000  | 0.0735        |
| 1.0454 | 70500  | 0.0734        |
| 1.0528 | 71000  | 0.0705        |
| 1.0602 | 71500  | 0.0854        |
| 1.0676 | 72000  | 0.0858        |
| 1.0750 | 72500  | 0.0453        |
| 1.0824 | 73000  | 0.0768        |
| 1.0899 | 73500  | 0.0949        |
| 1.0973 | 74000  | 0.1028        |
| 1.1047 | 74500  | 0.1192        |
| 1.1121 | 75000  | 0.0754        |
| 1.1195 | 75500  | 0.0818        |
| 1.1269 | 76000  | 0.0662        |
| 1.1343 | 76500  | 0.0659        |
| 1.1418 | 77000  | 0.0913        |
| 1.1492 | 77500  | 0.071         |
| 1.1566 | 78000  | 0.0682        |
| 1.1640 | 78500  | 0.0858        |
| 1.1714 | 79000  | 0.0781        |
| 1.1788 | 79500  | 0.0782        |
| 1.1862 | 80000  | 0.0722        |
| 1.1937 | 80500  | 0.0686        |
| 1.2011 | 81000  | 0.0751        |
| 1.2085 | 81500  | 0.0611        |
| 1.2159 | 82000  | 0.1114        |
| 1.2233 | 82500  | 0.0856        |
| 1.2307 | 83000  | 0.0789        |
| 1.2381 | 83500  | 0.0932        |
| 1.2456 | 84000  | 0.0873        |
| 1.2530 | 84500  | 0.0691        |
| 1.2604 | 85000  | 0.0609        |
| 1.2678 | 85500  | 0.0568        |
| 1.2752 | 86000  | 0.0797        |
| 1.2826 | 86500  | 0.0968        |
| 1.2900 | 87000  | 0.1113        |
| 1.2974 | 87500  | 0.0936        |
| 1.3049 | 88000  | 0.091         |
| 1.3123 | 88500  | 0.0482        |
| 1.3197 | 89000  | 0.0898        |
| 1.3271 | 89500  | 0.0766        |
| 1.3345 | 90000  | 0.0859        |
| 1.3419 | 90500  | 0.0851        |
| 1.3493 | 91000  | 0.0695        |
| 1.3568 | 91500  | 0.0881        |
| 1.3642 | 92000  | 0.1095        |
| 1.3716 | 92500  | 0.0676        |
| 1.3790 | 93000  | 0.094         |
| 1.3864 | 93500  | 0.0986        |
| 1.3938 | 94000  | 0.0844        |
| 1.4012 | 94500  | 0.0929        |
| 1.4087 | 95000  | 0.0783        |
| 1.4161 | 95500  | 0.0963        |
| 1.4235 | 96000  | 0.1003        |
| 1.4309 | 96500  | 0.0817        |
| 1.4383 | 97000  | 0.0754        |
| 1.4457 | 97500  | 0.0858        |
| 1.4531 | 98000  | 0.0746        |
| 1.4606 | 98500  | 0.0916        |
| 1.4680 | 99000  | 0.0738        |
| 1.4754 | 99500  | 0.0778        |
| 1.4828 | 100000 | 0.0897        |
| 1.4902 | 100500 | 0.1028        |
| 1.4976 | 101000 | 0.0914        |
| 1.5050 | 101500 | 0.0771        |
| 1.5125 | 102000 | 0.0716        |
| 1.5199 | 102500 | 0.1127        |
| 1.5273 | 103000 | 0.0785        |
| 1.5347 | 103500 | 0.0868        |
| 1.5421 | 104000 | 0.118         |
| 1.5495 | 104500 | 0.0838        |
| 1.5569 | 105000 | 0.0963        |
| 1.5644 | 105500 | 0.0579        |
| 1.5718 | 106000 | 0.0738        |
| 1.5792 | 106500 | 0.1182        |
| 1.5866 | 107000 | 0.1025        |
| 1.5940 | 107500 | 0.0747        |
| 1.6014 | 108000 | 0.0604        |
| 1.6088 | 108500 | 0.0607        |
| 1.6163 | 109000 | 0.0794        |
| 1.6237 | 109500 | 0.0793        |
| 1.6311 | 110000 | 0.084         |
| 1.6385 | 110500 | 0.1315        |
| 1.6459 | 111000 | 0.0782        |
| 1.6533 | 111500 | 0.0724        |
| 1.6607 | 112000 | 0.0864        |
| 1.6681 | 112500 | 0.0791        |
| 1.6756 | 113000 | 0.0772        |
| 1.6830 | 113500 | 0.0923        |
| 1.6904 | 114000 | 0.0897        |
| 1.6978 | 114500 | 0.0833        |
| 1.7052 | 115000 | 0.0819        |
| 1.7126 | 115500 | 0.0695        |
| 1.7200 | 116000 | 0.0919        |
| 1.7275 | 116500 | 0.074         |
| 1.7349 | 117000 | 0.0893        |
| 1.7423 | 117500 | 0.1042        |
| 1.7497 | 118000 | 0.0648        |
| 1.7571 | 118500 | 0.0965        |
| 1.7645 | 119000 | 0.0634        |
| 1.7719 | 119500 | 0.0705        |
| 1.7794 | 120000 | 0.0928        |
| 1.7868 | 120500 | 0.0817        |
| 1.7942 | 121000 | 0.0756        |
| 1.8016 | 121500 | 0.0769        |
| 1.8090 | 122000 | 0.0877        |
| 1.8164 | 122500 | 0.0697        |
| 1.8238 | 123000 | 0.1095        |
| 1.8313 | 123500 | 0.1056        |
| 1.8387 | 124000 | 0.0931        |
| 1.8461 | 124500 | 0.0772        |
| 1.8535 | 125000 | 0.0867        |
| 1.8609 | 125500 | 0.0706        |
| 1.8683 | 126000 | 0.091         |
| 1.8757 | 126500 | 0.0751        |
| 1.8832 | 127000 | 0.0732        |
| 1.8906 | 127500 | 0.0615        |
| 1.8980 | 128000 | 0.0947        |
| 1.9054 | 128500 | 0.1067        |
| 1.9128 | 129000 | 0.0692        |
| 1.9202 | 129500 | 0.064         |
| 1.9276 | 130000 | 0.109         |
| 1.9351 | 130500 | 0.0843        |
| 1.9425 | 131000 | 0.0897        |
| 1.9499 | 131500 | 0.0999        |
| 1.9573 | 132000 | 0.0866        |
| 1.9647 | 132500 | 0.083         |
| 1.9721 | 133000 | 0.0859        |
| 1.9795 | 133500 | 0.0761        |
| 1.9870 | 134000 | 0.1089        |
| 1.9944 | 134500 | 0.1053        |
| 2.0018 | 135000 | 0.0581        |
| 2.0092 | 135500 | 0.0781        |
| 2.0166 | 136000 | 0.1286        |
| 2.0240 | 136500 | 0.1309        |
| 2.0314 | 137000 | 0.0476        |
| 2.0388 | 137500 | 0.0695        |
| 2.0463 | 138000 | 0.0746        |
| 2.0537 | 138500 | 0.063         |
| 2.0611 | 139000 | 0.0816        |
| 2.0685 | 139500 | 0.0821        |
| 2.0759 | 140000 | 0.0671        |
| 2.0833 | 140500 | 0.0865        |
| 2.0907 | 141000 | 0.0638        |
| 2.0982 | 141500 | 0.0803        |
| 2.1056 | 142000 | 0.0872        |
| 2.1130 | 142500 | 0.0968        |
| 2.1204 | 143000 | 0.1052        |
| 2.1278 | 143500 | 0.0554        |
| 2.1352 | 144000 | 0.1057        |
| 2.1426 | 144500 | 0.0565        |
| 2.1501 | 145000 | 0.0798        |
| 2.1575 | 145500 | 0.098         |
| 2.1649 | 146000 | 0.0832        |
| 2.1723 | 146500 | 0.067         |
| 2.1797 | 147000 | 0.0604        |
| 2.1871 | 147500 | 0.0808        |
| 2.1945 | 148000 | 0.0921        |
| 2.2020 | 148500 | 0.0767        |
| 2.2094 | 149000 | 0.0856        |
| 2.2168 | 149500 | 0.0966        |
| 2.2242 | 150000 | 0.0643        |
| 2.2316 | 150500 | 0.068         |
| 2.2390 | 151000 | 0.1007        |
| 2.2464 | 151500 | 0.0765        |
| 2.2539 | 152000 | 0.0662        |
| 2.2613 | 152500 | 0.067         |
| 2.2687 | 153000 | 0.0547        |
| 2.2761 | 153500 | 0.0833        |
| 2.2835 | 154000 | 0.1087        |
| 2.2909 | 154500 | 0.0868        |
| 2.2983 | 155000 | 0.0836        |
| 2.3058 | 155500 | 0.063         |
| 2.3132 | 156000 | 0.0459        |
| 2.3206 | 156500 | 0.0771        |
| 2.3280 | 157000 | 0.0856        |
| 2.3354 | 157500 | 0.0513        |
| 2.3428 | 158000 | 0.0584        |
| 2.3502 | 158500 | 0.0817        |
| 2.3577 | 159000 | 0.0948        |
| 2.3651 | 159500 | 0.0945        |
| 2.3725 | 160000 | 0.0746        |
| 2.3799 | 160500 | 0.0923        |
| 2.3873 | 161000 | 0.0933        |
| 2.3947 | 161500 | 0.08          |
| 2.4021 | 162000 | 0.1343        |
| 2.4095 | 162500 | 0.0699        |
| 2.4170 | 163000 | 0.0861        |
| 2.4244 | 163500 | 0.0811        |
| 2.4318 | 164000 | 0.0671        |
| 2.4392 | 164500 | 0.0877        |
| 2.4466 | 165000 | 0.0741        |
| 2.4540 | 165500 | 0.0834        |
| 2.4614 | 166000 | 0.0966        |
| 2.4689 | 166500 | 0.0739        |
| 2.4763 | 167000 | 0.0916        |
| 2.4837 | 167500 | 0.087         |
| 2.4911 | 168000 | 0.0974        |
| 2.4985 | 168500 | 0.0876        |
| 2.5059 | 169000 | 0.0954        |
| 2.5133 | 169500 | 0.0936        |
| 2.5208 | 170000 | 0.0866        |
| 2.5282 | 170500 | 0.0789        |
| 2.5356 | 171000 | 0.0932        |
| 2.5430 | 171500 | 0.094         |
| 2.5504 | 172000 | 0.0897        |
| 2.5578 | 172500 | 0.08          |
| 2.5652 | 173000 | 0.0664        |
| 2.5727 | 173500 | 0.0807        |
| 2.5801 | 174000 | 0.1157        |
| 2.5875 | 174500 | 0.1272        |
| 2.5949 | 175000 | 0.0843        |
| 2.6023 | 175500 | 0.067         |
| 2.6097 | 176000 | 0.084         |
| 2.6171 | 176500 | 0.0848        |
| 2.6246 | 177000 | 0.0805        |
| 2.6320 | 177500 | 0.0828        |
| 2.6394 | 178000 | 0.1059        |
| 2.6468 | 178500 | 0.0912        |
| 2.6542 | 179000 | 0.0683        |
| 2.6616 | 179500 | 0.0754        |
| 2.6690 | 180000 | 0.0844        |
| 2.6765 | 180500 | 0.0824        |
| 2.6839 | 181000 | 0.0729        |
| 2.6913 | 181500 | 0.0771        |
| 2.6987 | 182000 | 0.0993        |
| 2.7061 | 182500 | 0.0895        |
| 2.7135 | 183000 | 0.0706        |
| 2.7209 | 183500 | 0.0731        |
| 2.7284 | 184000 | 0.0682        |
| 2.7358 | 184500 | 0.0775        |
| 2.7432 | 185000 | 0.0956        |
| 2.7506 | 185500 | 0.0801        |
| 2.7580 | 186000 | 0.106         |
| 2.7654 | 186500 | 0.079         |
| 2.7728 | 187000 | 0.0636        |
| 2.7802 | 187500 | 0.0819        |
| 2.7877 | 188000 | 0.0763        |
| 2.7951 | 188500 | 0.0963        |
| 2.8025 | 189000 | 0.0714        |
| 2.8099 | 189500 | 0.0721        |
| 2.8173 | 190000 | 0.0599        |
| 2.8247 | 190500 | 0.0998        |
| 2.8321 | 191000 | 0.0629        |
| 2.8396 | 191500 | 0.1043        |
| 2.8470 | 192000 | 0.0973        |
| 2.8544 | 192500 | 0.1069        |
| 2.8618 | 193000 | 0.0557        |
| 2.8692 | 193500 | 0.07          |
| 2.8766 | 194000 | 0.1252        |
| 2.8840 | 194500 | 0.0801        |
| 2.8915 | 195000 | 0.0759        |
| 2.8989 | 195500 | 0.105         |
| 2.9063 | 196000 | 0.0806        |
| 2.9137 | 196500 | 0.0737        |
| 2.9211 | 197000 | 0.0841        |
| 2.9285 | 197500 | 0.1031        |
| 2.9359 | 198000 | 0.0828        |
| 2.9434 | 198500 | 0.0842        |
| 2.9508 | 199000 | 0.083         |
| 2.9582 | 199500 | 0.0745        |
| 2.9656 | 200000 | 0.0946        |
| 2.9730 | 200500 | 0.0802        |
| 2.9804 | 201000 | 0.0847        |
| 2.9878 | 201500 | 0.0991        |
| 2.9953 | 202000 | 0.0747        |

</details>

### Framework Versions
- Python: 3.10.12
- Sentence Transformers: 3.1.0
- Transformers: 4.43.3
- PyTorch: 2.3.0+cu121
- Accelerate: 0.33.0
- Datasets: 3.2.0
- Tokenizers: 0.19.1

## Citation

### BibTeX

#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
```

#### CoSENTLoss
```bibtex
@online{kexuefm-8847,
    title={CoSENT: A more efficient sentence vector scheme than Sentence-BERT},
    author={Su Jianlin},
    year={2022},
    month={Jan},
    url={https://kexue.fm/archives/8847},
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->