sentence-transformers safetensors bert sentence-similarity feature-extraction generated_from_trainer dataset_size:510287 loss:CoSENTLoss en arxiv:1908.10084 base_model:thenlper/gte-small base_model:finetune:thenlper/gte-small license:apache-2.0 model-index autotrain_compatible text-embeddings-inference endpoints_compatible region:us
---
base_model: thenlper/gte-small
language:
- en
library_name: sentence-transformers
license: apache-2.0
metrics:
- pearson_cosine
- spearman_cosine
- pearson_manhattan
- spearman_manhattan
- pearson_euclidean
- spearman_euclidean
- pearson_dot
- spearman_dot
- pearson_max
- spearman_max
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:510287
- loss:CoSENTLoss
widget:
- source_sentence: bag
  sentences:
  - bag
  - summer colors bag
  - carry all bag
- source_sentence: bean bag
  sentences:
  - bag
  - havan bag
  - black yellow shoes
- source_sentence: pyramid shaped cushion mattress
  sentences:
  - dress
  - silver bag
  - women shoes
- source_sentence: handcrafted rug
  sentences:
  - amaga  cross bag - white
  - handcrafted boots
  - polyester top
- source_sentence: bean bag
  sentences:
  - bag
  - v-neck dress
  - bag
model-index:
- name: gte-small-pair_score
  results:
  - task:
      type: semantic-similarity
      name: Semantic Similarity
    dataset:
      name: sts dev
      type: sts-dev
    metrics:
    - type: pearson_cosine
      value: -0.17233834277204704
      name: Pearson Cosine
    - type: spearman_cosine
      value: -0.2198666606268324
      name: Spearman Cosine
    - type: pearson_manhattan
      value: -0.18762372004757433
      name: Pearson Manhattan
    - type: spearman_manhattan
      value: -0.2263824285497944
      name: Spearman Manhattan
    - type: pearson_euclidean
      value: -0.1815229012953811
      name: Pearson Euclidean
    - type: spearman_euclidean
      value: -0.21986651824620543
      name: Spearman Euclidean
    - type: pearson_dot
      value: -0.17233841453151344
      name: Pearson Dot
    - type: spearman_dot
      value: -0.21986648743251272
      name: Spearman Dot
    - type: pearson_max
      value: -0.17233834277204704
      name: Pearson Max
    - type: spearman_max
      value: -0.21986648743251272
      name: Spearman Max
---

# gte-small-pair_score

This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [thenlper/gte-small](https://huggingface.co/thenlper/gte-small). It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [thenlper/gte-small](https://huggingface.co/thenlper/gte-small) <!-- at revision 50c7dd33df1027ef560fd504d95e277948c3c886 -->
- **Maximum Sequence Length:** 512 tokens
- **Output Dimensionality:** 384 tokens
- **Similarity Function:** Cosine Similarity
<!-- - **Training Dataset:** Unknown -->
- **Language:** en
- **License:** apache-2.0

### Model Sources

- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)

### Full Model Architecture

```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)
```

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'bean bag',
    'bag',
    'v-neck dress',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```

<!--
### Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details>
-->

<!--
### Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details>
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

## Evaluation

### Metrics

#### Semantic Similarity
* Dataset: `sts-dev`
* Evaluated with [<code>EmbeddingSimilarityEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.EmbeddingSimilarityEvaluator)

| Metric              | Value       |
|:--------------------|:------------|
| pearson_cosine      | -0.1723     |
| **spearman_cosine** | **-0.2199** |
| pearson_manhattan   | -0.1876     |
| spearman_manhattan  | -0.2264     |
| pearson_euclidean   | -0.1815     |
| spearman_euclidean  | -0.2199     |
| pearson_dot         | -0.1723     |
| spearman_dot        | -0.2199     |
| pearson_max         | -0.1723     |
| spearman_max        | -0.2199     |

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Hyperparameters
#### Non-Default Hyperparameters

- `eval_strategy`: steps
- `per_device_train_batch_size`: 32
- `per_device_eval_batch_size`: 32
- `learning_rate`: 2e-05
- `num_train_epochs`: 4
- `warmup_ratio`: 0.1
- `fp16`: True
- `load_best_model_at_end`: True

#### All Hyperparameters
<details><summary>Click to expand</summary>

- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: steps
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 32
- `per_device_eval_batch_size`: 32
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 2e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 4
- `max_steps`: -1
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.1
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: False
- `fp16`: True
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: True
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: False
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`: 
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `use_liger_kernel`: False
- `eval_use_gather_object`: False
- `batch_sampler`: batch_sampler
- `multi_dataset_batch_sampler`: proportional

</details>

### Training Logs
<details><summary>Click to expand</summary>

| Epoch  | Step  | Training Loss | loss   | sts-dev_spearman_cosine |
|:------:|:-----:|:-------------:|:------:|:-----------------------:|
| 0      | 0     | -             | -      | -0.2199                 |
| 0.0063 | 100   | 6.3669        | 6.6513 | -                       |
| 0.0125 | 200   | 6.1795        | 6.2541 | -                       |
| 0.0188 | 300   | 5.893         | 5.7733 | -                       |
| 0.0251 | 400   | 5.5099        | 5.3626 | -                       |
| 0.0314 | 500   | 5.1589        | 4.9902 | -                       |
| 0.0376 | 600   | 4.8599        | 4.6523 | -                       |
| 0.0439 | 700   | 4.6075        | 4.4233 | -                       |
| 0.0502 | 800   | 4.3831        | 4.2431 | -                       |
| 0.0564 | 900   | 4.1737        | 4.1350 | -                       |
| 0.0627 | 1000  | 4.0266        | 4.0327 | -                       |
| 0.0690 | 1100  | 3.9526        | 3.9281 | -                       |
| 0.0752 | 1200  | 3.8773        | 3.8735 | -                       |
| 0.0815 | 1300  | 3.7856        | 3.7779 | -                       |
| 0.0878 | 1400  | 3.5994        | 3.7054 | -                       |
| 0.0941 | 1500  | 3.7067        | 3.6155 | -                       |
| 0.1003 | 1600  | 3.5471        | 3.5798 | -                       |
| 0.1066 | 1700  | 3.6679        | 3.4654 | -                       |
| 0.1129 | 1800  | 3.4484        | 3.4175 | -                       |
| 0.1191 | 1900  | 3.377         | 3.4129 | -                       |
| 0.1254 | 2000  | 3.4259        | 3.3347 | -                       |
| 0.1317 | 2100  | 3.4832        | 3.2113 | -                       |
| 0.1380 | 2200  | 3.3043        | 3.1641 | -                       |
| 0.1442 | 2300  | 3.2344        | 3.1529 | -                       |
| 0.1505 | 2400  | 3.1238        | 3.2577 | -                       |
| 0.1568 | 2500  | 3.1456        | 3.0678 | -                       |
| 0.1630 | 2600  | 3.0223        | 3.0006 | -                       |
| 0.1693 | 2700  | 3.2046        | 2.9682 | -                       |
| 0.1756 | 2800  | 3.0866        | 3.0524 | -                       |
| 0.1819 | 2900  | 2.9271        | 3.0573 | -                       |
| 0.1881 | 3000  | 2.7692        | 3.0558 | -                       |
| 0.1944 | 3100  | 3.1498        | 2.7866 | -                       |
| 0.2007 | 3200  | 3.0683        | 2.8478 | -                       |
| 0.2069 | 3300  | 2.5776        | 2.9459 | -                       |
| 0.2132 | 3400  | 2.9394        | 2.7133 | -                       |
| 0.2195 | 3500  | 2.6996        | 2.8582 | -                       |
| 0.2257 | 3600  | 2.569         | 2.8092 | -                       |
| 0.2320 | 3700  | 2.6535        | 2.7977 | -                       |
| 0.2383 | 3800  | 2.6679        | 2.8578 | -                       |
| 0.2446 | 3900  | 2.592         | 2.8251 | -                       |
| 0.2508 | 4000  | 2.4931        | 2.5976 | -                       |
| 0.2571 | 4100  | 2.3012        | 2.9260 | -                       |
| 0.2634 | 4200  | 2.4728        | 2.7869 | -                       |
| 0.2696 | 4300  | 2.4391        | 2.8987 | -                       |
| 0.2759 | 4400  | 2.3825        | 2.7804 | -                       |
| 0.2822 | 4500  | 2.6257        | 2.8309 | -                       |
| 0.2885 | 4600  | 2.4304        | 3.2419 | -                       |
| 0.2947 | 4700  | 3.0246        | 2.5732 | -                       |
| 0.3010 | 4800  | 2.6894        | 2.8058 | -                       |
| 0.3073 | 4900  | 2.5333        | 2.4582 | -                       |
| 0.3135 | 5000  | 2.3268        | 2.8622 | -                       |
| 0.3198 | 5100  | 2.6996        | 2.7515 | -                       |
| 0.3261 | 5200  | 2.8175        | 2.5842 | -                       |
| 0.3324 | 5300  | 2.1244        | 2.7252 | -                       |
| 0.3386 | 5400  | 2.7331        | 2.5053 | -                       |
| 0.3449 | 5500  | 2.3226        | 2.2430 | -                       |
| 0.3512 | 5600  | 2.0706        | 2.6055 | -                       |
| 0.3574 | 5700  | 2.2461        | 2.8949 | -                       |
| 0.3637 | 5800  | 2.6365        | 2.5272 | -                       |
| 0.3700 | 5900  | 2.7119        | 2.4331 | -                       |
| 0.3762 | 6000  | 2.6146        | 2.3858 | -                       |
| 0.3825 | 6100  | 2.1998        | 2.6891 | -                       |
| 0.3888 | 6200  | 2.5076        | 2.3827 | -                       |
| 0.3951 | 6300  | 2.5244        | 2.6522 | -                       |
| 0.4013 | 6400  | 2.0613        | 2.4750 | -                       |
| 0.4076 | 6500  | 2.465         | 2.5254 | -                       |
| 0.4139 | 6600  | 2.3201        | 2.2249 | -                       |
| 0.4201 | 6700  | 2.234         | 2.5168 | -                       |
| 0.4264 | 6800  | 2.1277        | 2.5358 | -                       |
| 0.4327 | 6900  | 2.3801        | 2.4992 | -                       |
| 0.4390 | 7000  | 2.1443        | 2.4043 | -                       |
| 0.4452 | 7100  | 1.9136        | 2.3874 | -                       |
| 0.4515 | 7200  | 2.3067        | 2.6475 | -                       |
| 0.4578 | 7300  | 2.1464        | 2.4704 | -                       |
| 0.4640 | 7400  | 2.2151        | 2.5199 | -                       |
| 0.4703 | 7500  | 2.4653        | 2.5293 | -                       |
| 0.4766 | 7600  | 2.4425        | 2.1264 | -                       |
| 0.4828 | 7700  | 2.3138        | 2.1810 | -                       |
| 0.4891 | 7800  | 2.247         | 2.1404 | -                       |
| 0.4954 | 7900  | 2.1621        | 2.2123 | -                       |
| 0.5017 | 8000  | 2.1338        | 2.5108 | -                       |
| 0.5079 | 8100  | 2.1846        | 2.1493 | -                       |
| 0.5142 | 8200  | 2.1167        | 2.2879 | -                       |
| 0.5205 | 8300  | 2.2143        | 2.1664 | -                       |
| 0.5267 | 8400  | 2.3152        | 2.1072 | -                       |
| 0.5330 | 8500  | 1.7618        | 2.0324 | -                       |
| 0.5393 | 8600  | 2.0777        | 2.4468 | -                       |
| 0.5456 | 8700  | 2.1573        | 2.2053 | -                       |
| 0.5518 | 8800  | 1.9831        | 2.3277 | -                       |
| 0.5581 | 8900  | 1.9083        | 1.9949 | -                       |
| 0.5644 | 9000  | 1.932         | 1.9848 | -                       |
| 0.5706 | 9100  | 2.3223        | 1.9192 | -                       |
| 0.5769 | 9200  | 1.7583        | 2.0066 | -                       |
| 0.5832 | 9300  | 1.6394        | 2.0322 | -                       |
| 0.5895 | 9400  | 1.973         | 2.1010 | -                       |
| 0.5957 | 9500  | 2.2377        | 2.1176 | -                       |
| 0.6020 | 9600  | 2.2269        | 2.0027 | -                       |
| 0.6083 | 9700  | 1.971         | 1.9329 | -                       |
| 0.6145 | 9800  | 1.8982        | 1.9797 | -                       |
| 0.6208 | 9900  | 2.2853        | 1.8433 | -                       |
| 0.6271 | 10000 | 1.6657        | 2.0091 | -                       |
| 0.6333 | 10100 | 2.0732        | 1.7602 | -                       |
| 0.6396 | 10200 | 1.6951        | 1.8849 | -                       |
| 0.6459 | 10300 | 1.6548        | 2.0066 | -                       |
| 0.6522 | 10400 | 1.7187        | 1.9644 | -                       |
| 0.6584 | 10500 | 2.1948        | 1.8392 | -                       |
| 0.6647 | 10600 | 1.9756        | 1.8404 | -                       |
| 0.6710 | 10700 | 1.7644        | 1.9101 | -                       |
| 0.6772 | 10800 | 1.6295        | 1.9440 | -                       |
| 0.6835 | 10900 | 1.7687        | 1.9031 | -                       |
| 0.6898 | 11000 | 1.8203        | 1.9650 | -                       |
| 0.6961 | 11100 | 2.3055        | 1.8432 | -                       |
| 0.7023 | 11200 | 1.8294        | 1.7364 | -                       |
| 0.7086 | 11300 | 2.0026        | 1.7894 | -                       |
| 0.7149 | 11400 | 1.9916        | 1.8343 | -                       |
| 0.7211 | 11500 | 1.8698        | 1.8079 | -                       |
| 0.7274 | 11600 | 1.5213        | 1.6849 | -                       |
| 0.7337 | 11700 | 1.7462        | 1.7328 | -                       |
| 0.7400 | 11800 | 1.3519        | 1.8370 | -                       |
| 0.7462 | 11900 | 1.4935        | 1.7247 | -                       |
| 0.7525 | 12000 | 1.1721        | 1.6529 | -                       |
| 0.7588 | 12100 | 2.2432        | 1.6329 | -                       |
| 0.7650 | 12200 | 1.6931        | 1.6563 | -                       |
| 0.7713 | 12300 | 1.2736        | 1.6984 | -                       |
| 0.7776 | 12400 | 1.7063        | 1.6574 | -                       |
| 0.7838 | 12500 | 1.7921        | 1.7760 | -                       |
| 0.7901 | 12600 | 1.875         | 1.7149 | -                       |
| 0.7964 | 12700 | 1.4435        | 1.8085 | -                       |
| 0.8027 | 12800 | 1.5271        | 1.7247 | -                       |
| 0.8089 | 12900 | 1.618         | 1.6542 | -                       |
| 0.8152 | 13000 | 1.9788        | 1.5686 | -                       |
| 0.8215 | 13100 | 1.8213        | 1.5603 | -                       |
| 0.8277 | 13200 | 1.3661        | 1.6376 | -                       |
| 0.8340 | 13300 | 1.3852        | 1.5953 | -                       |
| 0.8403 | 13400 | 1.4673        | 1.6346 | -                       |
| 0.8466 | 13500 | 1.6684        | 1.5818 | -                       |
| 0.8528 | 13600 | 1.686         | 1.5840 | -                       |
| 0.8591 | 13700 | 1.4397        | 1.5855 | -                       |
| 0.8654 | 13800 | 1.5973        | 1.7207 | -                       |
| 0.8716 | 13900 | 1.221         | 1.6381 | -                       |
| 0.8779 | 14000 | 1.2082        | 1.6335 | -                       |
| 0.8842 | 14100 | 1.5399        | 1.6434 | -                       |
| 0.8904 | 14200 | 1.5265        | 1.7266 | -                       |
| 0.8967 | 14300 | 0.9321        | 1.5981 | -                       |
| 0.9030 | 14400 | 1.1133        | 1.6126 | -                       |
| 0.9093 | 14500 | 1.0754        | 1.6227 | -                       |
| 0.9155 | 14600 | 1.3486        | 1.6143 | -                       |
| 0.9218 | 14700 | 1.6338        | 1.5452 | -                       |
| 0.9281 | 14800 | 1.389         | 1.6099 | -                       |
| 0.9343 | 14900 | 1.3776        | 1.6435 | -                       |
| 0.9406 | 15000 | 1.8714        | 1.5377 | -                       |
| 0.9469 | 15100 | 1.1286        | 1.6326 | -                       |
| 0.9532 | 15200 | 1.4029        | 1.6256 | -                       |
| 0.9594 | 15300 | 1.7772        | 1.5221 | -                       |
| 0.9657 | 15400 | 1.3415        | 1.5604 | -                       |
| 0.9720 | 15500 | 1.1088        | 1.5749 | -                       |
| 0.9782 | 15600 | 1.4602        | 1.4941 | -                       |
| 0.9845 | 15700 | 1.867         | 1.3731 | -                       |
| 0.9908 | 15800 | 1.4541        | 1.4206 | -                       |
| 0.9971 | 15900 | 1.1966        | 1.4982 | -                       |
| 1.0033 | 16000 | 1.1447        | 1.5121 | -                       |
| 1.0096 | 16100 | 1.1266        | 1.4103 | -                       |
| 1.0159 | 16200 | 1.1971        | 1.5044 | -                       |
| 1.0221 | 16300 | 1.3376        | 1.5337 | -                       |
| 1.0284 | 16400 | 1.7977        | 1.5712 | -                       |
| 1.0347 | 16500 | 1.6946        | 1.5323 | -                       |
| 1.0409 | 16600 | 0.8674        | 1.4462 | -                       |
| 1.0472 | 16700 | 1.6447        | 1.4831 | -                       |
| 1.0535 | 16800 | 1.2709        | 1.5756 | -                       |
| 1.0598 | 16900 | 1.5217        | 1.5060 | -                       |
| 1.0660 | 17000 | 1.2986        | 1.4834 | -                       |
| 1.0723 | 17100 | 0.9976        | 1.4841 | -                       |
| 1.0786 | 17200 | 1.3457        | 1.4228 | -                       |
| 1.0848 | 17300 | 0.987         | 1.3807 | -                       |
| 1.0911 | 17400 | 1.2714        | 1.3471 | -                       |
| 1.0974 | 17500 | 1.298         | 1.4134 | -                       |
| 1.1037 | 17600 | 0.9522        | 1.4225 | -                       |
| 1.1099 | 17700 | 1.0634        | 1.4474 | -                       |
| 1.1162 | 17800 | 1.2889        | 1.4679 | -                       |
| 1.1225 | 17900 | 1.7532        | 1.3757 | -                       |
| 1.1287 | 18000 | 1.6613        | 1.3808 | -                       |
| 1.1350 | 18100 | 1.1765        | 1.3903 | -                       |
| 1.1413 | 18200 | 1.2787        | 1.3921 | -                       |
| 1.1476 | 18300 | 1.2532        | 1.3519 | -                       |
| 1.1538 | 18400 | 1.8056        | 1.2984 | -                       |
| 1.1601 | 18500 | 1.0985        | 1.3322 | -                       |
| 1.1664 | 18600 | 1.8665        | 1.4060 | -                       |
| 1.1726 | 18700 | 1.2427        | 1.3775 | -                       |
| 1.1789 | 18800 | 1.1241        | 1.3168 | -                       |
| 1.1852 | 18900 | 1.2348        | 1.3539 | -                       |
| 1.1914 | 19000 | 1.1709        | 1.3540 | -                       |
| 1.1977 | 19100 | 0.8844        | 1.3142 | -                       |
| 1.2040 | 19200 | 1.0035        | 1.3781 | -                       |
| 1.2103 | 19300 | 1.4279        | 1.2615 | -                       |
| 1.2165 | 19400 | 1.3327        | 1.2696 | -                       |
| 1.2228 | 19500 | 0.993         | 1.3170 | -                       |
| 1.2291 | 19600 | 0.7869        | 1.2967 | -                       |
| 1.2353 | 19700 | 0.985         | 1.3057 | -                       |
| 1.2416 | 19800 | 1.1603        | 1.2797 | -                       |
| 1.2479 | 19900 | 1.2469        | 1.2394 | -                       |
| 1.2542 | 20000 | 1.521         | 1.2309 | -                       |
| 1.2604 | 20100 | 1.2632        | 1.2353 | -                       |
| 1.2667 | 20200 | 1.3621        | 1.2433 | -                       |
| 1.2730 | 20300 | 1.5145        | 1.3065 | -                       |
| 1.2792 | 20400 | 1.3708        | 1.2423 | -                       |
| 1.2855 | 20500 | 1.1716        | 1.2923 | -                       |
| 1.2918 | 20600 | 1.419         | 1.2194 | -                       |
| 1.2980 | 20700 | 1.1644        | 1.2369 | -                       |
| 1.3043 | 20800 | 1.6589        | 1.1971 | -                       |
| 1.3106 | 20900 | 1.0299        | 1.2343 | -                       |
| 1.3169 | 21000 | 1.3452        | 1.2725 | -                       |
| 1.3231 | 21100 | 1.4234        | 1.2416 | -                       |
| 1.3294 | 21200 | 1.2496        | 1.3609 | -                       |
| 1.3357 | 21300 | 1.2133        | 1.2893 | -                       |
| 1.3419 | 21400 | 0.8682        | 1.2353 | -                       |
| 1.3482 | 21500 | 0.9499        | 1.2423 | -                       |
| 1.3545 | 21600 | 1.2896        | 1.1797 | -                       |
| 1.3608 | 21700 | 1.2392        | 1.1962 | -                       |
| 1.3670 | 21800 | 0.9206        | 1.2483 | -                       |
| 1.3733 | 21900 | 1.174         | 1.2328 | -                       |
| 1.3796 | 22000 | 1.6361        | 1.1558 | -                       |
| 1.3858 | 22100 | 0.8284        | 1.2711 | -                       |
| 1.3921 | 22200 | 1.2814        | 1.2462 | -                       |
| 1.3984 | 22300 | 1.1595        | 1.2613 | -                       |
| 1.4047 | 22400 | 1.3129        | 1.1816 | -                       |
| 1.4109 | 22500 | 1.1353        | 1.2454 | -                       |
| 1.4172 | 22600 | 1.3302        | 1.1398 | -                       |
| 1.4235 | 22700 | 1.1591        | 1.2936 | -                       |
| 1.4297 | 22800 | 0.6551        | 1.2345 | -                       |
| 1.4360 | 22900 | 1.2884        | 1.1629 | -                       |
| 1.4423 | 23000 | 1.1769        | 1.2340 | -                       |
| 1.4485 | 23100 | 1.1331        | 1.2036 | -                       |
| 1.4548 | 23200 | 1.1008        | 1.1685 | -                       |
| 1.4611 | 23300 | 1.1487        | 1.1274 | -                       |
| 1.4674 | 23400 | 0.7753        | 1.1738 | -                       |
| 1.4736 | 23500 | 1.3236        | 1.2377 | -                       |
| 1.4799 | 23600 | 0.919         | 1.2018 | -                       |
| 1.4862 | 23700 | 0.8516        | 1.2297 | -                       |
| 1.4924 | 23800 | 1.092         | 1.1629 | -                       |
| 1.4987 | 23900 | 0.673         | 1.2162 | -                       |
| 1.5050 | 24000 | 0.994         | 1.1778 | -                       |
| 1.5113 | 24100 | 0.8766        | 1.1902 | -                       |
| 1.5175 | 24200 | 1.3818        | 1.1638 | -                       |
| 1.5238 | 24300 | 1.1215        | 1.1666 | -                       |
| 1.5301 | 24400 | 0.8485        | 1.1907 | -                       |
| 1.5363 | 24500 | 1.1033        | 1.2318 | -                       |
| 1.5426 | 24600 | 0.9001        | 1.2113 | -                       |
| 1.5489 | 24700 | 1.3256        | 1.2309 | -                       |
| 1.5552 | 24800 | 0.8162        | 1.2139 | -                       |
| 1.5614 | 24900 | 0.5741        | 1.2375 | -                       |
| 1.5677 | 25000 | 0.883         | 1.2039 | -                       |
| 1.5740 | 25100 | 1.1212        | 1.1428 | -                       |
| 1.5802 | 25200 | 0.8229        | 1.2338 | -                       |
| 1.5865 | 25300 | 0.8856        | 1.1461 | -                       |
| 1.5928 | 25400 | 1.2323        | 1.1569 | -                       |
| 1.5990 | 25500 | 0.9724        | 1.1549 | -                       |
| 1.6053 | 25600 | 1.0791        | 1.1161 | -                       |
| 1.6116 | 25700 | 0.9845        | 1.1061 | -                       |
| 1.6179 | 25800 | 1.1591        | 1.1103 | -                       |
| 1.6241 | 25900 | 1.116         | 1.1405 | -                       |
| 1.6304 | 26000 | 1.2221        | 1.1528 | -                       |
| 1.6367 | 26100 | 0.9085        | 1.1396 | -                       |
| 1.6429 | 26200 | 0.9543        | 1.1953 | -                       |
| 1.6492 | 26300 | 1.1855        | 1.1792 | -                       |
| 1.6555 | 26400 | 1.0583        | 1.1666 | -                       |
| 1.6618 | 26500 | 0.6583        | 1.1152 | -                       |
| 1.6680 | 26600 | 1.3067        | 1.0397 | -                       |
| 1.6743 | 26700 | 1.5336        | 1.1244 | -                       |
| 1.6806 | 26800 | 0.614         | 1.1273 | -                       |
| 1.6868 | 26900 | 1.0336        | 1.0680 | -                       |
| 1.6931 | 27000 | 1.462         | 1.0983 | -                       |
| 1.6994 | 27100 | 0.8858        | 1.0672 | -                       |
| 1.7056 | 27200 | 0.7494        | 1.1624 | -                       |
| 1.7119 | 27300 | 0.8152        | 1.0928 | -                       |
| 1.7182 | 27400 | 0.7785        | 1.0952 | -                       |
| 1.7245 | 27500 | 1.0471        | 1.0999 | -                       |
| 1.7307 | 27600 | 1.0994        | 0.9880 | -                       |
| 1.7370 | 27700 | 1.0706        | 1.0416 | -                       |
| 1.7433 | 27800 | 1.1158        | 1.0676 | -                       |
| 1.7495 | 27900 | 0.9893        | 1.0289 | -                       |
| 1.7558 | 28000 | 1.2939        | 1.0150 | -                       |
| 1.7621 | 28100 | 0.9543        | 1.0767 | -                       |
| 1.7684 | 28200 | 0.7907        | 1.0717 | -                       |
| 1.7746 | 28300 | 0.92          | 1.1133 | -                       |
| 1.7809 | 28400 | 0.8636        | 1.0702 | -                       |
| 1.7872 | 28500 | 0.9118        | 1.0536 | -                       |
| 1.7934 | 28600 | 1.2643        | 1.1354 | -                       |
| 1.7997 | 28700 | 0.8284        | 1.0715 | -                       |
| 1.8060 | 28800 | 0.8447        | 1.0458 | -                       |
| 1.8123 | 28900 | 1.2102        | 1.1001 | -                       |
| 1.8185 | 29000 | 1.1042        | 1.0364 | -                       |
| 1.8248 | 29100 | 0.9638        | 1.0947 | -                       |
| 1.8311 | 29200 | 0.6847        | 1.0312 | -                       |
| 1.8373 | 29300 | 1.7671        | 1.0470 | -                       |
| 1.8436 | 29400 | 0.7525        | 1.1158 | -                       |
| 1.8499 | 29500 | 1.2843        | 1.0140 | -                       |
| 1.8561 | 29600 | 0.6844        | 1.1604 | -                       |
| 1.8624 | 29700 | 1.2824        | 1.0052 | -                       |
| 1.8687 | 29800 | 1.314         | 1.0323 | -                       |
| 1.8750 | 29900 | 1.0796        | 1.0885 | -                       |
| 1.8812 | 30000 | 1.0012        | 1.0267 | -                       |
| 1.8875 | 30100 | 1.4932        | 1.0438 | -                       |
| 1.8938 | 30200 | 1.0404        | 1.0163 | -                       |
| 1.9000 | 30300 | 0.614         | 1.0367 | -                       |
| 1.9063 | 30400 | 1.2676        | 1.0803 | -                       |
| 1.9126 | 30500 | 1.2431        | 1.0428 | -                       |
| 1.9189 | 30600 | 1.4063        | 1.0319 | -                       |
| 1.9251 | 30700 | 0.7787        | 0.9666 | -                       |
| 1.9314 | 30800 | 1.0311        | 1.0376 | -                       |
| 1.9377 | 30900 | 1.0353        | 0.9869 | -                       |
| 1.9439 | 31000 | 1.2221        | 0.9686 | -                       |
| 1.9502 | 31100 | 0.5806        | 0.9663 | -                       |
| 1.9565 | 31200 | 0.6919        | 0.9838 | -                       |
| 1.9628 | 31300 | 0.8028        | 0.9759 | -                       |
| 1.9690 | 31400 | 0.8365        | 0.9641 | -                       |
| 1.9753 | 31500 | 0.7518        | 1.0081 | -                       |
| 1.9816 | 31600 | 1.0654        | 0.9843 | -                       |
| 1.9878 | 31700 | 0.8637        | 0.9635 | -                       |
| 1.9941 | 31800 | 0.8663        | 0.9538 | -                       |
| 2.0004 | 31900 | 0.8524        | 0.9628 | -                       |
| 2.0066 | 32000 | 1.2748        | 0.9382 | -                       |
| 2.0129 | 32100 | 0.8138        | 0.9461 | -                       |
| 2.0192 | 32200 | 0.4484        | 0.9221 | -                       |
| 2.0255 | 32300 | 0.8839        | 0.9567 | -                       |
| 2.0317 | 32400 | 0.7599        | 0.9440 | -                       |
| 2.0380 | 32500 | 0.8665        | 0.9652 | -                       |
| 2.0443 | 32600 | 0.5802        | 0.9475 | -                       |
| 2.0505 | 32700 | 0.7731        | 0.9197 | -                       |
| 2.0568 | 32800 | 0.7913        | 1.0024 | -                       |
| 2.0631 | 32900 | 0.7758        | 0.9257 | -                       |
| 2.0694 | 33000 | 0.7468        | 0.9663 | -                       |
| 2.0756 | 33100 | 0.9947        | 0.9788 | -                       |
| 2.0819 | 33200 | 0.5618        | 0.9480 | -                       |
| 2.0882 | 33300 | 0.8805        | 0.9520 | -                       |
| 2.0944 | 33400 | 0.9755        | 0.9288 | -                       |
| 2.1007 | 33500 | 0.8942        | 0.9234 | -                       |
| 2.1070 | 33600 | 0.7242        | 0.9413 | -                       |
| 2.1133 | 33700 | 0.6231        | 0.9660 | -                       |
| 2.1195 | 33800 | 0.7144        | 0.8901 | -                       |
| 2.1258 | 33900 | 0.7139        | 0.9536 | -                       |
| 2.1321 | 34000 | 0.6378        | 0.9370 | -                       |
| 2.1383 | 34100 | 0.7607        | 0.9209 | -                       |
| 2.1446 | 34200 | 0.8667        | 0.9734 | -                       |
| 2.1509 | 34300 | 0.8533        | 0.9177 | -                       |
| 2.1571 | 34400 | 0.6395        | 0.9285 | -                       |
| 2.1634 | 34500 | 0.7377        | 0.9047 | -                       |
| 2.1697 | 34600 | 0.7787        | 0.9968 | -                       |
| 2.1760 | 34700 | 0.6561        | 0.9653 | -                       |
| 2.1822 | 34800 | 0.6169        | 0.9404 | -                       |
| 2.1885 | 34900 | 0.7643        | 0.9397 | -                       |
| 2.1948 | 35000 | 0.998         | 0.9152 | -                       |
| 2.2010 | 35100 | 0.8246        | 0.9513 | -                       |
| 2.2073 | 35200 | 0.6655        | 0.9354 | -                       |
| 2.2136 | 35300 | 0.9279        | 0.9034 | -                       |
| 2.2199 | 35400 | 0.4239        | 0.9607 | -                       |
| 2.2261 | 35500 | 1.0023        | 0.8732 | -                       |
| 2.2324 | 35600 | 0.7426        | 0.8883 | -                       |
| 2.2387 | 35700 | 0.8675        | 0.9296 | -                       |
| 2.2449 | 35800 | 0.9226        | 0.9146 | -                       |
| 2.2512 | 35900 | 0.4944        | 0.9145 | -                       |
| 2.2575 | 36000 | 0.9663        | 0.8893 | -                       |
| 2.2637 | 36100 | 0.6455        | 0.9238 | -                       |
| 2.2700 | 36200 | 0.9673        | 0.8943 | -                       |
| 2.2763 | 36300 | 0.7974        | 0.8620 | -                       |
| 2.2826 | 36400 | 0.9777        | 0.8812 | -                       |
| 2.2888 | 36500 | 0.8741        | 0.8862 | -                       |
| 2.2951 | 36600 | 0.9642        | 0.9157 | -                       |
| 2.3014 | 36700 | 0.9225        | 0.8784 | -                       |
| 2.3076 | 36800 | 0.6789        | 0.9066 | -                       |
| 2.3139 | 36900 | 0.6726        | 0.9091 | -                       |
| 2.3202 | 37000 | 0.7326        | 0.9203 | -                       |
| 2.3265 | 37100 | 1.007         | 0.9125 | -                       |
| 2.3327 | 37200 | 0.6134        | 0.8837 | -                       |
| 2.3390 | 37300 | 0.9051        | 0.8945 | -                       |
| 2.3453 | 37400 | 0.837         | 0.8740 | -                       |
| 2.3515 | 37500 | 0.7615        | 0.9165 | -                       |
| 2.3578 | 37600 | 0.8304        | 0.9107 | -                       |
| 2.3641 | 37700 | 0.6255        | 0.8891 | -                       |
| 2.3704 | 37800 | 0.6775        | 0.8908 | -                       |
| 2.3766 | 37900 | 0.7159        | 0.8590 | -                       |
| 2.3829 | 38000 | 0.6422        | 0.8559 | -                       |
| 2.3892 | 38100 | 0.7773        | 0.8601 | -                       |
| 2.3954 | 38200 | 0.5457        | 0.8856 | -                       |
| 2.4017 | 38300 | 0.4997        | 0.8785 | -                       |
| 2.4080 | 38400 | 0.6319        | 0.8850 | -                       |
| 2.4142 | 38500 | 0.7096        | 0.8234 | -                       |

</details>

### Framework Versions
- Python: 3.8.10
- Sentence Transformers: 3.1.1
- Transformers: 4.45.1
- PyTorch: 2.4.0+cu121
- Accelerate: 0.34.2
- Datasets: 3.0.1
- Tokenizers: 0.20.0

## Citation

### BibTeX

#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
```

#### CoSENTLoss
```bibtex
@online{kexuefm-8847,
    title={CoSENT: A more efficient sentence vector scheme than Sentence-BERT},
    author={Su Jianlin},
    year={2022},
    month={Jan},
    url={https://kexue.fm/archives/8847},
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->