sentence-transformers safetensors bert sentence-similarity feature-extraction generated_from_trainer dataset_size:100000 loss:CosineSimilarityLoss arxiv:1908.10084 base_model:sentence-transformers/all-MiniLM-L12-v2 base_model:finetune:sentence-transformers/all-MiniLM-L12-v2 model-index autotrain_compatible text-embeddings-inference endpoints_compatible region:us
---
base_model: sentence-transformers/all-MiniLM-L12-v2
library_name: sentence-transformers
metrics:
- pearson_cosine
- spearman_cosine
- pearson_manhattan
- spearman_manhattan
- pearson_euclidean
- spearman_euclidean
- pearson_dot
- spearman_dot
- pearson_max
- spearman_max
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:100000
- loss:CosineSimilarityLoss
widget:
- source_sentence: The church has granite statues of Jesus and the Apostles adorning
    its porch .
  sentences:
  - There were no statues in the church .
  - L' Afrique du sud et le reste de l' Afrique sont les mêmes .
  - Tours on foot are a great way to see LA .
- source_sentence: Au Centre du réseau routier de la région , Alicante est également
    une base logique pour les automobilistes et pour les liaisons ferroviaires et
    ferroviaires .
  sentences:
  - Alicante est fréquentée par les automobilistes et les touristes .
  - Les examinateurs ont passé sept mois à étudier leurs conclusions .
  - Ferries to the island depart from the central station every 2 hours .
- source_sentence: Scheduled to reopen in 2002 or 2003 , the Malibu site will house
    only the Getty holdings in Greek and Roman antiquities , some of which date as
    far back as 3000 b.c.
  sentences:
  - C' est impossible d' avoir des billets pour les enregistrements télévisés .
  - The Getty holdings were taken hold of thanks to the researchers ' effort .
  - After the first of may ends the peak season for ferries .
- source_sentence: Une nouvelle recherche relie ces bactéries parodontale aux maladies
    cardiaques , au diabète , aux bébés à faible poids de naissance , et à d' autres
    saletés que vous attendez des bactéries qui se déchaînent dans le sang .
  sentences:
  - Le prix des actions de Caterpillar a baissé en 1991 quand ils ont fait grève .
  - Ils agissent comme chaque année est la même .
  - La recherche indique qu' il n' y a pas de lien entre les bactéries parodontale
    et les maladies cardiaques ou le diabète .
- source_sentence: L' ancien n' est pas une classification juridique qui entraîne
    une perte automatique de ces droits .
  sentences:
  - Some degree of uncertainty is inherent in free-market systems .
  - Les villes grecques d' Anatolie ont été exclues de l' appartenance à la Confédération
    Delian .
  - Ils voulaient plaider pour les personnes âgées .
model-index:
- name: SentenceTransformer based on sentence-transformers/all-MiniLM-L12-v2
  results:
  - task:
      type: semantic-similarity
      name: Semantic Similarity
    dataset:
      name: snli dev
      type: snli-dev
    metrics:
    - type: pearson_cosine
      value: 0.35421287329686374
      name: Pearson Cosine
    - type: spearman_cosine
      value: 0.3592670991851331
      name: Spearman Cosine
    - type: pearson_manhattan
      value: 0.34936411192844985
      name: Pearson Manhattan
    - type: spearman_manhattan
      value: 0.3583327923327215
      name: Spearman Manhattan
    - type: pearson_euclidean
      value: 0.34982920048695176
      name: Pearson Euclidean
    - type: spearman_euclidean
      value: 0.35926709915022625
      name: Spearman Euclidean
    - type: pearson_dot
      value: 0.3542128787197555
      name: Pearson Dot
    - type: spearman_dot
      value: 0.35926727022169175
      name: Spearman Dot
    - type: pearson_max
      value: 0.3542128787197555
      name: Pearson Max
    - type: spearman_max
      value: 0.35926727022169175
      name: Spearman Max
---

# SentenceTransformer based on sentence-transformers/all-MiniLM-L12-v2

This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/all-MiniLM-L12-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L12-v2). It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [sentence-transformers/all-MiniLM-L12-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L12-v2) <!-- at revision 30ce63ae64e71b9199b3d2eae9de99f64a26eedc -->
- **Maximum Sequence Length:** 128 tokens
- **Output Dimensionality:** 384 tokens
- **Similarity Function:** Cosine Similarity
<!-- - **Training Dataset:** Unknown -->
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->

### Model Sources

- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)

### Full Model Architecture

```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)
```

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("Nessrine9/finetuned-snli-MiniLM-L12-v2-100k-en-fr")
# Run inference
sentences = [
    "L' ancien n' est pas une classification juridique qui entraîne une perte automatique de ces droits .",
    'Ils voulaient plaider pour les personnes âgées .',
    "Les villes grecques d' Anatolie ont été exclues de l' appartenance à la Confédération Delian .",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```

<!--
### Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details>
-->

<!--
### Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details>
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

## Evaluation

### Metrics

#### Semantic Similarity
* Dataset: `snli-dev`
* Evaluated with [<code>EmbeddingSimilarityEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.EmbeddingSimilarityEvaluator)

| Metric             | Value      |
|:-------------------|:-----------|
| pearson_cosine     | 0.3542     |
| spearman_cosine    | 0.3593     |
| pearson_manhattan  | 0.3494     |
| spearman_manhattan | 0.3583     |
| pearson_euclidean  | 0.3498     |
| spearman_euclidean | 0.3593     |
| pearson_dot        | 0.3542     |
| spearman_dot       | 0.3593     |
| pearson_max        | 0.3542     |
| **spearman_max**   | **0.3593** |

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Dataset

#### Unnamed Dataset


* Size: 100,000 training samples
* Columns: <code>sentence_0</code>, <code>sentence_1</code>, and <code>label</code>
* Approximate statistics based on the first 1000 samples:
  |         | sentence_0                                                                         | sentence_1                                                                        | label                                                         |
  |:--------|:-----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:--------------------------------------------------------------|
  | type    | string                                                                             | string                                                                            | float                                                         |
  | details | <ul><li>min: 5 tokens</li><li>mean: 34.31 tokens</li><li>max: 128 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 18.24 tokens</li><li>max: 51 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.5</li><li>max: 1.0</li></ul> |
* Samples:
  | sentence_0                                                                                                                                                                                                                                                                         | sentence_1                                                                                                  | label            |
  |:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------|:-----------------|
  | <code>We 're off ! "</code>                                                                                                                                                                                                                                                        | <code>We 're not headed off .</code>                                                                        | <code>1.0</code> |
  | <code>Il y en a eu un ici récemment qui me vient à l' esprit que c' est à propos d' une femme que c' est ridicule je veux dire que c' est presque euh ce serait drôle si ce n' était pas si triste je veux dire cette femme cette femme est sortie et a engagé quelqu' un à</code> | <code>Cette femme a engagé quelqu' un récemment pour le faire et s' est fait prendre immédiatement .</code> | <code>0.5</code> |
  | <code>Gentilello a précisé qu' il n' avait pas critiqué le processus d' examen par les pairs , mais que les panels qui examinent les interventions en matière d' alcool dans l' eds devraient inclure des représentants de la médecine d' urgence .</code>                         | <code>Gentilello S' est ensuite battu avec un psychiatre sur le parking .</code>                            | <code>0.5</code> |
* Loss: [<code>CosineSimilarityLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosinesimilarityloss) with these parameters:
  ```json
  {
      "loss_fct": "torch.nn.modules.loss.MSELoss"
  }
  ```

### Training Hyperparameters
#### Non-Default Hyperparameters

- `eval_strategy`: steps
- `per_device_train_batch_size`: 16
- `per_device_eval_batch_size`: 16
- `num_train_epochs`: 4
- `fp16`: True
- `multi_dataset_batch_sampler`: round_robin

#### All Hyperparameters
<details><summary>Click to expand</summary>

- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: steps
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 16
- `per_device_eval_batch_size`: 16
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 5e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1
- `num_train_epochs`: 4
- `max_steps`: -1
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.0
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: False
- `fp16`: True
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: False
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: False
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`: 
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `eval_use_gather_object`: False
- `batch_sampler`: batch_sampler
- `multi_dataset_batch_sampler`: round_robin

</details>

### Training Logs
| Epoch  | Step  | Training Loss | snli-dev_spearman_max |
|:------:|:-----:|:-------------:|:---------------------:|
| 0.08   | 500   | 0.1948        | 0.0484                |
| 0.16   | 1000  | 0.1752        | 0.1177                |
| 0.24   | 1500  | 0.1727        | 0.1136                |
| 0.32   | 2000  | 0.1668        | 0.2050                |
| 0.4    | 2500  | 0.1673        | 0.2227                |
| 0.48   | 3000  | 0.1651        | 0.1760                |
| 0.56   | 3500  | 0.1619        | 0.2195                |
| 0.64   | 4000  | 0.1625        | 0.2308                |
| 0.72   | 4500  | 0.1563        | 0.2405                |
| 0.8    | 5000  | 0.1598        | 0.2773                |
| 0.88   | 5500  | 0.1589        | 0.2359                |
| 0.96   | 6000  | 0.1587        | 0.2084                |
| 1.0    | 6250  | -             | 0.2615                |
| 1.04   | 6500  | 0.158         | 0.2958                |
| 1.12   | 7000  | 0.1557        | 0.2887                |
| 1.2    | 7500  | 0.1544        | 0.2960                |
| 1.28   | 8000  | 0.1535        | 0.2977                |
| 1.3600 | 8500  | 0.1559        | 0.2546                |
| 1.44   | 9000  | 0.1518        | 0.3201                |
| 1.52   | 9500  | 0.1551        | 0.2894                |
| 1.6    | 10000 | 0.149         | 0.2981                |
| 1.6800 | 10500 | 0.152         | 0.3140                |
| 1.76   | 11000 | 0.1484        | 0.3056                |
| 1.8400 | 11500 | 0.1497        | 0.3051                |
| 1.92   | 12000 | 0.1522        | 0.2893                |
| 2.0    | 12500 | 0.1503        | 0.2944                |
| 2.08   | 13000 | 0.1496        | 0.3039                |
| 2.16   | 13500 | 0.1462        | 0.3314                |
| 2.24   | 14000 | 0.1505        | 0.2470                |
| 2.32   | 14500 | 0.1457        | 0.3081                |
| 2.4    | 15000 | 0.1478        | 0.3204                |
| 2.48   | 15500 | 0.1464        | 0.3248                |
| 2.56   | 16000 | 0.1442        | 0.3360                |
| 2.64   | 16500 | 0.1437        | 0.3418                |
| 2.7200 | 17000 | 0.1416        | 0.3496                |
| 2.8    | 17500 | 0.1434        | 0.3283                |
| 2.88   | 18000 | 0.146         | 0.3246                |
| 2.96   | 18500 | 0.1448        | 0.3352                |
| 3.0    | 18750 | -             | 0.3248                |
| 3.04   | 19000 | 0.1445        | 0.3394                |
| 3.12   | 19500 | 0.1423        | 0.3430                |
| 3.2    | 20000 | 0.1415        | 0.3410                |
| 3.2800 | 20500 | 0.1411        | 0.3367                |
| 3.36   | 21000 | 0.1445        | 0.3497                |
| 3.44   | 21500 | 0.1383        | 0.3640                |
| 3.52   | 22000 | 0.1408        | 0.3497                |
| 3.6    | 22500 | 0.1374        | 0.3452                |
| 3.68   | 23000 | 0.1401        | 0.3519                |
| 3.76   | 23500 | 0.137         | 0.3582                |
| 3.84   | 24000 | 0.1393        | 0.3610                |
| 3.92   | 24500 | 0.1408        | 0.3575                |
| 4.0    | 25000 | 0.1388        | 0.3593                |


### Framework Versions
- Python: 3.10.12
- Sentence Transformers: 3.2.1
- Transformers: 4.44.2
- PyTorch: 2.5.0+cu121
- Accelerate: 0.34.2
- Datasets: 3.0.2
- Tokenizers: 0.19.1

## Citation

### BibTeX

#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->