sentence-transformers safetensors llama feature-extraction sentence-similarity mteb model-index autotrain_compatible endpoints_compatible region:us
---
library_name: sentence-transformers
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- feature-extraction
- sentence-similarity
- mteb
model-index:
- name: sentence_croissant_alpha_v0.2
  results:
  - task:
      type: Clustering
    dataset:
      name: MTEB AlloProfClusteringP2P
      type: lyon-nlp/alloprof
      config: default
      split: test
      revision: 392ba3f5bcc8c51f578786c1fc3dae648662cb9b
    metrics:
    - type: v_measure
      value: 59.14629497199997
    - type: v_measure
      value: 36.450870830351036
  - task:
      type: Reranking
    dataset:
      name: MTEB AlloprofReranking
      type: lyon-nlp/mteb-fr-reranking-alloprof-s2p
      config: default
      split: test
      revision: e40c8a63ce02da43200eccb5b0846fcaa888f562
    metrics:
    - type: map
      value: 67.23549444979429
    - type: mrr
      value: 68.49382830276612
  - task:
      type: Retrieval
    dataset:
      name: MTEB AlloprofRetrieval
      type: lyon-nlp/alloprof
      config: default
      split: test
      revision: 2df7bee4080bedf2e97de3da6bd5c7bc9fc9c4d2
    metrics:
    - type: map_at_1
      value: 30.285
    - type: map_at_10
      value: 41.724
    - type: map_at_100
      value: 42.696
    - type: map_at_1000
      value: 42.739
    - type: map_at_3
      value: 38.68
    - type: map_at_5
      value: 40.474
    - type: mrr_at_1
      value: 30.285
    - type: mrr_at_10
      value: 41.724
    - type: mrr_at_100
      value: 42.696
    - type: mrr_at_1000
      value: 42.739
    - type: mrr_at_3
      value: 38.68
    - type: mrr_at_5
      value: 40.474
    - type: ndcg_at_1
      value: 30.285
    - type: ndcg_at_10
      value: 47.687000000000005
    - type: ndcg_at_100
      value: 52.580000000000005
    - type: ndcg_at_1000
      value: 53.738
    - type: ndcg_at_3
      value: 41.439
    - type: ndcg_at_5
      value: 44.67
    - type: precision_at_1
      value: 30.285
    - type: precision_at_10
      value: 6.657
    - type: precision_at_100
      value: 0.898
    - type: precision_at_1000
      value: 0.099
    - type: precision_at_3
      value: 16.477
    - type: precision_at_5
      value: 11.454
    - type: recall_at_1
      value: 30.285
    - type: recall_at_10
      value: 66.572
    - type: recall_at_100
      value: 89.819
    - type: recall_at_1000
      value: 98.955
    - type: recall_at_3
      value: 49.43
    - type: recall_at_5
      value: 57.27100000000001
  - task:
      type: Classification
    dataset:
      name: MTEB AmazonReviewsClassification (fr)
      type: mteb/amazon_reviews_multi
      config: fr
      split: test
      revision: 1399c76144fd37290681b995c656ef9b2e06e26d
    metrics:
    - type: accuracy
      value: 36.484
    - type: f1
      value: 36.358267416839176
  - task:
      type: Retrieval
    dataset:
      name: MTEB BSARDRetrieval
      type: maastrichtlawtech/bsard
      config: default
      split: test
      revision: 5effa1b9b5fa3b0f9e12523e6e43e5f86a6e6d59
    metrics:
    - type: map_at_1
      value: 0.44999999999999996
    - type: map_at_10
      value: 1.184
    - type: map_at_100
      value: 1.5939999999999999
    - type: map_at_1000
      value: 1.6680000000000001
    - type: map_at_3
      value: 0.901
    - type: map_at_5
      value: 1.014
    - type: mrr_at_1
      value: 0.44999999999999996
    - type: mrr_at_10
      value: 1.184
    - type: mrr_at_100
      value: 1.5939999999999999
    - type: mrr_at_1000
      value: 1.6680000000000001
    - type: mrr_at_3
      value: 0.901
    - type: mrr_at_5
      value: 1.014
    - type: ndcg_at_1
      value: 0.44999999999999996
    - type: ndcg_at_10
      value: 1.746
    - type: ndcg_at_100
      value: 4.271
    - type: ndcg_at_1000
      value: 6.662
    - type: ndcg_at_3
      value: 1.126
    - type: ndcg_at_5
      value: 1.32
    - type: precision_at_1
      value: 0.44999999999999996
    - type: precision_at_10
      value: 0.36
    - type: precision_at_100
      value: 0.167
    - type: precision_at_1000
      value: 0.036000000000000004
    - type: precision_at_3
      value: 0.601
    - type: precision_at_5
      value: 0.44999999999999996
    - type: recall_at_1
      value: 0.44999999999999996
    - type: recall_at_10
      value: 3.604
    - type: recall_at_100
      value: 16.667
    - type: recall_at_1000
      value: 36.486000000000004
    - type: recall_at_3
      value: 1.802
    - type: recall_at_5
      value: 2.252
  - task:
      type: Clustering
    dataset:
      name: MTEB HALClusteringS2S
      type: lyon-nlp/clustering-hal-s2s
      config: default
      split: test
      revision: e06ebbbb123f8144bef1a5d18796f3dec9ae2915
    metrics:
    - type: v_measure
      value: 24.970553942854256
  - task:
      type: Clustering
    dataset:
      name: MTEB MLSUMClusteringP2P
      type: mlsum
      config: default
      split: test
      revision: b5d54f8f3b61ae17845046286940f03c6bc79bc7
    metrics:
    - type: v_measure
      value: 42.48794423025542
    - type: v_measure
      value: 34.44830504100088
  - task:
      type: Classification
    dataset:
      name: MTEB MTOPDomainClassification (fr)
      type: mteb/mtop_domain
      config: fr
      split: test
      revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
    metrics:
    - type: accuracy
      value: 88.96335734419041
    - type: f1
      value: 88.77543132157024
  - task:
      type: Classification
    dataset:
      name: MTEB MTOPIntentClassification (fr)
      type: mteb/mtop_intent
      config: fr
      split: test
      revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
    metrics:
    - type: accuracy
      value: 67.76072658941435
    - type: f1
      value: 47.98533031010631
  - task:
      type: Classification
    dataset:
      name: MTEB MasakhaNEWSClassification (fra)
      type: masakhane/masakhanews
      config: fra
      split: test
      revision: 8ccc72e69e65f40c70e117d8b3c08306bb788b60
    metrics:
    - type: accuracy
      value: 73.17535545023696
    - type: f1
      value: 69.07397342867827
  - task:
      type: Clustering
    dataset:
      name: MTEB MasakhaNEWSClusteringP2P (fra)
      type: masakhane/masakhanews
      config: fra
      split: test
      revision: 8ccc72e69e65f40c70e117d8b3c08306bb788b60
    metrics:
    - type: v_measure
      value: 47.584542055968335
    - type: v_measure
      value: 33.58141573894578
  - task:
      type: Classification
    dataset:
      name: MTEB MassiveIntentClassification (fr)
      type: mteb/amazon_massive_intent
      config: fr
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 66.29791526563551
    - type: f1
      value: 64.11383858035595
  - task:
      type: Classification
    dataset:
      name: MTEB MassiveScenarioClassification (fr)
      type: mteb/amazon_massive_scenario
      config: fr
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 71.7014122394082
    - type: f1
      value: 71.28396788755553
  - task:
      type: Retrieval
    dataset:
      name: MTEB MintakaRetrieval (fr)
      type: jinaai/mintakaqa
      config: fr
      split: test
      revision: efa78cc2f74bbcd21eff2261f9e13aebe40b814e
    metrics:
    - type: map_at_1
      value: 14.824000000000002
    - type: map_at_10
      value: 23.217
    - type: map_at_100
      value: 24.484
    - type: map_at_1000
      value: 24.571
    - type: map_at_3
      value: 20.762
    - type: map_at_5
      value: 22.121
    - type: mrr_at_1
      value: 14.824000000000002
    - type: mrr_at_10
      value: 23.217
    - type: mrr_at_100
      value: 24.484
    - type: mrr_at_1000
      value: 24.571
    - type: mrr_at_3
      value: 20.762
    - type: mrr_at_5
      value: 22.121
    - type: ndcg_at_1
      value: 14.824000000000002
    - type: ndcg_at_10
      value: 27.876
    - type: ndcg_at_100
      value: 34.53
    - type: ndcg_at_1000
      value: 37.153999999999996
    - type: ndcg_at_3
      value: 22.746
    - type: ndcg_at_5
      value: 25.192999999999998
    - type: precision_at_1
      value: 14.824000000000002
    - type: precision_at_10
      value: 4.279
    - type: precision_at_100
      value: 0.75
    - type: precision_at_1000
      value: 0.096
    - type: precision_at_3
      value: 9.5
    - type: precision_at_5
      value: 6.888
    - type: recall_at_1
      value: 14.824000000000002
    - type: recall_at_10
      value: 42.793
    - type: recall_at_100
      value: 75.02
    - type: recall_at_1000
      value: 96.274
    - type: recall_at_3
      value: 28.500999999999998
    - type: recall_at_5
      value: 34.439
  - task:
      type: PairClassification
    dataset:
      name: MTEB OpusparcusPC (fr)
      type: GEM/opusparcus
      config: fr
      split: test
      revision: 9e9b1f8ef51616073f47f306f7f47dd91663f86a
    metrics:
    - type: cos_sim_accuracy
      value: 82.56130790190736
    - type: cos_sim_ap
      value: 93.47537508242819
    - type: cos_sim_f1
      value: 87.60250844187169
    - type: cos_sim_precision
      value: 85.17823639774859
    - type: cos_sim_recall
      value: 90.16881827209534
    - type: dot_accuracy
      value: 81.06267029972753
    - type: dot_ap
      value: 91.67254760894009
    - type: dot_f1
      value: 87.07172224760164
    - type: dot_precision
      value: 80.62605752961083
    - type: dot_recall
      value: 94.63753723932473
    - type: euclidean_accuracy
      value: 81.19891008174388
    - type: euclidean_ap
      value: 93.11746326702661
    - type: euclidean_f1
      value: 86.52278177458035
    - type: euclidean_precision
      value: 83.6734693877551
    - type: euclidean_recall
      value: 89.57298907646475
    - type: manhattan_accuracy
      value: 81.06267029972753
    - type: manhattan_ap
      value: 93.10511956552851
    - type: manhattan_f1
      value: 86.62175168431185
    - type: manhattan_precision
      value: 84.03361344537815
    - type: manhattan_recall
      value: 89.37437934458788
    - type: max_accuracy
      value: 82.56130790190736
    - type: max_ap
      value: 93.47537508242819
    - type: max_f1
      value: 87.60250844187169
  - task:
      type: PairClassification
    dataset:
      name: MTEB PawsX (fr)
      type: paws-x
      config: fr
      split: test
      revision: 8a04d940a42cd40658986fdd8e3da561533a3646
    metrics:
    - type: cos_sim_accuracy
      value: 64.7
    - type: cos_sim_ap
      value: 66.97936856243149
    - type: cos_sim_f1
      value: 64.10698878343399
    - type: cos_sim_precision
      value: 52.50883392226149
    - type: cos_sim_recall
      value: 82.281284606866
    - type: dot_accuracy
      value: 55.7
    - type: dot_ap
      value: 49.248259184437195
    - type: dot_f1
      value: 62.51298026998961
    - type: dot_precision
      value: 45.468277945619334
    - type: dot_recall
      value: 100.0
    - type: euclidean_accuracy
      value: 65.14999999999999
    - type: euclidean_ap
      value: 67.67376405881289
    - type: euclidean_f1
      value: 64.10034602076125
    - type: euclidean_precision
      value: 52.59048970901349
    - type: euclidean_recall
      value: 82.05980066445183
    - type: manhattan_accuracy
      value: 65.2
    - type: manhattan_ap
      value: 67.68415171194316
    - type: manhattan_f1
      value: 64.16899163013153
    - type: manhattan_precision
      value: 50.12453300124533
    - type: manhattan_recall
      value: 89.14728682170544
    - type: max_accuracy
      value: 65.2
    - type: max_ap
      value: 67.68415171194316
    - type: max_f1
      value: 64.16899163013153
  - task:
      type: STS
    dataset:
      name: MTEB SICKFr
      type: Lajavaness/SICK-fr
      config: default
      split: test
      revision: e077ab4cf4774a1e36d86d593b150422fafd8e8a
    metrics:
    - type: cos_sim_pearson
      value: 77.68761269197373
    - type: cos_sim_spearman
      value: 69.66744624141576
    - type: euclidean_pearson
      value: 72.05200050489465
    - type: euclidean_spearman
      value: 68.04895470259305
    - type: manhattan_pearson
      value: 72.16693522711834
    - type: manhattan_spearman
      value: 68.12086601967899
  - task:
      type: STS
    dataset:
      name: MTEB STS22 (fr)
      type: mteb/sts22-crosslingual-sts
      config: fr
      split: test
      revision: eea2b4fe26a775864c896887d910b76a8098ad3f
    metrics:
    - type: cos_sim_pearson
      value: 75.11874053715779
    - type: cos_sim_spearman
      value: 78.68085137779333
    - type: euclidean_pearson
      value: 68.83921367763453
    - type: euclidean_spearman
      value: 71.35148956255736
    - type: manhattan_pearson
      value: 69.46950072200525
    - type: manhattan_spearman
      value: 71.66493261411941
  - task:
      type: STS
    dataset:
      name: MTEB STSBenchmarkMultilingualSTS (fr)
      type: PhilipMay/stsb_multi_mt
      config: fr
      split: test
      revision: 93d57ef91790589e3ce9c365164337a8a78b7632
    metrics:
    - type: cos_sim_pearson
      value: 78.09242108846412
    - type: cos_sim_spearman
      value: 76.38442769094321
    - type: euclidean_pearson
      value: 76.19649405196662
    - type: euclidean_spearman
      value: 75.95441973818816
    - type: manhattan_pearson
      value: 76.13548797312832
    - type: manhattan_spearman
      value: 75.93264073187262
  - task:
      type: Summarization
    dataset:
      name: MTEB SummEvalFr
      type: lyon-nlp/summarization-summeval-fr-p2p
      config: default
      split: test
      revision: b385812de6a9577b6f4d0f88c6a6e35395a94054
    metrics:
    - type: cos_sim_pearson
      value: 30.511451950181858
    - type: cos_sim_spearman
      value: 30.267871792007288
    - type: dot_pearson
      value: 27.428950856263114
    - type: dot_spearman
      value: 26.895658072972395
  - task:
      type: Reranking
    dataset:
      name: MTEB SyntecReranking
      type: lyon-nlp/mteb-fr-reranking-syntec-s2p
      config: default
      split: test
      revision: b205c5084a0934ce8af14338bf03feb19499c84d
    metrics:
    - type: map
      value: 83.16666666666667
    - type: mrr
      value: 83.16666666666667
  - task:
      type: Retrieval
    dataset:
      name: MTEB SyntecRetrieval
      type: lyon-nlp/mteb-fr-retrieval-syntec-s2p
      config: default
      split: test
      revision: aa460cd4d177e6a3c04fcd2affd95e8243289033
    metrics:
    - type: map_at_1
      value: 61.0
    - type: map_at_10
      value: 71.863
    - type: map_at_100
      value: 72.115
    - type: map_at_1000
      value: 72.115
    - type: map_at_3
      value: 69.0
    - type: map_at_5
      value: 70.95
    - type: mrr_at_1
      value: 61.0
    - type: mrr_at_10
      value: 71.863
    - type: mrr_at_100
      value: 72.115
    - type: mrr_at_1000
      value: 72.115
    - type: mrr_at_3
      value: 69.0
    - type: mrr_at_5
      value: 70.95
    - type: ndcg_at_1
      value: 61.0
    - type: ndcg_at_10
      value: 77.666
    - type: ndcg_at_100
      value: 78.63900000000001
    - type: ndcg_at_1000
      value: 78.63900000000001
    - type: ndcg_at_3
      value: 71.809
    - type: ndcg_at_5
      value: 75.422
    - type: precision_at_1
      value: 61.0
    - type: precision_at_10
      value: 9.6
    - type: precision_at_100
      value: 1.0
    - type: precision_at_1000
      value: 0.1
    - type: precision_at_3
      value: 26.667
    - type: precision_at_5
      value: 17.8
    - type: recall_at_1
      value: 61.0
    - type: recall_at_10
      value: 96.0
    - type: recall_at_100
      value: 100.0
    - type: recall_at_1000
      value: 100.0
    - type: recall_at_3
      value: 80.0
    - type: recall_at_5
      value: 89.0
  - task:
      type: Retrieval
    dataset:
      name: MTEB XPQARetrieval (fr)
      type: jinaai/xpqa
      config: fr
      split: test
      revision: c99d599f0a6ab9b85b065da6f9d94f9cf731679f
    metrics:
    - type: map_at_1
      value: 37.736999999999995
    - type: map_at_10
      value: 57.842000000000006
    - type: map_at_100
      value: 59.373
    - type: map_at_1000
      value: 59.426
    - type: map_at_3
      value: 51.598
    - type: map_at_5
      value: 55.279999999999994
    - type: mrr_at_1
      value: 59.68
    - type: mrr_at_10
      value: 66.71000000000001
    - type: mrr_at_100
      value: 67.28699999999999
    - type: mrr_at_1000
      value: 67.301
    - type: mrr_at_3
      value: 64.486
    - type: mrr_at_5
      value: 65.888
    - type: ndcg_at_1
      value: 59.68
    - type: ndcg_at_10
      value: 64.27199999999999
    - type: ndcg_at_100
      value: 69.429
    - type: ndcg_at_1000
      value: 70.314
    - type: ndcg_at_3
      value: 58.569
    - type: ndcg_at_5
      value: 60.272999999999996
    - type: precision_at_1
      value: 59.68
    - type: precision_at_10
      value: 15.113
    - type: precision_at_100
      value: 1.941
    - type: precision_at_1000
      value: 0.20600000000000002
    - type: precision_at_3
      value: 35.514
    - type: precision_at_5
      value: 25.367
    - type: recall_at_1
      value: 37.736999999999995
    - type: recall_at_10
      value: 73.458
    - type: recall_at_100
      value: 93.554
    - type: recall_at_1000
      value: 99.346
    - type: recall_at_3
      value: 55.774
    - type: recall_at_5
      value: 63.836000000000006
---

# {MODEL_NAME}

This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 2048 dimensional dense vector space and can be used for tasks like clustering or semantic search.

<!--- Describe your model here -->

## Usage (Sentence-Transformers)

Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:

```
pip install -U sentence-transformers
```

Then you can use the model like this:

```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('{MODEL_NAME}')
embeddings = model.encode(sentences)
print(embeddings)
```



## Evaluation Results

<!--- Describe how your model was evaluated -->

For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name={MODEL_NAME})


## Training
The model was trained with the parameters:

**DataLoader**:

`torch.utils.data.dataloader.DataLoader` of length 350 with parameters:
```
{'batch_size': 512, 'sampler': 'torch.utils.data.sampler.SequentialSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
```

**Loss**:

`__main__.MultipleNegativesRankingLoss_with_logging` 

Parameters of the fit()-Method:
```
{
    "epochs": 1,
    "evaluation_steps": 0,
    "evaluator": "NoneType",
    "max_grad_norm": 1,
    "optimizer_class": "<class 'torch.optim.adamw.AdamW'>",
    "optimizer_params": {
        "lr": 2e-05
    },
    "scheduler": "WarmupLinear",
    "steps_per_epoch": null,
    "warmup_steps": 50,
    "weight_decay": 0.01
}
```


## Full Model Architecture
```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 1024, 'do_lower_case': False}) with Transformer model: LlamaModel 
  (1): Pooling({'word_embedding_dimension': 2048, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': True, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)
```

## Citing & Authors

<!--- Describe where people can find more information -->