sentence-transformers safetensors bert sentence-similarity feature-extraction generated_from_trainer dataset_size:10000 loss:MatryoshkaLoss loss:MultipleNegativesRankingLoss en arxiv:1908.10084 arxiv:2205.13147 arxiv:1705.00652 base_model:BAAI/bge-base-en-v1.5 base_model:finetune:BAAI/bge-base-en-v1.5 license:apache-2.0 model-index autotrain_compatible text-embeddings-inference endpoints_compatible region:us
---
base_model: BAAI/bge-base-en-v1.5
datasets: []
language:
- en
library_name: sentence-transformers
license: apache-2.0
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_ndcg@100
- cosine_mrr@10
- cosine_map@100
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:10000
- loss:MatryoshkaLoss
- loss:MultipleNegativesRankingLoss
widget:
- source_sentence: Politics is about action. The German government has to take some
    action on the issue of NSA surveillance and German privacy or it will look weak.
    Interior Minister Hans-Peter Friedrich went to Washington in July but was accused
    of “returning empty-handed” and having “not moved a single step forward on any
    of the key points”. [1] The stonewalling by the United States provides an opportunity
    for opponents to Damage Merkel’s new government as well as potentially to show
    gaps between the SDP and CSU. Merkel has been invited to visit Washington at some
    point in 2014 by President Obama, [2] Merkel can’t afford for her own diplomacy
    to have as little result as Friedrich’s.  [1] Deutsche Welle, ‘SPF, Greens slam
    Interior Minister Friedrich after US surveillance talks in Washington’, dw.de,
    13 July 2013,   [2] Reuters, ‘Obama invites Merkel to visit during call about
    trade, NATO’, 8 January 2014,
  sentences:
  - what was mrs griffin accused of doing
  - are alcohol cigarettes dangerous
  - could gmo help food production
- source_sentence: Schools such as those in the county of Harrold, TX [1] have already
    introduced laws allowing teachers to carry pistols, but largely in a concealed
    fashion. This therefore leaves children unawares and thus not distracted by seeing
    teachers prominently carrying guns. Furthermore, with teachers carrying concealed
    arms, any would-be attackers would be thrown by not knowing who to shoot first,
    which would not be the case if police officers were the first on the scene.  [1]
    McKinley, James C., ‘In Texas School, Teachers Carry Books and Guns’, The New
    York Times, 28 August 2008,
  sentences:
  - why are teachers allowed to carry guns?
  - why is it important to prosecute
  - what is victim mentality
- source_sentence: While any annexation would be mutually agreed there is no guarantee
    that the whole international community would see it positively; any resistance
    from groups within Lesotho and it could be a PR nightmare. Moreover the spin of
    it being a humanitarian gesture is reliant on it following through and improving
    conditions. If it succeeds then SA will likely be called upon to resolve other
    humanitarian situations in the region such as in Swaziland.
  sentences:
  - why is congress power so important
  - how africa is dependent on foreign aid
  - should lesotho be annexed
- source_sentence: In the last 20 years, the number of people in the UK who identify
    as religious has declined by 20%. This shows that religion as a whole is becoming
    less important and, with it, marriage is becoming less important. (British Social
    Attitudes Survey 2007)
  sentences:
  - why is it important for people to identify as religious
  - is negotiation necessary for the government?
  - does the lawyer have to be privy to mediation
- source_sentence: The ICC's ability to prosecute war criminals is both overstated
    and simplistic. It has no force of its own, and must rely on its member states
    to hand over criminals wanted for prosecution. This leads to cases like that of
    Serbia, where wanted war criminals like Ratko Mladic are believed to have been
    hidden with the complicity of the regime until finally handed over in 2011. The
    absence of a force or any coercive means to bring suspects to trial also leads
    to situations like that in Libya, whereby Colonel Gaddafi is wanted by the ICC
    but the prosecution's case is germane if he manages his grip on power. Furthermore,
    it relies on external funding to operate, and can only sustain cases so long as
    financial support exists to see them through.
  sentences:
  - does the icc prosecute war crimes
  - how to reduce phone usage
  - does evolution prove that the creator did the work
model-index:
- name: SentenceTransformer based on BAAI/bge-base-en-v1.5
  results:
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 768
      type: dim_768
    metrics:
    - type: cosine_accuracy@1
      value: 0.186
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.544
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.6685
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.7995
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.186
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.18133333333333332
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.13369999999999999
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.07995000000000001
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.186
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.544
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.6685
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.7995
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.4889853894775273
      name: Cosine Ndcg@10
    - type: cosine_ndcg@100
      value: 0.5263043331639856
      name: Cosine Ndcg@100
    - type: cosine_mrr@10
      value: 0.38976746031746196
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.39800392651408967
      name: Cosine Map@100
---

# SentenceTransformer based on BAAI/bge-base-en-v1.5

This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) <!-- at revision a5beb1e3e68b9ab74eb54cfd186867f64f240e1a -->
- **Maximum Sequence Length:** 512 tokens
- **Output Dimensionality:** 768 tokens
- **Similarity Function:** Cosine Similarity
<!-- - **Training Dataset:** Unknown -->
- **Language:** en
- **License:** apache-2.0

### Model Sources

- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)

### Full Model Architecture

```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)
```

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("MugheesAwan11/bge-base-arguana-dataset-10k-2k-e1")
# Run inference
sentences = [
    "The ICC's ability to prosecute war criminals is both overstated and simplistic. It has no force of its own, and must rely on its member states to hand over criminals wanted for prosecution. This leads to cases like that of Serbia, where wanted war criminals like Ratko Mladic are believed to have been hidden with the complicity of the regime until finally handed over in 2011. The absence of a force or any coercive means to bring suspects to trial also leads to situations like that in Libya, whereby Colonel Gaddafi is wanted by the ICC but the prosecution's case is germane if he manages his grip on power. Furthermore, it relies on external funding to operate, and can only sustain cases so long as financial support exists to see them through.",
    'does the icc prosecute war crimes',
    'does evolution prove that the creator did the work',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```

<!--
### Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details>
-->

<!--
### Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details>
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

## Evaluation

### Metrics

#### Information Retrieval
* Dataset: `dim_768`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)

| Metric              | Value     |
|:--------------------|:----------|
| cosine_accuracy@1   | 0.186     |
| cosine_accuracy@3   | 0.544     |
| cosine_accuracy@5   | 0.6685    |
| cosine_accuracy@10  | 0.7995    |
| cosine_precision@1  | 0.186     |
| cosine_precision@3  | 0.1813    |
| cosine_precision@5  | 0.1337    |
| cosine_precision@10 | 0.08      |
| cosine_recall@1     | 0.186     |
| cosine_recall@3     | 0.544     |
| cosine_recall@5     | 0.6685    |
| cosine_recall@10    | 0.7995    |
| cosine_ndcg@10      | 0.489     |
| cosine_ndcg@100     | 0.5263    |
| cosine_mrr@10       | 0.3898    |
| **cosine_map@100**  | **0.398** |

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Dataset

#### Unnamed Dataset


* Size: 10,000 training samples
* Columns: <code>positive</code> and <code>anchor</code>
* Approximate statistics based on the first 1000 samples:
  |         | positive                                                                             | anchor                                                                          |
  |:--------|:-------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------|
  | type    | string                                                                               | string                                                                          |
  | details | <ul><li>min: 29 tokens</li><li>mean: 203.36 tokens</li><li>max: 512 tokens</li></ul> | <ul><li>min: 4 tokens</li><li>mean: 9.5 tokens</li><li>max: 25 tokens</li></ul> |
* Samples:
  | positive                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     | anchor                                                          |
  |:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------|
  | <code>The act of killing is emotionally damaging  To actually be involved in the death of another person is an incredibly traumatic experience. Soldiers coming back from war often suffer from ‘post-traumatic stress disorder’ which suggests that being in a situation in which you have to take another persons life has a long lasting impact on your mental health. This is also true for people who are not directly involved in the act of killing. For instance, the people who worked on developing the atomic bomb described an incredible guilt for what they had created even though they were not involved in the decision to drop the bombs. The same traumatic experiences would likely affect the person responsible for pulling the lever.</code>                                                                                                                                                                                                                                                                                                          | <code>what is a killing and how can it affect the brain?</code> |
  | <code>Deal with Corruption  Guinea-Bissau’s institutions have become too corrupt to deal with the drug problem and require support. The police, army and judiciary have all been implicated in the drug trade. The involvement of state officials in drug trafficking means that criminals are not prosecuted against. When two soldiers and a civilian were apprehended with 635kg (worth £25.4 million in 2013), they were detained and then immediately released with Colonel Arsenio Blade claiming ‘They were on the road hitching a ride’1. Judges are often bribed or sent death threats when faced with sentencing those involved in the drug trade. The USA has provided restructuring assistance to institutions which have reduced corruption, such as in the Mexico Merida Initiative, and could do the same with Guinea Bissau.  1) Vulliamy,E. ‘How a tiny West African country became the world’s first narco state’, The Guardian, 9 March 2008  2) Corcoran,P. ‘Mexico Judicial Reforms Go Easy On Corrupt Judges’, In Sight Crime, 16 February 2012</code> | <code>what has changed guinea bissau</code>                     |
  | <code>Western countries already benefit from extremely liberal laws.  The USA is at present far better than most countries in their respect and regard for civil liberties. New security measures do not greatly compromise this liberty, and the US measures are at the very least comparable with similar measures already in effect in other democratic developed countries, e.g. Spain and the UK, which have had to cope with domestic terrorism for far longer than the USA. The facts speak for themselves – the USA enjoys a healthy western-liberalism the likes of which most of the world’s people cannot even conceive of. The issue of the erosion of a few minor liberties of (states like the US’s) citizens should be overlooked in favour of the much greater issue of protecting the very existence of that state. [1]  [1] Zetter, Kim, ‘The Patriot Act Is Your Friend’, Wired, 24 February 2004,  , accessed 9 September 2011</code>                                                                                                                    | <code>which political philosophy is true about the usa?</code>  |
* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
  ```json
  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          768
      ],
      "matryoshka_weights": [
          1
      ],
      "n_dims_per_step": -1
  }
  ```

### Training Hyperparameters
#### Non-Default Hyperparameters

- `eval_strategy`: epoch
- `per_device_train_batch_size`: 32
- `per_device_eval_batch_size`: 16
- `learning_rate`: 2e-05
- `num_train_epochs`: 1
- `lr_scheduler_type`: cosine
- `warmup_ratio`: 0.1
- `bf16`: True
- `tf32`: True
- `load_best_model_at_end`: True
- `optim`: adamw_torch_fused
- `batch_sampler`: no_duplicates

#### All Hyperparameters
<details><summary>Click to expand</summary>

- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: epoch
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 32
- `per_device_eval_batch_size`: 16
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `learning_rate`: 2e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 1
- `max_steps`: -1
- `lr_scheduler_type`: cosine
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.1
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: True
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: True
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: True
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch_fused
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: False
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`: 
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `batch_sampler`: no_duplicates
- `multi_dataset_batch_sampler`: proportional

</details>

### Training Logs
| Epoch   | Step    | Training Loss | dim_768_cosine_map@100 |
|:-------:|:-------:|:-------------:|:----------------------:|
| 0.0319  | 10      | 0.5613        | -                      |
| 0.0639  | 20      | 0.4543        | -                      |
| 0.0958  | 30      | 0.2893        | -                      |
| 0.1278  | 40      | 0.2127        | -                      |
| 0.1597  | 50      | 0.1528        | -                      |
| 0.1917  | 60      | 0.1689        | -                      |
| 0.2236  | 70      | 0.1812        | -                      |
| 0.2556  | 80      | 0.1531        | -                      |
| 0.2875  | 90      | 0.1685        | -                      |
| 0.3195  | 100     | 0.1666        | -                      |
| 0.3514  | 110     | 0.1504        | -                      |
| 0.3834  | 120     | 0.139         | -                      |
| 0.4153  | 130     | 0.1174        | -                      |
| 0.4473  | 140     | 0.1602        | -                      |
| 0.4792  | 150     | 0.178         | -                      |
| 0.5112  | 160     | 0.1481        | -                      |
| 0.5431  | 170     | 0.1145        | -                      |
| 0.5751  | 180     | 0.1502        | -                      |
| 0.6070  | 190     | 0.1189        | -                      |
| 0.6390  | 200     | 0.1648        | -                      |
| 0.6709  | 210     | 0.2004        | -                      |
| 0.7029  | 220     | 0.1565        | -                      |
| 0.7348  | 230     | 0.1447        | -                      |
| 0.7668  | 240     | 0.1411        | -                      |
| 0.7987  | 250     | 0.1326        | -                      |
| 0.8307  | 260     | 0.1562        | -                      |
| 0.8626  | 270     | 0.1571        | -                      |
| 0.8946  | 280     | 0.1211        | -                      |
| 0.9265  | 290     | 0.1399        | -                      |
| 0.9585  | 300     | 0.1884        | -                      |
| 0.9904  | 310     | 0.1537        | -                      |
| **1.0** | **313** | **-**         | **0.398**              |

* The bold row denotes the saved checkpoint.

### Framework Versions
- Python: 3.10.14
- Sentence Transformers: 3.0.1
- Transformers: 4.41.2
- PyTorch: 2.1.2+cu121
- Accelerate: 0.31.0
- Datasets: 2.19.1
- Tokenizers: 0.19.1

## Citation

### BibTeX

#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
```

#### MatryoshkaLoss
```bibtex
@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning}, 
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}
```

#### MultipleNegativesRankingLoss
```bibtex
@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply}, 
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->