sentence-transformers safetensors qwen2 sentence-similarity feature-extraction generated_from_trainer dataset_size:21769 loss:MultipleNegativesRankingLoss arxiv:1908.10084 arxiv:1705.00652 base_model:am-azadi/KaLM-embedding-multilingual-mini-v1_Fine_Tuned_1e base_model:finetune:am-azadi/KaLM-embedding-multilingual-mini-v1_Fine_Tuned_1e autotrain_compatible text-embeddings-inference endpoints_compatible region:us
---
base_model: am-azadi/KaLM-embedding-multilingual-mini-v1_Fine_Tuned_1e
library_name: sentence-transformers
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:21769
- loss:MultipleNegativesRankingLoss
widget:
- source_sentence: Day 1 - Job Losses Biden CANCELS Keystone Pipeline on - Day 1.
    XX 83k jobs lost. XX Get ready for $4 Gas prices by Summer.
  sentences:
  - Pedro Castillo manipulated a photograph of a rally Photo of a Castillo rally in
    Peru was altered to show alleged manipulation
  - Biden's Keystone XL pipeline reversal will cost tens of thousands of jobs Posts
    inflate job losses from Biden's Keystone pipeline reversal
  - Delete System32 folder to clear your privacy browsing history? This hoax has been
    around for years, don’t delete System32 folder
- source_sentence: ' Thiago Brazil  The "Russian missile" turns cars and destroys
    the street but the windows of the buildings did not break down and neither did
    smeared with dust. 1 more fake advertising piece of the neo-Nazi Zelensky. need
    do more next time!!'
  sentences:
  - Intact windows in Ukraine prove Russian attack simulation Intact windows and overturned
    cars do not prove simulated attack in Bucha, Ukraine
  - Editorial of the newspaper Le Monde says that Mexico is on the way to ruin The
    newspaper Le Monde did not publish an editorial titled "Mexico on a direct path
    to ruin"
  - A photo of 189-year-old Jonathan the turtle This photo does not show the 189-year-old
    Jonathan tortoise, but a giant tortoise in Australia
- source_sentence: ' madri Greta Thunberg, urged the Chinese to do without traditional
    chopsticks to protect the trees The Chinese then asked Greta to return to the
    school she was in could find out that traditional Chopsticks are made from bamboo,
    and bamboo is a grass! The Chinese have Greta and her friends too asked to give
    up toilet paper to wipe her ass ''cause this one will made from trees.'
  sentences:
  - This is how they repress in Bolivia tear gas grenade to the head The man killed
    by the shot of a tear gas grenade to the head was the victim of the repression
    of a protest in Iraq, not in Bolivia
  - Greta Thunberg urged China to ban chopsticks There is no evidence for Greta Thunberg's
    demand for a ban on chopsticks
  - Accurate reporting on Pfizer-BioNTech Covid-19 vaccination drive and deaths in
    Germany Social media posts misrepresent Pfizer-BioNTech Covid-19 vaccinations
    in Germany
- source_sentence: Do you want to know the truth? The "Vaccine" they are voting for
    is mRNA, it circulates through the blood until it finds a receptor for the enzyme
    ACE2, which exists mainly in the testicles, a little in the ovaries, and in the
    myelin of neurons. The m is for messenger, through the receptor, the RNA penetrates
    the cell and rewrites its genetic code. The cell no longer serves what nature
    created it for, it serves to create what the laboratories designed RNA for. Theoretically
    "defenses". The price is that 97% of the inoculated males will remain sterile,
    but also, if they are young children, they will never develop secondary sexual
    characteristics. They will be androgynous, without sexual desire, or very moderate,
    and probably more manageable and obedient. 45% of girls will be sterile. Neuronal
    damage, on the other hand, will affect part of your frontal cortex. You will be
    able to work, even drive a car, but not think deeply. Perfect slaves of the New
    Normality. It is PHASE 3 of the plan, as it was finalized in EVENT 201. PHASE
    1 was to scare you, isolate you and lock you up, due to a virus that, as a single
    cause, only killed a tiny handful of people. Much less than last year's flu. PHASE
    2 makes you wear a grotesque and useless mask, which depersonalizes you and deprives
    you of oxygen. May you lose your job, partner and affections. PHASE 3, when you
    are already desperate, is the "Vaccine". They're going to tell you I'm lying,
    so ask what's in it. They will answer that by law not even doctors can know. Secret.
    You don't have the maturity to know it, trust the government, the media, the WHO
    and the employees of George Soros and Bill Gates, like Dr. Pedro Cahn and his
    sinister Fundación Huésped. They will tell you that the laboratories are responsible,
    but by law you will not be able to claim anyone. Let's see, when your balls dry,
    you'll only have to cry about what's left of them. When you know that you will
    never have a grandchild, that you will never see your son become a father, nor
    graduate from a career, because his brain will be lobotomized. The opposition"?
    I don't want to make you bitter, but 90% receive money from Soros' Open Society,
    from the Bill and Melinda Gates Foundation, from the Ford Foundation, from the
    Rockefeller Foundation, from the Chinese Communist Party through its figurehead,
    the investor Ming Wai Lau. Who will tell you the truth? Dr. Roxana Bruno, Dr.
    Chinda Brandolino, Dr. Heiko Schöning, Doctors for Truth, Lawyers for Truth, Teachers
    for Truth. Turn off the TV, burn your muzzle, breathe, go out to hug your parents,
    embrace freedom, don't let any doctor who isn't the truth touch you. In this way
    the Global Sanitary Dictatorship will collapse, and we will wake up from this
    nightmare. From the wall of Horacio Rivara CLIPARTO CLIPARTO CLIPARTO CLIPARTO
    CLIPARTO CLIPARTO C C
  sentences:
  - Today- Mayor of São Paulo thanking the Bolsonaro government on vaccines against
    covid-19 Video with thanks to the federal government was for funds for a hospital
    in May 2020
  - This photo shows Bolsonaro surrounded by global leaders at a G20 meeting No, the
    photo is a montage and who appears in the original is US President Donald Trump
  - Vaccines to prevent covid-19 cause infertility Vaccines to prevent covid-19 are
    not designed to affect fertility
- source_sentence: 'The moment of the death of President Mohamed Morsi, may God have
    mercy on him, God willing '
  sentences:
  - Cuba has Interferon Alpha 2B, the cure, the vaccine against the coronavirus The
    Cuban antiviral Interferon Alfa 2B is used in China to treat patients with the
    new coronavirus, but it is neither a vaccine nor a cure
  - 'José Antonio Kast said: "Juvenile delinquent of 16 or 17 years will not go to
    sename, he will do compulsory military service" Chilean presidential candidate
    Kast does not propose that young people who commit crimes do military service'
  - The moment of the death of President Mohamed Morsi This video belongs to the trial
    of those accused of the Port Said events and does not show the moment of the death
    of former Egyptian President Mohamed Morsi
---

# SentenceTransformer based on am-azadi/KaLM-embedding-multilingual-mini-v1_Fine_Tuned_1e

This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [am-azadi/KaLM-embedding-multilingual-mini-v1_Fine_Tuned_1e](https://huggingface.co/am-azadi/KaLM-embedding-multilingual-mini-v1_Fine_Tuned_1e). It maps sentences & paragraphs to a 896-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [am-azadi/KaLM-embedding-multilingual-mini-v1_Fine_Tuned_1e](https://huggingface.co/am-azadi/KaLM-embedding-multilingual-mini-v1_Fine_Tuned_1e) <!-- at revision db90e52b2078421f04b71e31b5a90f5bf8d321d7 -->
- **Maximum Sequence Length:** 512 tokens
- **Output Dimensionality:** 896 dimensions
- **Similarity Function:** Cosine Similarity
<!-- - **Training Dataset:** Unknown -->
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->

### Model Sources

- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)

### Full Model Architecture

```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: Qwen2Model 
  (1): Pooling({'word_embedding_dimension': 896, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)
```

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'The moment of the death of President Mohamed Morsi, may God have mercy on him, God willing ',
    'The moment of the death of President Mohamed Morsi This video belongs to the trial of those accused of the Port Said events and does not show the moment of the death of former Egyptian President Mohamed Morsi',
    'Cuba has Interferon Alpha 2B, the cure, the vaccine against the coronavirus The Cuban antiviral Interferon Alfa 2B is used in China to treat patients with the new coronavirus, but it is neither a vaccine nor a cure',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 896]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```

<!--
### Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details>
-->

<!--
### Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details>
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Dataset

#### Unnamed Dataset

* Size: 21,769 training samples
* Columns: <code>sentence_0</code> and <code>sentence_1</code>
* Approximate statistics based on the first 1000 samples:
  |         | sentence_0                                                                         | sentence_1                                                                          |
  |:--------|:-----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
  | type    | string                                                                             | string                                                                              |
  | details | <ul><li>min: 4 tokens</li><li>mean: 109.9 tokens</li><li>max: 512 tokens</li></ul> | <ul><li>min: 14 tokens</li><li>mean: 34.57 tokens</li><li>max: 132 tokens</li></ul> |
* Samples:
  | sentence_0                                                                                                                                                                                                                                                                        | sentence_1                                                                                                                                                                                                                           |
  |:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
  | <code>Sad, and we are hostages!! If that doesn't make you think about the “measures” that governors and mayors are taking there is nothing that can be made in Brazil</code>                                                                                                      | <code>Action by a military police officer against a street vendor amid restrictive measures due to the covid-19 pandemic The photo in which a PM seizes products from a street vendor is from 2016, unrelated to the pandemic</code> |
  | <code> This is why it's important to know your history d4 Rare photo of Queen Elizabeth II and Prince Phillip bowing before the real original African Royalty, Empress Menen Asfaw and her husband Emperor Ras Tafari Makonnen Woldemikael Haile Selassie I of Ethiopia...</code> | <code>Queen Elizabeth II bows before Ethiopian Emperor Haile Selassie British monarch's first visit to Ethiopia came 10 years after this photo was taken</code>                                                                      |
  | <code>Public Reaction on Hyderabad Priyanka Reddy Case Drabad Encounter . Common people say</code>                                                                                                                                                                                | <code>Photo of suspects killed by police in Hyderabad rape-murder case This photo has circulated online since at least 2015 in connection with an unrelated case in a different Indian state</code>                                  |
* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
  ```json
  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  ```

### Training Hyperparameters
#### Non-Default Hyperparameters

- `per_device_train_batch_size`: 2
- `per_device_eval_batch_size`: 2
- `num_train_epochs`: 1
- `multi_dataset_batch_sampler`: round_robin

#### All Hyperparameters
<details><summary>Click to expand</summary>

- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: no
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 2
- `per_device_eval_batch_size`: 2
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 5e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1
- `num_train_epochs`: 1
- `max_steps`: -1
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.0
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: False
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: False
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: None
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `include_for_metrics`: []
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`: 
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `use_liger_kernel`: False
- `eval_use_gather_object`: False
- `average_tokens_across_devices`: False
- `prompts`: None
- `batch_sampler`: batch_sampler
- `multi_dataset_batch_sampler`: round_robin

</details>

### Training Logs
| Epoch  | Step  | Training Loss |
|:------:|:-----:|:-------------:|
| 0.0459 | 500   | 0.0083        |
| 0.0919 | 1000  | 0.019         |
| 0.1378 | 1500  | 0.0255        |
| 0.1837 | 2000  | 0.0372        |
| 0.2297 | 2500  | 0.0315        |
| 0.2756 | 3000  | 0.0258        |
| 0.3215 | 3500  | 0.0211        |
| 0.3675 | 4000  | 0.0187        |
| 0.4134 | 4500  | 0.0264        |
| 0.4593 | 5000  | 0.0348        |
| 0.5053 | 5500  | 0.0197        |
| 0.5512 | 6000  | 0.0102        |
| 0.5972 | 6500  | 0.0092        |
| 0.6431 | 7000  | 0.0169        |
| 0.6890 | 7500  | 0.0109        |
| 0.7350 | 8000  | 0.0115        |
| 0.7809 | 8500  | 0.0173        |
| 0.8268 | 9000  | 0.0196        |
| 0.8728 | 9500  | 0.028         |
| 0.9187 | 10000 | 0.0218        |
| 0.9646 | 10500 | 0.0169        |
| 0.0459 | 500   | 0.004         |
| 0.0919 | 1000  | 0.02          |
| 0.1378 | 1500  | 0.0154        |
| 0.1837 | 2000  | 0.0141        |
| 0.2297 | 2500  | 0.014         |
| 0.2756 | 3000  | 0.0077        |
| 0.3215 | 3500  | 0.018         |
| 0.3675 | 4000  | 0.0079        |
| 0.4134 | 4500  | 0.0238        |
| 0.4593 | 5000  | 0.0183        |
| 0.5053 | 5500  | 0.0159        |
| 0.5512 | 6000  | 0.0043        |
| 0.5972 | 6500  | 0.0066        |
| 0.6431 | 7000  | 0.0068        |
| 0.6890 | 7500  | 0.0035        |
| 0.7350 | 8000  | 0.0042        |
| 0.7809 | 8500  | 0.0084        |
| 0.8268 | 9000  | 0.0049        |
| 0.8728 | 9500  | 0.0102        |
| 0.9187 | 10000 | 0.0048        |
| 0.9646 | 10500 | 0.0045        |


### Framework Versions
- Python: 3.11.11
- Sentence Transformers: 3.4.1
- Transformers: 4.48.3
- PyTorch: 2.5.1+cu124
- Accelerate: 1.3.0
- Datasets: 3.3.2
- Tokenizers: 0.21.0

## Citation

### BibTeX

#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
```

#### MultipleNegativesRankingLoss
```bibtex
@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->