sentence-transformers safetensors xlm-roberta sentence-similarity feature-extraction generated_from_trainer dataset_size:1814 loss:MatryoshkaLoss loss:MultipleNegativesRankingLoss arxiv:1908.10084 arxiv:2205.13147 arxiv:1705.00652 base_model:BAAI/bge-m3 base_model:finetune:BAAI/bge-m3 model-index autotrain_compatible text-embeddings-inference endpoints_compatible region:us
---
base_model: BAAI/bge-m3
library_name: sentence-transformers
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:1814
- loss:MatryoshkaLoss
- loss:MultipleNegativesRankingLoss
widget:
- source_sentence: '


    KLF16, also known as Kruppel-like factor 16, is a multidimensional gene with a
    plethora of functions across various biological processes and cellular components.
    Here''s a summary of its characteristics:


    ### Function and Interaction


    1. **Function**: KLF16 is involved in the DNA binding, protein binding, ubiquitin-protein
    transferase activity, and ubiquitin protein ligase activity. It performs key functions
    in pathways such as neutrophil degranulation, antigen processing, ubiquitination,
    and proteasome degradation. It is also linked to processes like cell differentiation,
    membrane fusion, Golgi organization, and protein catabolic'
  sentences:
  - Which drugs, acting as selective inhibitors against both internal tandem duplication
    and tyrosine kinase domain mutations in the FLT3 receptor, also have interactions
    with genes or proteins responsible for binding to the myosin heavy chain?
  - Which cell structures or organelles engage with the genetic elements or proteins
    that are affected by the drug Ketodarolutamide?
  - Please find a gene or protein that interacts with the Neutrophil degranulation
    pathway, is connected to X-linked intellectual disability, and acts as an E3 ubiquitin-protein
    ligase for p53, core histones, and DNA polymerase beta.
- source_sentence: "\n\nThe provided information outlines a comprehensive list of\
    \ genetic variants associated with the development of coronary artery disease.\
    \ This condition is characterized by the narrowing of the coronary arteries due\
    \ to the buildup of plaque, which can lead to a variety of cardiovascular complications,\
    \ including heart attacks. The genetic variants mentioned are associated with\
    \ an increased susceptibility to the development of coronary artery disease.\n\
    \n### Key Points:\n\n1. **Genetic Variants**:\n   - **ZNF750**: Linked with coronary\
    \ artery disease, located on chromosome 11q11"
  sentences:
  - List conditions linked to Hepatocyte Growth Factor (HGF) that should not be treated
    with Nisoldipine.
  - Could you provide me with a list of medications that act on NMDA glutamate receptors
    by interacting with the genes or proteins associated with them and possess a half-life
    near 12 hours?
  - Which gene or protein, functioning as a part of the voltage-gated potassium channel
    present in cardiac cells, neurons, and microglial cells, mediates the transportation
    of compounds that exhibit synergistic effects when combined with Omega-3 fatty
    acid ethyl esters?
- source_sentence: '

    Lingual goiter subtypes manifest with specific effects or phenotypes that significantly
    impact various physiological systems. Among these, Euthyroid goiter stands out
    with a notable phenotype characterized by the absence of thyroid hormone imbalances,
    despite the presence of goiter. This subtype is often linked to conditions such
    as Euthyroid Multinodular Goiter, indicating benign thyroid nodules without hormonal
    disturbance. Additionally, individuals with Euthyroid goiter may present with
    symptoms associated with the Thyrocerebrorenal Syndrome and Branchiootorenal Syndrome,
    highlighting the broad spectrum of issues linked to these genetic conditions.
    Furthermore, the presence of a goiter as a symptom suggests an enlargement of
    the thyroid gland, which can lead to complications such as respiratory difficulties
    or other functional impairments. This specific manifestation of a lingual goiter
    subtype underscores the complexity and variability of thyroid disorders and their
    impact on a range of medical conditions.'
  sentences:
  - Which gene/protein, involved in controlling the breakdown of fatty acids and integral
    to both sugar and fat metabolic processes, interacts with the gene/protein essential
    for oxygen uptake and carbon dioxide release in red blood cells?
  - Could you describe the specific effects or phenotypes associated with lingual
    goiter subtypes?
  - Identify a gene or protein that interacts with CXCL1, where both are linked to
    a common effect or phenotype.
- source_sentence: '


    Epilepsy is a neurological disorder characterized by recurrent seizures, which
    can be partial (affecting one part of the body), generalized (involving both sides
    of the body), or both. Seizures occur due to abnormal electrical activity in the
    brain. The condition can affect individuals of all ages, including infants and
    children. While the epilepsy itself does not typically affect life expectancy,
    several complications can arise:


    ### Seizure Types

    - **Tonic-clonic seizures** (formerly known as grand mal seizures) involve a stiffening
    of the muscles, followed by loss of consciousness, and can result in convulsions.

    - **Absence seizures** feature brief lapses in consciousness, often not noticed
    by others unless a child is in the midst of an activity.

    - **Myoclonic seizures** cause sudden, jerky muscle contractions.

    - **Partial seizures** can affect only one part of the body and are often associated
    with auras (premonitory symptoms) or post-seizure confusion.


    ### Complications of Epilepsy

    1. **Status Epilepticus**: Continuous seizure activity that lasts longer than
    five minutes or frequent seizures without recovery in between can lead to brain
    damage or even death if not treated promptly.

    2. **SUDEP (Sudden Unexpected Death in Epilepsy)**: Although uncommon, SUDEP occurs
    in people with epilepsy, especially those with a high risk of death, including
    those with frequent tonic-clonic seizures or uncontrolled epilepsy.

    3. **Emotional Health Issues**: People with epilepsy may experience depression,
    anxiety, and suicidal thoughts due to the psychological impacts of the condition
    and its treatments.

    4. **Heart and Respiratory Conditions**: These can contribute to the risk of SUDEP,
    particularly in people with epilepsy.


    ### Genetic Factors

    Epilepsy can be influenced by genetic factors, as certain genes such as POMC,
    P'
  sentences:
  - Which afflictions linked to the PPP1R1B gene can lead to persistent depressive
    symptoms over a period of two years and are deemed incompatible with the use of
    Indomethacin as a treatment?
  - Which disease is associated with the AP2M1 gene and characterized by muscular
    weakness or hypotonia concurrent with seizures?
  - Which gene or protein is undetectable in both the frontal cortex of the brain
    and the nephron's tubular structures?
- source_sentence: '

    RNASE6, also known by aliases such as RAD1, RNS6, and RNasek6, functions as a
    member of the ribonuclease A superfamily. Specifically identified via the NCBI
    gene/protein database, this protein is related to the antimicrobial peptides pathway,
    showcasing broad-spectrum antimicrobial activity against pathogenic bacteria in
    the urinary tract. The provided gene summary emphasizes its role in the urinary
    tract, highlighting its enzymatic function and broad antimicrobial capability.


    With a genomic position spanning from 20781268 to 20782467 on chromosome 14, the
    RNASE6 gene encodes a protein named ribonuclease A family member k6. The protein''s
    interactions with cellular and molecular functions are integral to its role, including
    its interaction with molecular functions like ribonuclease activity and endonuclease
    activity, as well as its involvement in nucleic acid binding.


    RNASE6''s involvement in biological'
  sentences:
  - Identify the common gene targets that interact with both N-(3,5-dibromo-4-hydroxyphenyl)benzamide
    and Diflunisal.
  - What are the common genetic targets of Meclofenamic acid and Escherichia coli-derived
    Asparaginase?
  - Identify genes or proteins linked to encephalopathy that are involved in the Antimicrobial
    peptides pathway and have interactions with molecular functions associated with
    ribonuclease activity.
model-index:
- name: SentenceTransformer based on BAAI/bge-m3
  results:
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 768
      type: dim_768
    metrics:
    - type: cosine_accuracy@1
      value: 0.46534653465346537
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.5148514851485149
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.5297029702970297
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.5643564356435643
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.46534653465346537
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.17161716171617156
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.10594059405940594
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.05643564356435643
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.46534653465346537
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.5148514851485149
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.5297029702970297
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.5643564356435643
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.5136562593698354
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.4977212006914977
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.5030522827107842
      name: Cosine Map@100
---

# SentenceTransformer based on BAAI/bge-m3

This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3) on the json dataset. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3) <!-- at revision 5617a9f61b028005a4858fdac845db406aefb181 -->
- **Maximum Sequence Length:** 8192 tokens
- **Output Dimensionality:** 1024 tokens
- **Similarity Function:** Cosine Similarity
- **Training Dataset:**
    - json
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->

### Model Sources

- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)

### Full Model Architecture

```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: XLMRobertaModel 
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)
```

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("FareedKhan/BAAI_bge-m3_FareedKhan_prime_synthetic_data_2k_2_4")
# Run inference
sentences = [
    "\nRNASE6, also known by aliases such as RAD1, RNS6, and RNasek6, functions as a member of the ribonuclease A superfamily. Specifically identified via the NCBI gene/protein database, this protein is related to the antimicrobial peptides pathway, showcasing broad-spectrum antimicrobial activity against pathogenic bacteria in the urinary tract. The provided gene summary emphasizes its role in the urinary tract, highlighting its enzymatic function and broad antimicrobial capability.\n\nWith a genomic position spanning from 20781268 to 20782467 on chromosome 14, the RNASE6 gene encodes a protein named ribonuclease A family member k6. The protein's interactions with cellular and molecular functions are integral to its role, including its interaction with molecular functions like ribonuclease activity and endonuclease activity, as well as its involvement in nucleic acid binding.\n\nRNASE6's involvement in biological",
    'Identify genes or proteins linked to encephalopathy that are involved in the Antimicrobial peptides pathway and have interactions with molecular functions associated with ribonuclease activity.',
    'Identify the common gene targets that interact with both N-(3,5-dibromo-4-hydroxyphenyl)benzamide and Diflunisal.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```

<!--
### Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details>
-->

<!--
### Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details>
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

## Evaluation

### Metrics

#### Information Retrieval
* Dataset: `dim_768`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)

| Metric              | Value      |
|:--------------------|:-----------|
| cosine_accuracy@1   | 0.4653     |
| cosine_accuracy@3   | 0.5149     |
| cosine_accuracy@5   | 0.5297     |
| cosine_accuracy@10  | 0.5644     |
| cosine_precision@1  | 0.4653     |
| cosine_precision@3  | 0.1716     |
| cosine_precision@5  | 0.1059     |
| cosine_precision@10 | 0.0564     |
| cosine_recall@1     | 0.4653     |
| cosine_recall@3     | 0.5149     |
| cosine_recall@5     | 0.5297     |
| cosine_recall@10    | 0.5644     |
| cosine_ndcg@10      | 0.5137     |
| cosine_mrr@10       | 0.4977     |
| **cosine_map@100**  | **0.5031** |

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Dataset

#### json

* Dataset: json
* Size: 1,814 training samples
* Columns: <code>positive</code> and <code>anchor</code>
* Approximate statistics based on the first 1000 samples:
  |         | positive                                                                            | anchor                                                                              |
  |:--------|:------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
  | type    | string                                                                              | string                                                                              |
  | details | <ul><li>min: 6 tokens</li><li>mean: 269.24 tokens</li><li>max: 810 tokens</li></ul> | <ul><li>min: 16 tokens</li><li>mean: 40.47 tokens</li><li>max: 127 tokens</li></ul> |
* Samples:
  | positive                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             | anchor                                                                                                                                                                                                                                                                         |
  |:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
  | <code><br><br>The list you've provided seems to be a collection of various chemical compounds and drug names. These chemicals and drugs are used across multiple fields such as pharmacology, microbiology, and biochemistry. Here are some explanations and examples for the categories:<br><br>1. **Antimicrobial Compounds**:<br>   - **Cefamandole nafate**: An antibiotic used to treat serious bacterial infections.<br>   - **Dabigatran**: Used for the treatment of blood clots, particularly in people with atrial fibrillation.<br>   - **Flurbiprofen axetil**: An anti-inflammatory medication.<br>   - **Apramycin, Gentamicin C1a, Neamine, Arbekacin, etc.**: These are also antibiotics that are effective against various types of microorganisms.<br><br>2. **Pharmaceutical Compounds**:<br>   - **Biochanin A, Formononetin**: These are usually found in plants and can have estrogen-like actions, sometimes used in dietary supplements.<br>   - **Pristinamycin, Decamethoxine**: Antibiotics used for specific infections in humans.<br>  </code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          | <code>Is there a list of medications that not only exhibit a synergistic effect when combined with Reteplase for managing a common condition but also share Warfarin's mechanism of inhibiting vitamin K reductase, leading to a depletion of reduced vitamin K levels?</code> |
  | <code><br>Pterygium, a benign conjunctival neoplasm subtype, is associated with the BICD2 gene/protein, according to the document. This wedge-shaped fibrovascular lesion, originating from the bulbar conjunctiva and extending to the cornea, is linked to pterygium due to chronic exposure to solar ultraviolet radiation, heat, and dust. Pterygium may cause severe vision loss and studies suggest it could be related to neoplastic proliferation and be a stem cell disorder. The document also connects pterygium to the IFNA2 gene/protein alongside BICD2, and it is classified among familial pterygium of the conjunctiva, conjunctival pterygium, benign neoplasm of cornea, and benign conjunctival neoplasm.</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 | <code>Which benign conjunctival neoplasm subtype is associated with the BICD2 gene/protein?</code>                                                                                                                                                                             |
  | <code><br>The query you provided is a complex request that seems to be aimed at finding specific genes in the human genome. Here's a breakdown of the components of the query:<br><br>1. **Database and Filter**: It's searching a database of human long non-coding RNAs (lncRNAs) using an SQL-like query, which is a powerful way to filter through large datasets to find specific genes or transcripts.<br><br>2. **Basic Genetic Information**: A few columns are explicitly mentioned:<br>   - `gene`: This likely refers to the gene identifier.<br>   - `chromosome`: Identifies which chromosome the gene is located on.<br>   - `start`: The starting position of the gene on the chromosome.<br>   - `end`: The ending position of the gene on the chromosome.<br><br>3. **Sequence Information**: Some columns are related to the DNA sequence of the gene:<br>   - `sequence`: The full DNA sequence of the gene.<br>   - `sequence_length`: Length of the DNA sequence.<br><br>4. **LncRNA Annotations**: It looks for annotations related to lncRNAs:<br>   - `transcript_type`: The type of lncRNA.<br>   - `lnc_name`: A specific name for the lncRNA.<br><br>5. **Interactions and Overlaps**:<br>   - `LINC01873_overlap`: Checks for overlapping or interacting lncRNAs with the query lncRNA.<br><br>6. **MIRNA Annotations**:<br>   - `miRNA_name`: Name of the microRNA.<br>   - `miRNA_type`: Type of the microRNA.<br><br>7. **Miscellaneous Genes**: It also seems to include a list of other gene symbols in the form `GAS5-like`. These might be part of a query to filter or compare against a list of lncRNAs or genes.<br><br>### How It Works:<br>This query is highly specific and might be used by bioinformaticians or researchers to explore interactions between different types of RNA molecules, particularly long non-coding RNAs and microRNAs, within the context of human genetics. It allows for detailed inquiries into gene positions, interactions, and annotations, which are crucial for understanding regulatory processes</code> | <code>Which anatomical structures lack the expression of genes or proteins that play a role in the organization of microvilli?</code>                                                                                                                                          |
* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
  ```json
  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          768
      ],
      "matryoshka_weights": [
          1
      ],
      "n_dims_per_step": -1
  }
  ```

### Training Hyperparameters
#### Non-Default Hyperparameters

- `eval_strategy`: epoch
- `per_device_train_batch_size`: 4
- `learning_rate`: 1e-05
- `num_train_epochs`: 2
- `warmup_ratio`: 0.1
- `bf16`: True
- `tf32`: False
- `load_best_model_at_end`: True

#### All Hyperparameters
<details><summary>Click to expand</summary>

- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: epoch
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 4
- `per_device_eval_batch_size`: 8
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 1e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 2
- `max_steps`: -1
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.1
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: True
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: False
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: True
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: False
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`: 
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `use_liger_kernel`: False
- `eval_use_gather_object`: False
- `batch_sampler`: batch_sampler
- `multi_dataset_batch_sampler`: proportional

</details>

### Training Logs
| Epoch   | Step    | Training Loss | dim_768_cosine_map@100 |
|:-------:|:-------:|:-------------:|:----------------------:|
| 0       | 0       | -             | 0.4068                 |
| 0.0220  | 10      | 0.6723        | -                      |
| 0.0441  | 20      | 0.4606        | -                      |
| 0.0661  | 30      | 0.4175        | -                      |
| 0.0881  | 40      | 0.5941        | -                      |
| 0.1101  | 50      | 0.6731        | -                      |
| 0.1322  | 60      | 0.6091        | -                      |
| 0.1542  | 70      | 0.2194        | -                      |
| 0.1762  | 80      | 0.4262        | -                      |
| 0.1982  | 90      | 0.5397        | -                      |
| 0.2203  | 100     | 0.3848        | -                      |
| 0.2423  | 110     | 0.4412        | -                      |
| 0.2643  | 120     | 0.4608        | -                      |
| 0.2863  | 130     | 0.3824        | -                      |
| 0.3084  | 140     | 0.3241        | -                      |
| 0.3304  | 150     | 0.3768        | -                      |
| 0.3524  | 160     | 0.4086        | -                      |
| 0.3744  | 170     | 0.1928        | -                      |
| 0.3965  | 180     | 0.5528        | -                      |
| 0.4185  | 190     | 0.4628        | -                      |
| 0.4405  | 200     | 0.1467        | -                      |
| 0.4626  | 210     | 0.3845        | -                      |
| 0.4846  | 220     | 0.275         | -                      |
| 0.5066  | 230     | 0.3052        | -                      |
| 0.5286  | 240     | 0.3463        | -                      |
| 0.5507  | 250     | 0.3588        | -                      |
| 0.5727  | 260     | 0.2317        | -                      |
| 0.5947  | 270     | 0.14          | -                      |
| 0.6167  | 280     | 0.5766        | -                      |
| 0.6388  | 290     | 0.4077        | -                      |
| 0.6608  | 300     | 0.4412        | -                      |
| 0.6828  | 310     | 0.3011        | -                      |
| 0.7048  | 320     | 0.1437        | -                      |
| 0.7269  | 330     | 0.3659        | -                      |
| 0.7489  | 340     | 0.5164        | -                      |
| 0.7709  | 350     | 0.4847        | -                      |
| 0.7930  | 360     | 0.1533        | -                      |
| 0.8150  | 370     | 0.4391        | -                      |
| 0.8370  | 380     | 0.195         | -                      |
| 0.8590  | 390     | 0.1703        | -                      |
| 0.8811  | 400     | 0.5926        | -                      |
| 0.9031  | 410     | 0.193         | -                      |
| 0.9251  | 420     | 0.1602        | -                      |
| 0.9471  | 430     | 0.3054        | -                      |
| 0.9692  | 440     | 0.1246        | -                      |
| 0.9912  | 450     | 0.1128        | -                      |
| **1.0** | **454** | **-**         | **0.4923**             |
| 1.0132  | 460     | 0.3826        | -                      |
| 1.0352  | 470     | 0.3034        | -                      |
| 1.0573  | 480     | 0.416         | -                      |
| 1.0793  | 490     | 0.1682        | -                      |
| 1.1013  | 500     | 0.2183        | -                      |
| 1.1233  | 510     | 0.2287        | -                      |
| 1.1454  | 520     | 0.0765        | -                      |
| 1.1674  | 530     | 0.2437        | -                      |
| 1.1894  | 540     | 0.0393        | -                      |
| 1.2115  | 550     | 0.2171        | -                      |
| 1.2335  | 560     | 0.0611        | -                      |
| 1.2555  | 570     | 0.0328        | -                      |
| 1.2775  | 580     | 0.168         | -                      |
| 1.2996  | 590     | 0.1535        | -                      |
| 1.3216  | 600     | 0.1937        | -                      |
| 1.3436  | 610     | 0.3172        | -                      |
| 1.3656  | 620     | 0.3847        | -                      |
| 1.3877  | 630     | 0.0818        | -                      |
| 1.4097  | 640     | 0.3286        | -                      |
| 1.4317  | 650     | 0.3761        | -                      |
| 1.4537  | 660     | 0.1369        | -                      |
| 1.4758  | 670     | 0.1861        | -                      |
| 1.4978  | 680     | 0.2745        | -                      |
| 1.5198  | 690     | 0.3078        | -                      |
| 1.5419  | 700     | 0.205         | -                      |
| 1.5639  | 710     | 0.2251        | -                      |
| 1.5859  | 720     | 0.2627        | -                      |
| 1.6079  | 730     | 0.2258        | -                      |
| 1.6300  | 740     | 0.0863        | -                      |
| 1.6520  | 750     | 0.2577        | -                      |
| 1.6740  | 760     | 0.1943        | -                      |
| 1.6960  | 770     | 0.1056        | -                      |
| 1.7181  | 780     | 0.1382        | -                      |
| 1.7401  | 790     | 0.0169        | -                      |
| 1.7621  | 800     | 0.2602        | -                      |
| 1.7841  | 810     | 0.1094        | -                      |
| 1.8062  | 820     | 0.1261        | -                      |
| 1.8282  | 830     | 0.2647        | -                      |
| 1.8502  | 840     | 0.0952        | -                      |
| 1.8722  | 850     | 0.13          | -                      |
| 1.8943  | 860     | 0.0819        | -                      |
| 1.9163  | 870     | 0.1194        | -                      |
| 1.9383  | 880     | 0.2118        | -                      |
| 1.9604  | 890     | 0.0667        | -                      |
| 1.9824  | 900     | 0.2501        | -                      |
| 2.0     | 908     | -             | 0.5031                 |

* The bold row denotes the saved checkpoint.

### Framework Versions
- Python: 3.10.10
- Sentence Transformers: 3.1.1
- Transformers: 4.45.1
- PyTorch: 2.2.1+cu121
- Accelerate: 0.34.2
- Datasets: 3.0.1
- Tokenizers: 0.20.0

## Citation

### BibTeX

#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
```

#### MatryoshkaLoss
```bibtex
@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}
```

#### MultipleNegativesRankingLoss
```bibtex
@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->