sentence-transformers safetensors bert sentence-similarity feature-extraction generated_from_trainer dataset_size:1814 loss:MatryoshkaLoss loss:MultipleNegativesRankingLoss arxiv:1908.10084 arxiv:2205.13147 arxiv:1705.00652 base_model:flax-sentence-embeddings/all_datasets_v4_MiniLM-L6 base_model:finetune:flax-sentence-embeddings/all_datasets_v4_MiniLM-L6 model-index autotrain_compatible text-embeddings-inference endpoints_compatible region:us
---
base_model: flax-sentence-embeddings/all_datasets_v4_MiniLM-L6
library_name: sentence-transformers
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:1814
- loss:MatryoshkaLoss
- loss:MultipleNegativesRankingLoss
widget:
- source_sentence: '

    Optic nerve disease, as defined by the MONDO source, encompasses both non-neoplastic
    and neoplastic disorders affecting the optic nerve (second cranial nerve), making
    it a critical condition impacting the visual pathway from the optic nerve to the
    visual cortex. This disorder, according to the UMLS description, involves a disruption
    of neural transmission relevant to visual processing. The document highlights
    specific medications, including Amiodarone, Linezolid, Metronidazole, Trimethadione,
    and Ethadione, as having contraindication with optic nerve disease. This suggests
    that their use may not be advisable in cases of this condition due to potential
    exacerbation or interaction risks. The gene/protein PAX6 is associated with optic
    nerve disease, indicating genetic factors that contribute to its development.
    Furthermore, optic nerve disease is part of a broader classification including
    optic atrophy, central nervous system disease, optic neuritis, and other related
    conditions, emphasizing its significance within the spectrum of disorders affecting
    the visual system. Therefore, when considering the use of intracranial abscess
    medications in cases where optic nerve disease is implicated, healthcare providers
    should exercise caution due to the potential contraindication and the importance
    of preserving optic nerve function.'
  sentences:
  - Given my family health history with Mendelian disorders, I'm seeking information
    on a genetic disease associated with an absent or obstructed anal opening and
    potential kidney malformations, like agenesis of the kidneys. Could you identify
    conditions that present these characteristics?
  - Can you give me a list of pills or tablets that act on the RNASE2 gene or its
    associated protein?
  - Which condition impacting the visual pathway, from the optic nerve to the visual
    cortex, would deem the use of intracranial abscess medications inadvisable?
- source_sentence: '

    Bisindolylmaleimide I is a solid-form drug sourced from the DrugBank database.
    This compound belongs to the category of Acids, specifically Acyclic, Anti-Inflammatory
    Agents, Dicarboxylic Acids, Enzyme Inhibitors, Heterocyclic Compounds, Fused-Ring,
    Imides, and Maleates. It is notably experimental in nature. Notably, Bisindolylmaleimide
    I functions to target multiple kinases, namely PIM1, PDPK1, and PRKCI, showcasing
    its multifaceted approach to drug therapy and its potential utility in inhibiting
    various biological pathways or processes.'
  sentences:
  - What solid-form drugs simultaneously target PRKCI and PIM1 kinase proteins?
  - Which genes or proteins interact with IER3IP1 and are also associated with undifferentiated
    carcinoma of the corpus uteri, ensuring they code for active proteins?
  - Is there an interaction between ferric cation and the proteins or genes associated
    with the smooth endoplasmic reticulum?
- source_sentence: "\n\nThe gene in question is involved in a variety of functions\
    \ and biological processes, as well as exhibiting specific patterns of expression\
    \ across different tissues and anatomical locations.\n\n**Functional Annotations:**\n\
    \n- **Superfamily:** Transmembrane proteins (TM)\n- **Cellular Components:** Cell\
    \ membrane and plasma membrane\n- **Pathways:** ABC-family proteins mediated transport,\
    \ Abacavir transmembrane transport\n- **Biological Processes:**\n  - Transmembrane\
    \ transport\n  - Transepithelial transport\n  - Terpenoid transport\n  - Response\
    \ to drug\n  - Carboxylic acid transmembrane transport\n  - Export across plasma\
    \ membrane\n  - Regulation of chloride transport\n  - Transport across blood-brain\
    \ barrier\n  - Regulation of response to osmotic stress\n  - Phospholipid translocation\n\
    \  - Positive regulation of anion channel activity\n  - Stem cell proliferation\n\
    \  - Xenobiotic detoxification by transmembrane export across the plasma membrane\n\
    \  - Xenobiotic transport across blood-brain barrier\n  - Ceramide translocation\n\
    \  - G2/M transition of mitotic cell cycle\n\nThese annotations suggest the gene's\
    \ role in transporting substances across cellular membranes, often with a focus\
    \ on transmembrane and active transport mechanisms, and its involvement in processes\
    \ like response to chemical stimuli, detoxification, transport across barriers\
    \ like the blood-brain barrier, and specific pathways like ABC-family transporters.\n\
    \n**Expression Patterns:**\n\n- **Expression Present:** The gene is expressed\
    \ in many different tissues and anatomical locations, including uterine cervix,\
    \ islets of Langerhans, pituitary gland, zones of skin, lymph nodes, tendons,\
    \ large intestines, renal glomeruli,"
  sentences:
  - What is the name of the neuroendocrine carcinoma subtype that has the potential
    to metastasize to various parts of the body?
  - Which gene or protein is involved in the transport of pharmaceuticals that exhibit
    a synergistic effect when combined with Lanoteplase, for instance, the ATP-binding
    cassette transporter subfamily B member 1 (ABCB1) protein?
  - Can you identify a gene or protein involved in atrial fibrillation that also has
    a role in ubiquitin ligase binding and is active in the rapid repolarization phase
    of cardiac activity?
- source_sentence: '

    GZMK, also known as TRYP2, is a gene encoding a serine protease that is part of
    the cytoplasmic granules of cytotoxic lymphocytes such as cytolytic T lymphocytes
    (CTL) and natural killer (NK) cells. This group of enzymes is crucial for the
    ability of these cells to recognize and lyse target cells bearing ''nonself''
    antigens, typically peptides or proteins from intracellular pathogens. A key feature
    of GZMK is the absence of N-glycosylation consensus sequences found in other granzymes.
    This gene product is known for exhibiting serine-type peptidase activity and serine-type
    endopeptidase activity, playing an essential role in proteolysis which is part
    of the immune response.


    GZMK shows expression across a wide range of anatomical locations, including the
    uterus, zone of skin, lymph node'
  sentences:
  - Could you identify the cellular pathways that precede or follow the 'Formation
    of tubulin folding intermediates by CCT/TriC'?
  - Please find genes or proteins that interact with the extracellular matrix, exhibit
    serine peptidase activity, and trigger caspase-independent cell death through
    Bid-linked mitochondrial disruption.
  - Which genes and proteins interact with amino acid:proton symporter activity?
- source_sentence: '

    Epstein-Barr virus-associated mesenchymal tumor is a disease designated as a type
    of leiomyosarcoma within the disease nomenclature system MONDO. This specific
    condition is uniquely characterized by its association with the Epstein-Barr virus
    and exhibits symptoms commonly related to an underlying malignancy, such as fatigue,
    fever, and muscle pain. Identified as a subgroup of leiomyosarcoma, it also encompasses
    related diseases including Epstein-Barr virus-related tumor, follicular dendritic
    cell sarcoma, and myopericytoma, all of which share the hallmark of being influenced
    by the Epstein-Barr virus. This classification emphasizes the role of viral infection
    in the development and manifestation of these tumor types, offering insights into
    potential pathways of disease progression and suggesting avenues for targeted
    therapeutic interventions.'
  sentences:
  - Could you provide me with a list of medications that display synergistic effects
    when combined with Lemborexant, are prescribed for the same indications, and possess
    an elimination half-life close to 37 hours? I am interested in exploring alternative
    treatments compatible with Lemborexant's therapeutic indications that may offer
    extended efficacy through the prolonged half-life of the secondary drug when co-administered.
  - Find conditions linked to carboxylesterase 2 (CES2) that should not be treated
    with Rosiglitazone.
  - What type of leiomyosarcoma commonly manifests with fatigue, fever, and muscle
    pain?
model-index:
- name: SentenceTransformer based on flax-sentence-embeddings/all_datasets_v4_MiniLM-L6
  results:
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 384
      type: dim_384
    metrics:
    - type: cosine_accuracy@1
      value: 0.36633663366336633
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.4405940594059406
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.46534653465346537
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.5148514851485149
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.36633663366336633
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.14686468646864687
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.09306930693069305
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.051485148514851475
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.36633663366336633
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.4405940594059406
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.46534653465346537
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.5148514851485149
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.4369539416463861
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.41272591544868775
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.41870400471689506
      name: Cosine Map@100
---

# SentenceTransformer based on flax-sentence-embeddings/all_datasets_v4_MiniLM-L6

This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [flax-sentence-embeddings/all_datasets_v4_MiniLM-L6](https://huggingface.co/flax-sentence-embeddings/all_datasets_v4_MiniLM-L6) on the json dataset. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [flax-sentence-embeddings/all_datasets_v4_MiniLM-L6](https://huggingface.co/flax-sentence-embeddings/all_datasets_v4_MiniLM-L6) <!-- at revision a407cc0b7d85eec9a5617eaf51dbe7b353b0c79f -->
- **Maximum Sequence Length:** 128 tokens
- **Output Dimensionality:** 384 tokens
- **Similarity Function:** Cosine Similarity
- **Training Dataset:**
    - json
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->

### Model Sources

- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)

### Full Model Architecture

```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)
```

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("FareedKhan/flax-sentence-embeddings_all_datasets_v4_MiniLM-L6_FareedKhan_prime_synthetic_data_2k_4_16")
# Run inference
sentences = [
    '\nEpstein-Barr virus-associated mesenchymal tumor is a disease designated as a type of leiomyosarcoma within the disease nomenclature system MONDO. This specific condition is uniquely characterized by its association with the Epstein-Barr virus and exhibits symptoms commonly related to an underlying malignancy, such as fatigue, fever, and muscle pain. Identified as a subgroup of leiomyosarcoma, it also encompasses related diseases including Epstein-Barr virus-related tumor, follicular dendritic cell sarcoma, and myopericytoma, all of which share the hallmark of being influenced by the Epstein-Barr virus. This classification emphasizes the role of viral infection in the development and manifestation of these tumor types, offering insights into potential pathways of disease progression and suggesting avenues for targeted therapeutic interventions.',
    'What type of leiomyosarcoma commonly manifests with fatigue, fever, and muscle pain?',
    "Could you provide me with a list of medications that display synergistic effects when combined with Lemborexant, are prescribed for the same indications, and possess an elimination half-life close to 37 hours? I am interested in exploring alternative treatments compatible with Lemborexant's therapeutic indications that may offer extended efficacy through the prolonged half-life of the secondary drug when co-administered.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```

<!--
### Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details>
-->

<!--
### Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details>
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

## Evaluation

### Metrics

#### Information Retrieval
* Dataset: `dim_384`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)

| Metric              | Value      |
|:--------------------|:-----------|
| cosine_accuracy@1   | 0.3663     |
| cosine_accuracy@3   | 0.4406     |
| cosine_accuracy@5   | 0.4653     |
| cosine_accuracy@10  | 0.5149     |
| cosine_precision@1  | 0.3663     |
| cosine_precision@3  | 0.1469     |
| cosine_precision@5  | 0.0931     |
| cosine_precision@10 | 0.0515     |
| cosine_recall@1     | 0.3663     |
| cosine_recall@3     | 0.4406     |
| cosine_recall@5     | 0.4653     |
| cosine_recall@10    | 0.5149     |
| cosine_ndcg@10      | 0.437      |
| cosine_mrr@10       | 0.4127     |
| **cosine_map@100**  | **0.4187** |

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Dataset

#### json

* Dataset: json
* Size: 1,814 training samples
* Columns: <code>positive</code> and <code>anchor</code>
* Approximate statistics based on the first 1000 samples:
  |         | positive                                                                            | anchor                                                                              |
  |:--------|:------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
  | type    | string                                                                              | string                                                                              |
  | details | <ul><li>min: 2 tokens</li><li>mean: 117.23 tokens</li><li>max: 128 tokens</li></ul> | <ul><li>min: 13 tokens</li><li>mean: 35.72 tokens</li><li>max: 128 tokens</li></ul> |
* Samples:
  | positive                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      | anchor                                                                                                                                                                                                                                                              |
  |:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
  | <code><br>The list you've provided appears to include a wide range of substances and compounds from various fields such as chemistry, medicine, and pharmaceuticals. Here's a brief categorization and description of a few categories:<br><br>### Chemical Compounds and Their Uses<br><br>1. **Calcin**: An anticoagulant used to prevent blood clots by inhibiting the formation of calcium deposits in the blood.<br>2. **Colistimethate**: An antibiotic used to treat serious bacterial infections that are not responsive to other antibiotics.<br><br>### Medication and Drug Discovery<br><br>3. **Benznidazole**: An antiparasitic drug used to treat infections like American trypanosomiasis (Chagas' disease).<br>4. **Amediplase**: A promising stem cell derivative molecule developed as a cancer drug. It appears to have the potential to selectively target and kill cancer cells.<br><br>### Medical Imaging Agents<br><br>5. **Gadodiamide**, **Gadoteridol**, **Iothalamic acid**, **Ioversol**, and **Technetium Tc-99m exametazime**: These are contrast agents used in medical imaging to enhance the visibility of organs or tissues in MRI, CT, and other imaging scans.<br><br>### Drug Delivery and Coagulation Management<br><br>6. **Kebuzone**: Although not widely recognized, specific uses or pharmaceutical names like this might refer to a less common medication or generic name.<br>7. **Robenacoxib**: A selective COX-2 inhibitor used to manage pain and inflammation.<br>8. **Melagatran**: An anticoagulant, likely used in managing blood clotting.<br><br>### Immune Modulation and Cancer Therapy<br><br>9. **Idaruciz</code> | <code>Could you recommend any medications that effectively treat bacterial arthritis and are compatible with Alprostadil? Ideally, the medication should have a short half-life, being metabolized within an hour or so, to accommodate my active lifestyle.</code> |
  | <code><br>MYT1, also known by various aliases such as 'C20orf36', 'MTF1', 'MYTI', 'NZF2', 'PLPB1', 'ZC2H2C1', and 'ZC2HC4A', is a gene/protein that codes for a zinc finger-containing DNA-binding protein. This gene is part of a family of neural-specific proteins that play a crucial role in the developing nervous system by binding to the promoter regions of proteolipid proteins in the central nervous system. The protein encoded by MYT1 has been associated with certain diseases, including oculo-auriculo-vertebral spectrum and hemifacial microsomia. It is involved in various cellular components such as the nucleus, chromatin, cytosol, and nucleoplasm, and its biological processes include cell differentiation, regulation of transcription by RNA polymerase II, and nervous system development. MYT1 is expressed in a wide variety of tissues, such as the pituitary gland, pituitary gland</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              | <code>Which gene or protein is not expressed in the stomach fundus and nasal cavity epithelial tissue?</code>                                                                                                                                                       |
  | <code><br><br>### Key Information on Long QT Syndrome<br><br>#### What is Long QT Syndrome?<br>- **Definition**: A genetic condition that affects the heart's electrical system, causing abnormal heart rhythms.<br>- **Signs & Symptoms**: Syncope (fainting), seizures, abnormal heart rhythms, and sudden cardiac death.<br><br>#### Forms of Long QT Syndrome<br>- **Congenital (Romano-Ward Syndrome)**: Occurs when a single gene variant is inherited from one parent, often with normal hearing.<br>- **Acquired**: Resulting from medications, medical conditions, or electrolyte imbalances.<br><br>#### Risk Factors<br>- **Hereditary**: A family history increases risk.<br>- **Certain Medications**: Common antibiotics, antifungals, and some antidepressants can cause it.<br>- **Medical Conditions**: Diarrhea, vomiting, eating disorders, or acute kidney injury.<br><br>#### Complications<br>- **Torsades de Pointes**: Chaotic heart rhythm that can be fatal.<br>- **Ventricular Fibrillation**: Rapid, ineffective heartbeats; can lead to sudden death without immediate treatment.<br>- **Sudden Death**: Occurs in an otherwise healthy individual.<br><br>#### Prevention<br>- **Medication Review**: Regularly check medications for</code>                                                                                                                                                                                                                                                                                                                                                                                                    | <code>Which cardiac arrhythmia contraindicates the use of medications prescribed for bladder infections?</code>                                                                                                                                                     |
* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
  ```json
  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          384
      ],
      "matryoshka_weights": [
          1
      ],
      "n_dims_per_step": -1
  }
  ```

### Training Hyperparameters
#### Non-Default Hyperparameters

- `eval_strategy`: epoch
- `per_device_train_batch_size`: 16
- `learning_rate`: 1e-05
- `num_train_epochs`: 4
- `warmup_ratio`: 0.1
- `bf16`: True
- `tf32`: False
- `load_best_model_at_end`: True

#### All Hyperparameters
<details><summary>Click to expand</summary>

- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: epoch
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 16
- `per_device_eval_batch_size`: 8
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 1e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 4
- `max_steps`: -1
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.1
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: True
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: False
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: True
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: False
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`: 
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `use_liger_kernel`: False
- `eval_use_gather_object`: False
- `batch_sampler`: batch_sampler
- `multi_dataset_batch_sampler`: proportional

</details>

### Training Logs
| Epoch   | Step    | Training Loss | dim_384_cosine_map@100 |
|:-------:|:-------:|:-------------:|:----------------------:|
| 0       | 0       | -             | 0.3971                 |
| 0.0877  | 10      | 1.5497        | -                      |
| 0.1754  | 20      | 1.334         | -                      |
| 0.2632  | 30      | 1.2332        | -                      |
| 0.3509  | 40      | 1.1818        | -                      |
| 0.4386  | 50      | 1.087         | -                      |
| 0.5263  | 60      | 1.2103        | -                      |
| 0.6140  | 70      | 1.1323        | -                      |
| 0.7018  | 80      | 1.0869        | -                      |
| 0.7895  | 90      | 0.9275        | -                      |
| 0.8772  | 100     | 1.0684        | -                      |
| 0.9649  | 110     | 0.9702        | -                      |
| **1.0** | **114** | **-**         | **0.4142**             |
| 1.0526  | 120     | 1.0792        | -                      |
| 1.1404  | 130     | 1.1194        | -                      |
| 1.2281  | 140     | 0.9212        | -                      |
| 1.3158  | 150     | 1.0393        | -                      |
| 1.4035  | 160     | 1.099         | -                      |
| 1.4912  | 170     | 0.8902        | -                      |
| 1.5789  | 180     | 0.854         | -                      |
| 1.6667  | 190     | 0.6828        | -                      |
| 1.7544  | 200     | 0.9187        | -                      |
| 1.8421  | 210     | 0.8597        | -                      |
| 1.9298  | 220     | 1.0286        | -                      |
| 2.0     | 228     | -             | 0.4179                 |
| 2.0175  | 230     | 0.6874        | -                      |
| 2.1053  | 240     | 0.7523        | -                      |
| 2.1930  | 250     | 0.7594        | -                      |
| 2.2807  | 260     | 0.6929        | -                      |
| 2.3684  | 270     | 0.7718        | -                      |
| 2.4561  | 280     | 0.7803        | -                      |
| 2.5439  | 290     | 0.7324        | -                      |
| 2.6316  | 300     | 0.7252        | -                      |
| 2.7193  | 310     | 0.7532        | -                      |
| 2.8070  | 320     | 0.8368        | -                      |
| 2.8947  | 330     | 0.9413        | -                      |
| 2.9825  | 340     | 0.7401        | -                      |
| 3.0     | 342     | -             | 0.4185                 |
| 3.0702  | 350     | 0.6514        | -                      |
| 3.1579  | 360     | 0.6765        | -                      |
| 3.2456  | 370     | 0.8422        | -                      |
| 3.3333  | 380     | 0.6532        | -                      |
| 3.4211  | 390     | 0.7121        | -                      |
| 3.5088  | 400     | 0.5739        | -                      |
| 3.5965  | 410     | 0.7838        | -                      |
| 3.6842  | 420     | 0.7554        | -                      |
| 3.7719  | 430     | 0.743         | -                      |
| 3.8596  | 440     | 0.5219        | -                      |
| 3.9474  | 450     | 0.8437        | -                      |
| 4.0     | 456     | -             | 0.4187                 |

* The bold row denotes the saved checkpoint.

### Framework Versions
- Python: 3.10.10
- Sentence Transformers: 3.1.1
- Transformers: 4.45.1
- PyTorch: 2.2.1+cu121
- Accelerate: 0.34.2
- Datasets: 3.0.1
- Tokenizers: 0.20.0

## Citation

### BibTeX

#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
```

#### MatryoshkaLoss
```bibtex
@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}
```

#### MultipleNegativesRankingLoss
```bibtex
@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->