sentence-transformers safetensors bert sentence-similarity feature-extraction generated_from_trainer dataset_size:4247 loss:MatryoshkaLoss loss:MultipleNegativesRankingLoss en arxiv:1908.10084 arxiv:2205.13147 arxiv:1705.00652 base_model:BAAI/bge-base-en-v1.5 base_model:finetune:BAAI/bge-base-en-v1.5 license:apache-2.0 model-index autotrain_compatible text-embeddings-inference endpoints_compatible region:us
---
base_model: BAAI/bge-base-en-v1.5
language:
- en
library_name: sentence-transformers
license: apache-2.0
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:4247
- loss:MatryoshkaLoss
- loss:MultipleNegativesRankingLoss
widget:
- source_sentence: What is the effect of the absence of Saccharomyces cerevisiae Rrm3p?
  sentences:
  - The Saccharomyces cerevisiae RRM3 gene encodes a 5' to 3' DNA helicase. While
    replication of most of the yeast genome was not dependent upon Rrm3p, in its absence,
    replication forks paused and often broke at an estimated 1400 discrete sites,
    including tRNA genes, centromeres, inactive replication origins, and transcriptional
    silencers. These replication defects were associated with activation of the intra-S
    phase checkpoint. Activation of the checkpoint was critical for viability of rrm3Delta
    cells, especially at low temperatures.
  - MPE-seq (methidiumpropyl-EDTA sequencing) is a new method for the genome-wide
    characterization of chromatin that involves the digestion of nuclei with MPE-Fe(II)
    followed by massively parallel sequencing. Like micrococcal nuclease (MNase),
    MPE-Fe(II) preferentially cleaves the linker DNA between nucleosomes. However,
    there are differences in the cleavage of nuclear chromatin by MPE-Fe(II) relative
    to MNase. MPE-seq provides a unique and straightforward means for the genome-wide
    analysis of chromatin structure with minimal DNA sequence bias. In particular,
    the combined use of MPE-seq and MNase-seq enables the identification of noncanonical
    chromatin structures that are likely to be important for the regulation of gene
    expression.
  - 'Yes,

    Diffuse intrinsic pontine gliomas (DIPG) are the most aggressive brain tumors
    in children with 5-year survival rates of only 2%. About 85% of all DIPG are characterized
    by a lysine-to-methionine substitution in histone 3, which leads to global H3K27
    hypomethylation accompanied by H3K27 hyperacetylation.'
- source_sentence: What are the clinical trial outcomes of metformin use in polycystic
    ovary disease?
  sentences:
  - "Metformin treatment vs placebo significantly but modestly improves ovulation\
    \ frequency in women with abnormal ovarian function/oligomenorrhea and polycystic\
    \ ovaries,  the lower BMI women were more likely to become pregnant.  While in\
    \ naturally conceiving normal weight PCOS women pre-treatment with metformin tends\
    \ to improve pregnancy rates, pre-treatment with metformin prior to conventional\
    \ IVF/ICSI in women with PCOS does not improve stimulation or clinical outcome.\n\
    Metformin is an effective addition to Clomifene Citrate in term of reestablishment\
    \ of ovulation and full-term pregnancies achievement.\nStudies on the effect of\
    \ metformin on serum Anti-Mullerian Hormone levels /AMH concentrations bring conflicting\
    \ results, from Metformin having no effect on AMH  to Metformin treatment resulting\
    \ in significant decrease of AMH levels, antral follicle numbers and ovarian volume.\
    \ \nMetformin has been shown to cause significant weight loss (and leptin reduction)\
    \ in PCOS."
  - Mosdepth is a command-line tool for rapidly calculating genome-wide sequencing
    coverage. It measures depth from BAM or CRAM files at either each nucleotide position
    in a genome or for sets of genomic regions. Genomic regions may be specified as
    either a BED file to evaluate coverage across capture regions, or as a fixed-size
    window as required for copy-number calling. Mosdepth uses a simple algorithm that
    is computationally efficient and enables it to quickly produce coverage summaries.
  - The myotubularin family of proteins are lipid inositol phosphatases
- source_sentence: List the blood group antigens, associated with blood type
  sentences:
  - Filter Aided Sample Preparation (FASP), a type of proteomic reactor, in which
    samples dissolved in sodium dodecyl sulfate (SDS) are digested in an ultrafiltration
    unit.
  - No. Overexpression of Uc.63+ increases the expression of AR and its downstream
    molecule PSA and promotes resistance to docetaxel through AR regulation. In patients
    treated with docetaxel, the expression of serum Uc.63+ in the docetaxel-resistant
    patients is higher than that in the docetaxel-sensitive patients (P = 0.011).
    Moreover, Kaplan-Meier analysis indicates that the high expression of serum Uc.63+
    correlated with a worse prognosis (P = 0.020).
  - ABO antigens are highly abundant in many human cell types, including platelets,
    vascular endotheliums, and red blood cells.
- source_sentence: In which isochores are Alu elements enriched?
  sentences:
  - Yes, subdural empyema can be a complication of sinusitis
  - Alu elements are enriched in high GC% isochores due to reduced Alu loss by recombination
    in these regions. The frequency of Alu sequences increases with increasing GC,
    but attains a maximum in H2 isochores.
  - Prostaglandins are involved in tissue regeneration after myocardial infarction
    and inhibitors of prostanoid production, such as aspirin and COX-2 inhibitors,
    have negative impact in this process. Furthermore, pharmacological interference
    with prostaglandin synthesis following myocardial infarction is associated with
    reduced fibrillar collagen formation.
- source_sentence: Which proteins are regulated by Nrf2?
  sentences:
  - A ciliopathy is any genetic disorder that affects the cellular cilia or the cilia
    anchoring structures, the basal bodies, or ciliary function
  - Keap1-Nrf2 system is known as a sensor of electrophilic compounds, and protects
    cells from oxidative stress through induction of various antioxidant enzymes.
  - 'Muenke syndrome is an autosomal dominant disorder characterized by coronal suture
    craniosynostosis, hearing loss, developmental delay, carpal and tarsal fusions,
    and the presence of the Pro250Arg mutation in the FGFR3 gene.   Muenke syndrome
    is characterized by coronal craniosynostosis (bilateral more often than unilateral),
    hearing loss, developmental delay, and carpal and/or tarsal bone coalition. Tarsal
    coalition is a distinct feature of Muenke syndrome and has been reported since
    the initial description of the disorder in the 1990s.   '
model-index:
- name: bge_based_arg_minibio_matryoshka
  results:
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 768
      type: dim_768
    metrics:
    - type: cosine_accuracy@1
      value: 0.8559322033898306
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.9385593220338984
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.9533898305084746
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.972457627118644
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.8559322033898306
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.31285310734463273
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.19067796610169493
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.09724576271186441
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.8559322033898306
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.9385593220338984
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.9533898305084746
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.972457627118644
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.9175775315505871
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.8996334409469999
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.9001760395975993
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 512
      type: dim_512
    metrics:
    - type: cosine_accuracy@1
      value: 0.8495762711864406
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.934322033898305
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.9533898305084746
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.9745762711864406
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.8495762711864406
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.31144067796610164
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.1906779661016949
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.09745762711864407
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.8495762711864406
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.934322033898305
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.9533898305084746
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.9745762711864406
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.9159735691496526
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.8968918146354585
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.8973466678587073
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 256
      type: dim_256
    metrics:
    - type: cosine_accuracy@1
      value: 0.8432203389830508
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.9300847457627118
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.951271186440678
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.9661016949152542
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.8432203389830508
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.31002824858757055
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.19025423728813562
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.09661016949152544
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.8432203389830508
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.9300847457627118
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.951271186440678
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.9661016949152542
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.9095971879335102
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.8910327549098734
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.8918430577940674
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 128
      type: dim_128
    metrics:
    - type: cosine_accuracy@1
      value: 0.826271186440678
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.913135593220339
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.9385593220338984
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.9555084745762712
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.826271186440678
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.3043785310734463
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.18771186440677967
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.09555084745762712
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.826271186440678
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.913135593220339
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.9385593220338984
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.9555084745762712
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.8962001591462352
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.8767268630616086
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.8779707932197566
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 64
      type: dim_64
    metrics:
    - type: cosine_accuracy@1
      value: 0.788135593220339
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.8919491525423728
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.9110169491525424
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.9300847457627118
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.788135593220339
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.2973163841807909
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.18220338983050846
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.0930084745762712
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.788135593220339
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.8919491525423728
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.9110169491525424
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.9300847457627118
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.86422428009177
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.8425780199085281
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.844512693618036
      name: Cosine Map@100
---

# bge_based_arg_minibio_matryoshka

This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) on the json dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) <!-- at revision a5beb1e3e68b9ab74eb54cfd186867f64f240e1a -->
- **Maximum Sequence Length:** 512 tokens
- **Output Dimensionality:** 768 dimensions
- **Similarity Function:** Cosine Similarity
- **Training Dataset:**
    - json
- **Language:** en
- **License:** apache-2.0

### Model Sources

- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)

### Full Model Architecture

```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)
```

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("Thyme233/bge_based_arg_minibio_matryoshka")
# Run inference
sentences = [
    'Which proteins are regulated by Nrf2?',
    'Keap1-Nrf2 system is known as a sensor of electrophilic compounds, and protects cells from oxidative stress through induction of various antioxidant enzymes.',
    'Muenke syndrome is an autosomal dominant disorder characterized by coronal suture craniosynostosis, hearing loss, developmental delay, carpal and tarsal fusions, and the presence of the Pro250Arg mutation in the FGFR3 gene.   Muenke syndrome is characterized by coronal craniosynostosis (bilateral more often than unilateral), hearing loss, developmental delay, and carpal and/or tarsal bone coalition. Tarsal coalition is a distinct feature of Muenke syndrome and has been reported since the initial description of the disorder in the 1990s.   ',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```

<!--
### Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details>
-->

<!--
### Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details>
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

## Evaluation

### Metrics

#### Information Retrieval

* Datasets: `dim_768`, `dim_512`, `dim_256`, `dim_128` and `dim_64`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)

| Metric              | dim_768    | dim_512   | dim_256    | dim_128    | dim_64     |
|:--------------------|:-----------|:----------|:-----------|:-----------|:-----------|
| cosine_accuracy@1   | 0.8559     | 0.8496    | 0.8432     | 0.8263     | 0.7881     |
| cosine_accuracy@3   | 0.9386     | 0.9343    | 0.9301     | 0.9131     | 0.8919     |
| cosine_accuracy@5   | 0.9534     | 0.9534    | 0.9513     | 0.9386     | 0.911      |
| cosine_accuracy@10  | 0.9725     | 0.9746    | 0.9661     | 0.9555     | 0.9301     |
| cosine_precision@1  | 0.8559     | 0.8496    | 0.8432     | 0.8263     | 0.7881     |
| cosine_precision@3  | 0.3129     | 0.3114    | 0.31       | 0.3044     | 0.2973     |
| cosine_precision@5  | 0.1907     | 0.1907    | 0.1903     | 0.1877     | 0.1822     |
| cosine_precision@10 | 0.0972     | 0.0975    | 0.0966     | 0.0956     | 0.093      |
| cosine_recall@1     | 0.8559     | 0.8496    | 0.8432     | 0.8263     | 0.7881     |
| cosine_recall@3     | 0.9386     | 0.9343    | 0.9301     | 0.9131     | 0.8919     |
| cosine_recall@5     | 0.9534     | 0.9534    | 0.9513     | 0.9386     | 0.911      |
| cosine_recall@10    | 0.9725     | 0.9746    | 0.9661     | 0.9555     | 0.9301     |
| **cosine_ndcg@10**  | **0.9176** | **0.916** | **0.9096** | **0.8962** | **0.8642** |
| cosine_mrr@10       | 0.8996     | 0.8969    | 0.891      | 0.8767     | 0.8426     |
| cosine_map@100      | 0.9002     | 0.8973    | 0.8918     | 0.878      | 0.8445     |

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Dataset

#### json

* Dataset: json
* Size: 4,247 training samples
* Columns: <code>question</code> and <code>answer</code>
* Approximate statistics based on the first 1000 samples:
  |         | question                                                                          | answer                                                                             |
  |:--------|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
  | type    | string                                                                            | string                                                                             |
  | details | <ul><li>min: 6 tokens</li><li>mean: 15.83 tokens</li><li>max: 49 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 59.36 tokens</li><li>max: 512 tokens</li></ul> |
* Samples:
  | question                                                                        | answer                                                                                                                                                                                                                                                                                                                                                                                                                           |
  |:--------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
  | <code>Is TNNI3K a cardiac-specific protein?</code>                              | <code>Yes, TNNI3K is highly expressed in heart but is undetectable in other tissues.</code>                                                                                                                                                                                                                                                                                                                                      |
  | <code>Which are the effects of ALDH2 deficiency?</code>                         | <code>In alcohol drinkers, ALDH2-deficiency is a well-known risk factor for upper aerodigestive tract cancers, i.e., head and neck cancer and esophageal cancer. Diabetic patients with ALDH2 mutations are predisposed to worse diastolic dysfunction.<br>These data demonstrate that ALDH2 deficiency enhances EtOH-induced disruption of intestinal epithelial tight junctions, barrier dysfunction, and liver damage.</code> |
  | <code>Has intepirdine been evaluated in clinical trials? (November 2017)</code> | <code>Yes, intepirdine was in Phase III clinical trials in November 2017.</code>                                                                                                                                                                                                                                                                                                                                                 |
* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
  ```json
  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          768,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  ```

### Training Hyperparameters
#### Non-Default Hyperparameters

- `eval_strategy`: epoch
- `per_device_train_batch_size`: 32
- `per_device_eval_batch_size`: 16
- `gradient_accumulation_steps`: 16
- `learning_rate`: 2e-05
- `num_train_epochs`: 4
- `lr_scheduler_type`: cosine
- `warmup_ratio`: 0.1
- `load_best_model_at_end`: True
- `optim`: adamw_torch_fused
- `batch_sampler`: no_duplicates

#### All Hyperparameters
<details><summary>Click to expand</summary>

- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: epoch
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 32
- `per_device_eval_batch_size`: 16
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 16
- `eval_accumulation_steps`: None
- `learning_rate`: 2e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 4
- `max_steps`: -1
- `lr_scheduler_type`: cosine
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.1
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: False
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: True
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch_fused
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: False
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`: 
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `prompts`: None
- `batch_sampler`: no_duplicates
- `multi_dataset_batch_sampler`: proportional

</details>

### Training Logs
| Epoch      | Step   | Training Loss | dim_768_cosine_ndcg@10 | dim_512_cosine_ndcg@10 | dim_256_cosine_ndcg@10 | dim_128_cosine_ndcg@10 | dim_64_cosine_ndcg@10 |
|:----------:|:------:|:-------------:|:----------------------:|:----------------------:|:----------------------:|:----------------------:|:---------------------:|
| 0.9624     | 8      | -             | 0.9242                 | 0.9186                 | 0.9157                 | 0.8907                 | 0.8504                |
| 1.2030     | 10     | 1.6488        | -                      | -                      | -                      | -                      | -                     |
| 1.9248     | 16     | -             | 0.9166                 | 0.9169                 | 0.9099                 | 0.8949                 | 0.8623                |
| 2.4060     | 20     | 0.6601        | -                      | -                      | -                      | -                      | -                     |
| **2.8872** | **24** | **-**         | **0.9173**             | **0.916**              | **0.9096**             | **0.8966**             | **0.8645**            |
| 3.6090     | 30     | 0.5199        | -                      | -                      | -                      | -                      | -                     |
| 3.8496     | 32     | -             | 0.9176                 | 0.9160                 | 0.9096                 | 0.8962                 | 0.8642                |

* The bold row denotes the saved checkpoint.

### Framework Versions
- Python: 3.10.8
- Sentence Transformers: 3.3.1
- Transformers: 4.41.2
- PyTorch: 2.5.1+cu124
- Accelerate: 1.1.1
- Datasets: 2.19.1
- Tokenizers: 0.19.1

## Citation

### BibTeX

#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
```

#### MatryoshkaLoss
```bibtex
@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}
```

#### MultipleNegativesRankingLoss
```bibtex
@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->