sentence-transformers safetensors bert sentence-similarity feature-extraction generated_from_trainer dataset_size:90 loss:MatryoshkaLoss loss:MultipleNegativesRankingLoss en arxiv:1908.10084 arxiv:2205.13147 arxiv:1705.00652 base_model:BAAI/bge-base-en-v1.5 base_model:finetune:BAAI/bge-base-en-v1.5 license:apache-2.0 model-index autotrain_compatible text-embeddings-inference endpoints_compatible region:us
---
base_model: BAAI/bge-base-en-v1.5
datasets: []
language:
- en
library_name: sentence-transformers
license: apache-2.0
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:90
- loss:MatryoshkaLoss
- loss:MultipleNegativesRankingLoss
widget:
- source_sentence: Ownership of NVIDIA Securities Information regarding ownership
    of NVIDIA securities required by this item will be contained in our 2023 Proxy
    Statement under the caption “Security Ownership of Certain Beneficial Owners and
    Management,” and is hereby incorporated by reference.
  sentences:
  - What are the two operating segments of NVIDIA as mentioned in the text?
  - What major factors contributed to the decrease in cash provided by operating activities
    in fiscal year 2023?
  - Where can information regarding the ownership of NVIDIA securities be found?
- source_sentence: Development and Retention To support employee development, we provide
    opportunities to learn on-the-job through training programs, one on one coaching
    and ongoing feedback. We have a library of live and on-demand learning experiences
    that include workshops, panel discussions, and speaker forums. We curate learning
    paths focused on our most common development needs and constantly upgrade our
    offerings to ensure that our employees are exposed to the most current programs
    and technologies available.
  sentences:
  - How much is authorized for the repurchase of additional shares of common stock
    as of January 29, 2023?
  - What position did Timothy S. Teter acquire at NVIDIA in 2018?
  - What types of learning opportunities does the company provide to support employee
    development?
- source_sentence: Data Center The NVIDIA computing platform is focused on accelerating
    the most compute-intensive workloads, such as AI, data analytics, graphics and
    scientific computing, across hyperscale, cloud, enterprise, public sector, and
    edge data centers. The platform consists of our energy efficient GPUs, data processing
    units, or DPUs, interconnects and systems, our CUDA programming model, and a growing
    body of software libraries, software development kits, or SDKs, application frameworks
    and services, which are either available as part of the platform or packaged and
    sold separately.
  sentences:
  - What position did Colette M. Kress hold before joining NVIDIA in 2013?
  - Where can NVIDIA's financial reports be accessed?
  - What are the key components of the NVIDIA computing platform?
- source_sentence: Human Capital Management We believe that our employees are our
    greatest assets, and they play a key role in creating long-term value for our
    stakeholders. As of the end of fiscal year 2023, we had 26,196 employees in 35
    countries, 19,532 were engaged in research and development and 6,664 were engaged
    in sales, marketing, operations, and administrative positions.
  sentences:
  - What industries use NVIDIA's GPUs and software for automation?
  - How many employees did the company have at the end of fiscal year 2023, and in
    how many countries were they located?
  - How does NVIDIA's platform strategy contribute to the markets it serves?
- source_sentence: Equity Compensation Plan Information Information regarding our
    equity compensation plans required by this item will be contained in our 2023
    Proxy Statement under the caption "Equity Compensation Plan Information," and
    is hereby incorporated by reference.
  sentences:
  - What amount is recorded as unrecognized tax benefits at the end of fiscal year
    2023?
  - What is the total amount authorized for the repurchase of common stock up to December
    2023?
  - What document contains details about NVIDIA's equity compensation plans?
model-index:
- name: BGE base Financial Matryoshka
  results:
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 768
      type: dim_768
    metrics:
    - type: cosine_accuracy@1
      value: 0.7
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.9
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 1.0
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 1.0
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.7
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.3
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.2
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.1
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.7
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.9
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 1.0
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 1.0
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.869253606521631
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.825
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.825
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 512
      type: dim_512
    metrics:
    - type: cosine_accuracy@1
      value: 0.7
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.9
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 1.0
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 1.0
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.7
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.3
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.2
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.1
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.7
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.9
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 1.0
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 1.0
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.869253606521631
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.825
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.825
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 256
      type: dim_256
    metrics:
    - type: cosine_accuracy@1
      value: 0.7
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.9
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 1.0
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 1.0
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.7
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.3
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.2
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.1
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.7
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.9
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 1.0
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 1.0
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.869253606521631
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.825
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.825
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 128
      type: dim_128
    metrics:
    - type: cosine_accuracy@1
      value: 0.7
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.9
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 1.0
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 1.0
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.7
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.3
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.2
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.1
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.7
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.9
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 1.0
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 1.0
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.869253606521631
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.825
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.825
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 64
      type: dim_64
    metrics:
    - type: cosine_accuracy@1
      value: 0.6
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.8
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.9
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 1.0
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.6
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.2666666666666667
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.18
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.1
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.6
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.8
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.9
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 1.0
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.7993566060880289
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.736111111111111
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.736111111111111
      name: Cosine Map@100
---

# BGE base Financial Matryoshka

This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) <!-- at revision a5beb1e3e68b9ab74eb54cfd186867f64f240e1a -->
- **Maximum Sequence Length:** 512 tokens
- **Output Dimensionality:** 768 tokens
- **Similarity Function:** Cosine Similarity
<!-- - **Training Dataset:** Unknown -->
- **Language:** en
- **License:** apache-2.0

### Model Sources

- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)

### Full Model Architecture

```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)
```

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("anikulkar/bge-base-financial-matryoshka-nvda-iter20")
# Run inference
sentences = [
    'Equity Compensation Plan Information Information regarding our equity compensation plans required by this item will be contained in our 2023 Proxy Statement under the caption "Equity Compensation Plan Information," and is hereby incorporated by reference.',
    "What document contains details about NVIDIA's equity compensation plans?",
    'What is the total amount authorized for the repurchase of common stock up to December 2023?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```

<!--
### Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details>
-->

<!--
### Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details>
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

## Evaluation

### Metrics

#### Information Retrieval
* Dataset: `dim_768`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)

| Metric              | Value     |
|:--------------------|:----------|
| cosine_accuracy@1   | 0.7       |
| cosine_accuracy@3   | 0.9       |
| cosine_accuracy@5   | 1.0       |
| cosine_accuracy@10  | 1.0       |
| cosine_precision@1  | 0.7       |
| cosine_precision@3  | 0.3       |
| cosine_precision@5  | 0.2       |
| cosine_precision@10 | 0.1       |
| cosine_recall@1     | 0.7       |
| cosine_recall@3     | 0.9       |
| cosine_recall@5     | 1.0       |
| cosine_recall@10    | 1.0       |
| cosine_ndcg@10      | 0.8693    |
| cosine_mrr@10       | 0.825     |
| **cosine_map@100**  | **0.825** |

#### Information Retrieval
* Dataset: `dim_512`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)

| Metric              | Value     |
|:--------------------|:----------|
| cosine_accuracy@1   | 0.7       |
| cosine_accuracy@3   | 0.9       |
| cosine_accuracy@5   | 1.0       |
| cosine_accuracy@10  | 1.0       |
| cosine_precision@1  | 0.7       |
| cosine_precision@3  | 0.3       |
| cosine_precision@5  | 0.2       |
| cosine_precision@10 | 0.1       |
| cosine_recall@1     | 0.7       |
| cosine_recall@3     | 0.9       |
| cosine_recall@5     | 1.0       |
| cosine_recall@10    | 1.0       |
| cosine_ndcg@10      | 0.8693    |
| cosine_mrr@10       | 0.825     |
| **cosine_map@100**  | **0.825** |

#### Information Retrieval
* Dataset: `dim_256`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)

| Metric              | Value     |
|:--------------------|:----------|
| cosine_accuracy@1   | 0.7       |
| cosine_accuracy@3   | 0.9       |
| cosine_accuracy@5   | 1.0       |
| cosine_accuracy@10  | 1.0       |
| cosine_precision@1  | 0.7       |
| cosine_precision@3  | 0.3       |
| cosine_precision@5  | 0.2       |
| cosine_precision@10 | 0.1       |
| cosine_recall@1     | 0.7       |
| cosine_recall@3     | 0.9       |
| cosine_recall@5     | 1.0       |
| cosine_recall@10    | 1.0       |
| cosine_ndcg@10      | 0.8693    |
| cosine_mrr@10       | 0.825     |
| **cosine_map@100**  | **0.825** |

#### Information Retrieval
* Dataset: `dim_128`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)

| Metric              | Value     |
|:--------------------|:----------|
| cosine_accuracy@1   | 0.7       |
| cosine_accuracy@3   | 0.9       |
| cosine_accuracy@5   | 1.0       |
| cosine_accuracy@10  | 1.0       |
| cosine_precision@1  | 0.7       |
| cosine_precision@3  | 0.3       |
| cosine_precision@5  | 0.2       |
| cosine_precision@10 | 0.1       |
| cosine_recall@1     | 0.7       |
| cosine_recall@3     | 0.9       |
| cosine_recall@5     | 1.0       |
| cosine_recall@10    | 1.0       |
| cosine_ndcg@10      | 0.8693    |
| cosine_mrr@10       | 0.825     |
| **cosine_map@100**  | **0.825** |

#### Information Retrieval
* Dataset: `dim_64`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)

| Metric              | Value      |
|:--------------------|:-----------|
| cosine_accuracy@1   | 0.6        |
| cosine_accuracy@3   | 0.8        |
| cosine_accuracy@5   | 0.9        |
| cosine_accuracy@10  | 1.0        |
| cosine_precision@1  | 0.6        |
| cosine_precision@3  | 0.2667     |
| cosine_precision@5  | 0.18       |
| cosine_precision@10 | 0.1        |
| cosine_recall@1     | 0.6        |
| cosine_recall@3     | 0.8        |
| cosine_recall@5     | 0.9        |
| cosine_recall@10    | 1.0        |
| cosine_ndcg@10      | 0.7994     |
| cosine_mrr@10       | 0.7361     |
| **cosine_map@100**  | **0.7361** |

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Dataset

#### Unnamed Dataset


* Size: 90 training samples
* Columns: <code>positive</code> and <code>anchor</code>
* Approximate statistics based on the first 1000 samples:
  |         | positive                                                                            | anchor                                                                             |
  |:--------|:------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
  | type    | string                                                                              | string                                                                             |
  | details | <ul><li>min: 22 tokens</li><li>mean: 56.66 tokens</li><li>max: 142 tokens</li></ul> | <ul><li>min: 11 tokens</li><li>mean: 19.33 tokens</li><li>max: 32 tokens</li></ul> |
* Samples:
  | positive                                                                                                                                                                                                                                                                                                                                                                                                                                                                 | anchor                                                                                                                              |
  |:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------|
  | <code>We also offer the NVIDIA GPU Cloud registry, or NGC, a comprehensive catalog of easy-to-use, optimized software stacks across a range of domains including scientific computing, deep learning, and machine learning. With NGC, AI developers, researchers and data scientists can get started with the development of AI and HPC applications and deploy them on DGX systems, NVIDIA-Certified systems from our partners, or with NVIDIA’s cloud partners.</code> | <code>What does the NVIDIA GPU Cloud registry offer?</code>                                                                         |
  | <code>To the extent realization of the deferred tax assets becomes more-likely-than-not, we would recognize such deferred tax assets as income tax benefits during the period.</code>                                                                                                                                                                                                                                                                                    | <code>What will be recognized as income tax benefits if the realization of deferred tax assets becomes more-likely-than-not?</code> |
  | <code>Fueled by the sustained demand for exceptional 3D graphics and the scale of the gaming market, NVIDIA has leveraged its GPU architecture to create platforms for scientific computing, AI, data science, AV, robotics, metaverse and 3D internet applications.</code>                                                                                                                                                                                              | <code>How did NVIDIA pivot its GPU architecture usage beyond PC graphics?</code>                                                    |
* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
  ```json
  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          768,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  ```

### Training Hyperparameters
#### Non-Default Hyperparameters

- `eval_strategy`: epoch
- `per_device_train_batch_size`: 32
- `per_device_eval_batch_size`: 16
- `gradient_accumulation_steps`: 16
- `learning_rate`: 2e-05
- `num_train_epochs`: 20
- `lr_scheduler_type`: cosine
- `warmup_ratio`: 0.1
- `tf32`: False
- `load_best_model_at_end`: True
- `batch_sampler`: no_duplicates

#### All Hyperparameters
<details><summary>Click to expand</summary>

- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: epoch
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 32
- `per_device_eval_batch_size`: 16
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 16
- `eval_accumulation_steps`: None
- `learning_rate`: 2e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 20
- `max_steps`: -1
- `lr_scheduler_type`: cosine
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.1
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: False
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: False
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: True
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: False
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`: 
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `batch_sampler`: no_duplicates
- `multi_dataset_batch_sampler`: proportional

</details>

### Training Logs
| Epoch      | Step  | Training Loss | dim_128_cosine_map@100 | dim_256_cosine_map@100 | dim_512_cosine_map@100 | dim_64_cosine_map@100 | dim_768_cosine_map@100 |
|:----------:|:-----:|:-------------:|:----------------------:|:----------------------:|:----------------------:|:---------------------:|:----------------------:|
| 1.0        | 1     | -             | 0.6952                 | 0.6617                 | 0.725                  | 0.5966                | 0.7167                 |
| 2.0        | 2     | -             | 0.7060                 | 0.75                   | 0.8                    | 0.6086                | 0.8                    |
| 3.0        | 3     | -             | 0.72                   | 0.75                   | 0.8                    | 0.6277                | 0.75                   |
| 4.0        | 4     | -             | 0.8                    | 0.75                   | 0.8                    | 0.6283                | 0.75                   |
| 1.0        | 1     | -             | 0.8                    | 0.75                   | 0.8                    | 0.6283                | 0.75                   |
| 2.0        | 2     | -             | 0.8                    | 0.75                   | 0.8                    | 0.6424                | 0.775                  |
| 3.0        | 3     | -             | 0.8                    | 0.75                   | 0.825                  | 0.71                  | 0.775                  |
| **4.3333** | **5** | **-**         | **0.825**              | **0.825**              | **0.825**              | **0.735**             | **0.775**              |
| 5.0        | 6     | -             | 0.825                  | 0.825                  | 0.825                  | 0.735                 | 0.775                  |
| 6.0        | 7     | -             | 0.825                  | 0.825                  | 0.825                  | 0.735                 | 0.825                  |
| 7.0        | 8     | -             | 0.825                  | 0.825                  | 0.825                  | 0.735                 | 0.825                  |
| 8.3333     | 10    | 0.1558        | 0.825                  | 0.825                  | 0.825                  | 0.7375                | 0.825                  |
| 9.0        | 11    | -             | 0.825                  | 0.825                  | 0.825                  | 0.7375                | 0.825                  |
| 10.0       | 12    | -             | 0.825                  | 0.825                  | 0.825                  | 0.7375                | 0.825                  |
| 11.0       | 13    | -             | 0.825                  | 0.825                  | 0.825                  | 0.7361                | 0.825                  |
| 12.3333    | 15    | -             | 0.825                  | 0.825                  | 0.825                  | 0.7361                | 0.825                  |
| 13.0       | 16    | -             | 0.825                  | 0.825                  | 0.825                  | 0.7361                | 0.825                  |
| 14.0       | 17    | -             | 0.825                  | 0.825                  | 0.825                  | 0.7361                | 0.825                  |
| 15.0       | 18    | -             | 0.825                  | 0.825                  | 0.825                  | 0.7361                | 0.825                  |
| 16.3333    | 20    | 0.0272        | 0.825                  | 0.825                  | 0.825                  | 0.7361                | 0.825                  |

* The bold row denotes the saved checkpoint.

### Framework Versions
- Python: 3.10.12
- Sentence Transformers: 3.0.1
- Transformers: 4.41.2
- PyTorch: 2.3.0+cu121
- Accelerate: 0.32.1
- Datasets: 2.20.0
- Tokenizers: 0.19.1

## Citation

### BibTeX

#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
```

#### MatryoshkaLoss
```bibtex
@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning}, 
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}
```

#### MultipleNegativesRankingLoss
```bibtex
@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply}, 
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->