sentence-transformers safetensors bert sentence-similarity feature-extraction generated_from_trainer dataset_size:6300 loss:MatryoshkaLoss loss:MultipleNegativesRankingLoss en arxiv:1908.10084 arxiv:2205.13147 arxiv:1705.00652 base_model:BAAI/bge-base-en-v1.5 base_model:finetune:BAAI/bge-base-en-v1.5 license:apache-2.0 model-index autotrain_compatible text-embeddings-inference endpoints_compatible region:us
---
base_model: BAAI/bge-base-en-v1.5
language:
- en
library_name: sentence-transformers
license: apache-2.0
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:6300
- loss:MatryoshkaLoss
- loss:MultipleNegativesRankingLoss
widget:
- source_sentence: What was the percentage change in operating revenue from 2021 to
    2022?
  sentences:
  - General and administrative expense increased $1.1 billion, or 113%, in 2023, compared
    to 2022, primarily due to an increase of $991 million related to business and
    operational taxes.
  - From 2021 to 2022, the operating revenue increased by 4%, from $4,923.9 million
    to $5,122.2 million.
  - The Financial Statement Schedule is located on page S-1 of the IBM Form 10-K for
    2023.
- source_sentence: What is the volume of natural gas Chevron is contractually committed
    to deliver in the United States from 2024 through 2026?
  sentences:
  - From 2024 through 2026, Chevron is contractually committed to deliver 746 billion
    cubic feet of natural gas to third parties and affiliates in the United States.
  - In the fiscal fourth quarter of 2021, the Company recognized a loss on certain
    U.S. affiliates related to the previously impaired book value of certain intangibles,
    which reduced the 2021 effective tax rate by approximately 1.6%. This reduction
    is reflected as a Tax benefits from loss on capital assets on the effective tax
    rate reconciliation.
  - In the United States, the company purchases HFCS to meet its and its bottlers’
    requirements with the assistance of Coca-Cola Bottlers’ Sales & Services Company
    LLC, which is a procurement service provider for their North American operations.
- source_sentence: What is the purpose of using non-GAAP measures such as contribution
    margin and adjusted income from operations in financial reporting?
  sentences:
  - We use the non-GAAP measures contribution margin; gross profit and gross margin,
    excluding stock-based compensation; and adjusted income from operations, which
    excludes stock-based compensation and related employer payroll taxes, to help
    us evaluate our business, identify trends affecting our business, formulate business
    plans and financial projections, and make strategic decisions.
  - A portion of the defense and/or settlement costs associated with such litigation
    is covered by indemnification from third parties in limited cases.
  - In Fall 2023, we moved to certify the decision for an interlocutory appeal or
    for reconsideration, and briefing related to that motion is now complete.
- source_sentence: How does Chubb respond to the risks associated with climate change?
  sentences:
  - Equifax Inc. is a global data, analytics and technology company. We provide information
    solutions for businesses, governments and consumers.
  - Chubb mitigates exposure to climate change risk by ceding catastrophe risk in
    our insurance portfolio through both reinsurance and capital markets, and our
    investment portfolio through the diversification of risk, industry, location,
    type and duration of security.
  - 'The company believes that the funds available, including cash expected to be
    generated from operations, funds from a commercial paper program, or lines of
    credit, are adequate to meet its working capital needs for the year, including
    the repayment of a $500 million debt. '
- source_sentence: What was the total long-term debt, including the current portion,
    for AbbVie as of December 31, 2023?
  sentences:
  - 'Depreciation is calculated on a straight-line basis over the estimated useful
    lives of those assets as follows: Buildings and building improvements | 10 to
    40 years  Computers, equipment and software | 3 to 5 years  Furniture and fixtures
    | 5 years  Leasehold improvements | Shorter of the estimated lease term or 10
    years.'
  - In the first quarter of 2023, we began production shipments of Enphase branded
    EV chargers at our existing contract manufacturing facility in Mexico.
  - The following table summarizes AbbVie's estimated material contractual obligations
    as of December 31, 2023, which includes long-term debt, both current and long-term
    portions, totaling $59,245 million.
model-index:
- name: BGE base Financial Matryoshka
  results:
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 768
      type: dim_768
    metrics:
    - type: cosine_accuracy@1
      value: 0.7328571428571429
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.8428571428571429
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.8857142857142857
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.9171428571428571
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.7328571428571429
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.28095238095238095
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.17714285714285713
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.09171428571428569
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.7328571428571429
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.8428571428571429
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.8857142857142857
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.9171428571428571
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.8263158679921938
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.7970068027210883
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.8009257946986601
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 512
      type: dim_512
    metrics:
    - type: cosine_accuracy@1
      value: 0.73
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.85
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.8885714285714286
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.9185714285714286
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.73
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.2833333333333333
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.17771428571428569
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.09185714285714283
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.73
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.85
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.8885714285714286
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.9185714285714286
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.8268840111497038
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.7970912698412697
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.8008761750139807
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 256
      type: dim_256
    metrics:
    - type: cosine_accuracy@1
      value: 0.7314285714285714
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.8385714285714285
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.8771428571428571
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.92
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.7314285714285714
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.27952380952380956
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.1754285714285714
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.09199999999999997
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.7314285714285714
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.8385714285714285
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.8771428571428571
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.92
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.8244558099874894
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.793972789115646
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.7976616174443595
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 128
      type: dim_128
    metrics:
    - type: cosine_accuracy@1
      value: 0.7157142857142857
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.8285714285714286
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.8657142857142858
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.9142857142857143
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.7157142857142857
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.2761904761904762
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.17314285714285713
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.09142857142857141
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.7157142857142857
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.8285714285714286
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.8657142857142858
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.9142857142857143
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.8143397407863787
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.7824767573696144
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.7860396743589053
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 64
      type: dim_64
    metrics:
    - type: cosine_accuracy@1
      value: 0.6842857142857143
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.7971428571428572
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.8514285714285714
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.8957142857142857
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.6842857142857143
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.26571428571428574
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.17028571428571426
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.08957142857142855
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.6842857142857143
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.7971428571428572
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.8514285714285714
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.8957142857142857
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.7897515442737716
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.7559229024943309
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.7602008369032786
      name: Cosine Map@100
---

# BGE base Financial Matryoshka

This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) on the json dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) <!-- at revision a5beb1e3e68b9ab74eb54cfd186867f64f240e1a -->
- **Maximum Sequence Length:** 512 tokens
- **Output Dimensionality:** 768 dimensions
- **Similarity Function:** Cosine Similarity
- **Training Dataset:**
    - json
- **Language:** en
- **License:** apache-2.0

### Model Sources

- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)

### Full Model Architecture

```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)
```

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("ChristianBernhard/bge-base-financial-matryoshka-2")
# Run inference
sentences = [
    'What was the total long-term debt, including the current portion, for AbbVie as of December 31, 2023?',
    "The following table summarizes AbbVie's estimated material contractual obligations as of December 31, 2023, which includes long-term debt, both current and long-term portions, totaling $59,245 million.",
    'In the first quarter of 2023, we began production shipments of Enphase branded EV chargers at our existing contract manufacturing facility in Mexico.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```

<!--
### Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details>
-->

<!--
### Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details>
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

## Evaluation

### Metrics

#### Information Retrieval

* Datasets: `dim_768`, `dim_512`, `dim_256`, `dim_128` and `dim_64`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)

| Metric              | dim_768    | dim_512    | dim_256    | dim_128    | dim_64     |
|:--------------------|:-----------|:-----------|:-----------|:-----------|:-----------|
| cosine_accuracy@1   | 0.7329     | 0.73       | 0.7314     | 0.7157     | 0.6843     |
| cosine_accuracy@3   | 0.8429     | 0.85       | 0.8386     | 0.8286     | 0.7971     |
| cosine_accuracy@5   | 0.8857     | 0.8886     | 0.8771     | 0.8657     | 0.8514     |
| cosine_accuracy@10  | 0.9171     | 0.9186     | 0.92       | 0.9143     | 0.8957     |
| cosine_precision@1  | 0.7329     | 0.73       | 0.7314     | 0.7157     | 0.6843     |
| cosine_precision@3  | 0.281      | 0.2833     | 0.2795     | 0.2762     | 0.2657     |
| cosine_precision@5  | 0.1771     | 0.1777     | 0.1754     | 0.1731     | 0.1703     |
| cosine_precision@10 | 0.0917     | 0.0919     | 0.092      | 0.0914     | 0.0896     |
| cosine_recall@1     | 0.7329     | 0.73       | 0.7314     | 0.7157     | 0.6843     |
| cosine_recall@3     | 0.8429     | 0.85       | 0.8386     | 0.8286     | 0.7971     |
| cosine_recall@5     | 0.8857     | 0.8886     | 0.8771     | 0.8657     | 0.8514     |
| cosine_recall@10    | 0.9171     | 0.9186     | 0.92       | 0.9143     | 0.8957     |
| **cosine_ndcg@10**  | **0.8263** | **0.8269** | **0.8245** | **0.8143** | **0.7898** |
| cosine_mrr@10       | 0.797      | 0.7971     | 0.794      | 0.7825     | 0.7559     |
| cosine_map@100      | 0.8009     | 0.8009     | 0.7977     | 0.786      | 0.7602     |

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Dataset

#### json

* Dataset: json
* Size: 6,300 training samples
* Columns: <code>anchor</code> and <code>positive</code>
* Approximate statistics based on the first 1000 samples:
  |         | anchor                                                                            | positive                                                                           |
  |:--------|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
  | type    | string                                                                            | string                                                                             |
  | details | <ul><li>min: 2 tokens</li><li>mean: 20.58 tokens</li><li>max: 42 tokens</li></ul> | <ul><li>min: 7 tokens</li><li>mean: 46.12 tokens</li><li>max: 512 tokens</li></ul> |
* Samples:
  | anchor                                                                                                                                                                        | positive                                                                                                                                                                                                                                       |
  |:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
  | <code>How many new stores did the company open in fiscal 2023?</code>                                                                                                         | <code>We had net new store openings of 197 for fiscal 2023.</code>                                                                                                                                                                             |
  | <code>What can users do with Windows 365?</code>                                                                                                                              | <code>From another internet-connected device, including Android or macOS devices, users can run Windows 365, just like a virtual machine.</code>                                                                                               |
  | <code>What is the effective date for the fifteen percent minimum tax based on adjusted financial statement income as specified in the Inflation Reduction Act of 2022?</code> | <code>On August 16, 2022, the U.S. government enacted the Inflation Reduction Act of 2022 that includes a fifteen percent minimum tax based on adjusted financial statement income, which is effective for NIKE beginning June 1, 2023.</code> |
* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
  ```json
  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          768,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  ```

### Training Hyperparameters
#### Non-Default Hyperparameters

- `eval_strategy`: epoch
- `per_device_train_batch_size`: 32
- `per_device_eval_batch_size`: 16
- `gradient_accumulation_steps`: 16
- `learning_rate`: 2e-05
- `num_train_epochs`: 10
- `lr_scheduler_type`: cosine
- `warmup_ratio`: 0.1
- `bf16`: True
- `tf32`: True
- `load_best_model_at_end`: True
- `optim`: adamw_torch_fused
- `batch_sampler`: no_duplicates

#### All Hyperparameters
<details><summary>Click to expand</summary>

- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: epoch
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 32
- `per_device_eval_batch_size`: 16
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 16
- `eval_accumulation_steps`: None
- `learning_rate`: 2e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 10
- `max_steps`: -1
- `lr_scheduler_type`: cosine
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.1
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: True
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: True
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: True
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch_fused
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: False
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`: 
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `prompts`: None
- `batch_sampler`: no_duplicates
- `multi_dataset_batch_sampler`: proportional

</details>

### Training Logs
| Epoch      | Step   | Training Loss | dim_768_cosine_ndcg@10 | dim_512_cosine_ndcg@10 | dim_256_cosine_ndcg@10 | dim_128_cosine_ndcg@10 | dim_64_cosine_ndcg@10 |
|:----------:|:------:|:-------------:|:----------------------:|:----------------------:|:----------------------:|:----------------------:|:---------------------:|
| 0.8122     | 10     | 1.7456        | -                      | -                      | -                      | -                      | -                     |
| 0.9746     | 12     | -             | 0.7938                 | 0.7946                 | 0.7889                 | 0.7703                 | 0.7223                |
| 1.6244     | 20     | 0.7675        | -                      | -                      | -                      | -                      | -                     |
| 1.9492     | 24     | -             | 0.8159                 | 0.8171                 | 0.8085                 | 0.7989                 | 0.7705                |
| 2.4365     | 30     | 0.4625        | -                      | -                      | -                      | -                      | -                     |
| 2.9239     | 36     | -             | 0.8221                 | 0.8223                 | 0.8196                 | 0.8059                 | 0.7776                |
| 3.2487     | 40     | 0.3201        | -                      | -                      | -                      | -                      | -                     |
| 3.9797     | 49     | -             | 0.8232                 | 0.8257                 | 0.8221                 | 0.8135                 | 0.7865                |
| 4.0609     | 50     | 0.2489        | -                      | -                      | -                      | -                      | -                     |
| 4.8731     | 60     | 0.196         | -                      | -                      | -                      | -                      | -                     |
| **4.9543** | **61** | **-**         | **0.8253**             | **0.8255**             | **0.8219**             | **0.816**              | **0.7856**            |
| 5.6853     | 70     | 0.1628        | -                      | -                      | -                      | -                      | -                     |
| 5.9289     | 73     | -             | 0.8259                 | 0.8252                 | 0.8219                 | 0.8155                 | 0.7864                |
| 6.4975     | 80     | 0.1605        | -                      | -                      | -                      | -                      | -                     |
| 6.9848     | 86     | -             | 0.8264                 | 0.8250                 | 0.8240                 | 0.8139                 | 0.7887                |
| 7.3096     | 90     | 0.1357        | -                      | -                      | -                      | -                      | -                     |
| 7.9594     | 98     | -             | 0.8266                 | 0.8256                 | 0.8240                 | 0.8147                 | 0.7904                |
| 8.1218     | 100    | 0.1374        | -                      | -                      | -                      | -                      | -                     |
| 8.9340     | 110    | 0.1272        | 0.8267                 | 0.8254                 | 0.8249                 | 0.8143                 | 0.7882                |
| 9.7462     | 120    | 0.1253        | 0.8263                 | 0.8269                 | 0.8245                 | 0.8143                 | 0.7898                |

* The bold row denotes the saved checkpoint.

### Framework Versions
- Python: 3.10.12
- Sentence Transformers: 3.3.1
- Transformers: 4.41.2
- PyTorch: 2.1.2+cu121
- Accelerate: 1.2.0
- Datasets: 2.19.1
- Tokenizers: 0.19.1

## Citation

### BibTeX

#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
```

#### MatryoshkaLoss
```bibtex
@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}
```

#### MultipleNegativesRankingLoss
```bibtex
@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->