sentence-transformers safetensors bert sentence-similarity feature-extraction generated_from_trainer dataset_size:6300 loss:MatryoshkaLoss loss:MultipleNegativesRankingLoss en arxiv:1908.10084 arxiv:2205.13147 arxiv:1705.00652 base_model:BAAI/bge-base-en-v1.5 base_model:finetune:BAAI/bge-base-en-v1.5 license:apache-2.0 model-index autotrain_compatible text-embeddings-inference endpoints_compatible region:us
---
base_model: BAAI/bge-base-en-v1.5
datasets: []
language:
- en
library_name: sentence-transformers
license: apache-2.0
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:6300
- loss:MatryoshkaLoss
- loss:MultipleNegativesRankingLoss
widget:
- source_sentence: The sales contracts for Israel contain formulas that generally
    reflect an initial base price subject to price indexation, Brent-linked or other,
    over the life of the contract.
  sentences:
  - What was the change in HP's net deferred tax assets from 2022 to 2023?
  - What are the pricing mechanisms for crude oil sales contracts in Israel?
  - What was the total net income tax benefit HP received related to foreign tax audit
    matters?
- source_sentence: The FCA imposes severe penalties for the knowing and improper retention
    of overpayments from government programs. In addition, the defendant must follow
    certain notification and repayment processes within 60 days of identifying and
    quantifying an overpayment.
  sentences:
  - What does Note 21 pertain to in this report?
  - What types of penalties does the FCA impose for the knowing and improper retention
    of overpayments from government payors?
  - What impact did discrete tax items have on the tax provision in 2023 compared
    to 2022?
- source_sentence: The expected long-term rate of return is evaluated on an annual
    basis. We consider a number of factors when setting assumptions with respect to
    the long-term rate of return, including current and expected asset allocation
    and historical and expected returns on the plan asset categories. Actual asset
    allocations are regularly reviewed and periodically rebalanced to the targeted
    allocations when considered appropriate.
  sentences:
  - How is the expected long-term rate of return on plan assets determined?
  - What is the accumulated benefit obligation for AT&T's pension plans as of December
    31, 2023?
  - What is the management philosophy of Johnson & Johnson known as?
- source_sentence: The functional currency of our foreign entities is the currency
    of the primary economic environment in which the entity operates.
  sentences:
  - By what percent did Other Income (Expense) change in 2023 compared to 2022?
  - What are the Canadian class actions against Equifax seeking in relation to the
    2017 cybersecurity incident?
  - What is the functional currency for a company's foreign entities?
- source_sentence: Our products compete with other commercially available products
    based primarily on efficacy, safety, tolerability, acceptance by doctors, ease
    of patient compliance, ease of use, price, insurance and other reimbursement coverage,
    distribution and marketing.
  sentences:
  - What are the main factors influencing competition for the company's products?
  - What was the impact of restructuring charges in 2022 on the company and what changes
    occurred in 2023?
  - What are the penalties for non-compliance with Brazil's data protection laws?
model-index:
- name: BGE base Financial Matryoshka
  results:
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 768
      type: dim_768
    metrics:
    - type: cosine_accuracy@1
      value: 0.6985714285714286
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.83
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.88
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.9257142857142857
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.6985714285714286
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.27666666666666667
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.176
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.09257142857142854
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.6985714285714286
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.83
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.88
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.9257142857142857
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.8141629079228132
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.7782318594104309
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.7807867705374557
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 512
      type: dim_512
    metrics:
    - type: cosine_accuracy@1
      value: 0.7014285714285714
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.8328571428571429
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.8857142857142857
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.9228571428571428
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.7014285714285714
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.2776190476190476
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.17714285714285713
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.09228571428571428
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.7014285714285714
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.8328571428571429
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.8857142857142857
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.9228571428571428
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.8133531244983723
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.7781366213151925
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.7808747462599953
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 256
      type: dim_256
    metrics:
    - type: cosine_accuracy@1
      value: 0.7
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.84
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.8714285714285714
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.9085714285714286
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.7
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.28
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.17428571428571427
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.09085714285714284
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.7
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.84
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.8714285714285714
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.9085714285714286
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.8077154994184018
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.7749937641723353
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.7785241448057054
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 128
      type: dim_128
    metrics:
    - type: cosine_accuracy@1
      value: 0.6942857142857143
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.82
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.8557142857142858
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.9028571428571428
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.6942857142857143
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.2733333333333333
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.17114285714285712
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.09028571428571427
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.6942857142857143
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.82
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.8557142857142858
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.9028571428571428
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.7990640908671799
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.7658554421768706
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.7697199109144424
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 64
      type: dim_64
    metrics:
    - type: cosine_accuracy@1
      value: 0.6614285714285715
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.7842857142857143
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.8271428571428572
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.8885714285714286
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.6614285714285715
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.26142857142857145
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.1654285714285714
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.08885714285714284
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.6614285714285715
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.7842857142857143
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.8271428571428572
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.8885714285714286
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.7730930913085324
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.7365589569160996
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.7404183138657333
      name: Cosine Map@100
---

# BGE base Financial Matryoshka

This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) <!-- at revision a5beb1e3e68b9ab74eb54cfd186867f64f240e1a -->
- **Maximum Sequence Length:** 512 tokens
- **Output Dimensionality:** 768 tokens
- **Similarity Function:** Cosine Similarity
<!-- - **Training Dataset:** Unknown -->
- **Language:** en
- **License:** apache-2.0

### Model Sources

- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)

### Full Model Architecture

```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)
```

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("felipehsilveira/bge-base-financial-matryoshka")
# Run inference
sentences = [
    'Our products compete with other commercially available products based primarily on efficacy, safety, tolerability, acceptance by doctors, ease of patient compliance, ease of use, price, insurance and other reimbursement coverage, distribution and marketing.',
    "What are the main factors influencing competition for the company's products?",
    'What was the impact of restructuring charges in 2022 on the company and what changes occurred in 2023?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```

<!--
### Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details>
-->

<!--
### Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details>
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

## Evaluation

### Metrics

#### Information Retrieval
* Dataset: `dim_768`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)

| Metric              | Value      |
|:--------------------|:-----------|
| cosine_accuracy@1   | 0.6986     |
| cosine_accuracy@3   | 0.83       |
| cosine_accuracy@5   | 0.88       |
| cosine_accuracy@10  | 0.9257     |
| cosine_precision@1  | 0.6986     |
| cosine_precision@3  | 0.2767     |
| cosine_precision@5  | 0.176      |
| cosine_precision@10 | 0.0926     |
| cosine_recall@1     | 0.6986     |
| cosine_recall@3     | 0.83       |
| cosine_recall@5     | 0.88       |
| cosine_recall@10    | 0.9257     |
| cosine_ndcg@10      | 0.8142     |
| cosine_mrr@10       | 0.7782     |
| **cosine_map@100**  | **0.7808** |

#### Information Retrieval
* Dataset: `dim_512`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)

| Metric              | Value      |
|:--------------------|:-----------|
| cosine_accuracy@1   | 0.7014     |
| cosine_accuracy@3   | 0.8329     |
| cosine_accuracy@5   | 0.8857     |
| cosine_accuracy@10  | 0.9229     |
| cosine_precision@1  | 0.7014     |
| cosine_precision@3  | 0.2776     |
| cosine_precision@5  | 0.1771     |
| cosine_precision@10 | 0.0923     |
| cosine_recall@1     | 0.7014     |
| cosine_recall@3     | 0.8329     |
| cosine_recall@5     | 0.8857     |
| cosine_recall@10    | 0.9229     |
| cosine_ndcg@10      | 0.8134     |
| cosine_mrr@10       | 0.7781     |
| **cosine_map@100**  | **0.7809** |

#### Information Retrieval
* Dataset: `dim_256`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)

| Metric              | Value      |
|:--------------------|:-----------|
| cosine_accuracy@1   | 0.7        |
| cosine_accuracy@3   | 0.84       |
| cosine_accuracy@5   | 0.8714     |
| cosine_accuracy@10  | 0.9086     |
| cosine_precision@1  | 0.7        |
| cosine_precision@3  | 0.28       |
| cosine_precision@5  | 0.1743     |
| cosine_precision@10 | 0.0909     |
| cosine_recall@1     | 0.7        |
| cosine_recall@3     | 0.84       |
| cosine_recall@5     | 0.8714     |
| cosine_recall@10    | 0.9086     |
| cosine_ndcg@10      | 0.8077     |
| cosine_mrr@10       | 0.775      |
| **cosine_map@100**  | **0.7785** |

#### Information Retrieval
* Dataset: `dim_128`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)

| Metric              | Value      |
|:--------------------|:-----------|
| cosine_accuracy@1   | 0.6943     |
| cosine_accuracy@3   | 0.82       |
| cosine_accuracy@5   | 0.8557     |
| cosine_accuracy@10  | 0.9029     |
| cosine_precision@1  | 0.6943     |
| cosine_precision@3  | 0.2733     |
| cosine_precision@5  | 0.1711     |
| cosine_precision@10 | 0.0903     |
| cosine_recall@1     | 0.6943     |
| cosine_recall@3     | 0.82       |
| cosine_recall@5     | 0.8557     |
| cosine_recall@10    | 0.9029     |
| cosine_ndcg@10      | 0.7991     |
| cosine_mrr@10       | 0.7659     |
| **cosine_map@100**  | **0.7697** |

#### Information Retrieval
* Dataset: `dim_64`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)

| Metric              | Value      |
|:--------------------|:-----------|
| cosine_accuracy@1   | 0.6614     |
| cosine_accuracy@3   | 0.7843     |
| cosine_accuracy@5   | 0.8271     |
| cosine_accuracy@10  | 0.8886     |
| cosine_precision@1  | 0.6614     |
| cosine_precision@3  | 0.2614     |
| cosine_precision@5  | 0.1654     |
| cosine_precision@10 | 0.0889     |
| cosine_recall@1     | 0.6614     |
| cosine_recall@3     | 0.7843     |
| cosine_recall@5     | 0.8271     |
| cosine_recall@10    | 0.8886     |
| cosine_ndcg@10      | 0.7731     |
| cosine_mrr@10       | 0.7366     |
| **cosine_map@100**  | **0.7404** |

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Dataset

#### Unnamed Dataset


* Size: 6,300 training samples
* Columns: <code>positive</code> and <code>anchor</code>
* Approximate statistics based on the first 1000 samples:
  |         | positive                                                                           | anchor                                                                           |
  |:--------|:-----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|
  | type    | string                                                                             | string                                                                           |
  | details | <ul><li>min: 6 tokens</li><li>mean: 45.44 tokens</li><li>max: 301 tokens</li></ul> | <ul><li>min: 7 tokens</li><li>mean: 20.3 tokens</li><li>max: 51 tokens</li></ul> |
* Samples:
  | positive                                                                                                                                                                                                                                                                                   | anchor                                                                                                                             |
  |:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------|
  | <code>The Centers for Medicare & Medicaid Services issued a final rule in October 2023 for the calendar year 2024, estimating a productivity-adjusted market basket increase of 2.1% in average reimbursement to ESRD facilities.</code>                                                   | <code>What is the projected impact on average reimbursement to ESRD facilities in 2024 due to the final rule issued by CMS?</code> |
  | <code>Company Adjusted EBIT Margin is derived by dividing the Company adjusted EBIT by Company revenue, which is a non-GAAP measure useful for evaluating the company's operating results.</code>                                                                                          | <code>How is the Company Adjusted EBIT Margin calculated?</code>                                                                   |
  | <code>The provision for credit losses was $4 million for the year ended December 31, 202 serviLists of account holders responsible for and the state of the economy, our credit standards, our risk assessments, and the judgment of our employees responsible for granting credit.</code> | <code>What factors influence the provision for credit losses at Las Vegas Sands Corp.?</code>                                      |
* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
  ```json
  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          768,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  ```

### Training Hyperparameters
#### Non-Default Hyperparameters

- `eval_strategy`: epoch
- `per_device_train_batch_size`: 32
- `per_device_eval_batch_size`: 16
- `gradient_accumulation_steps`: 16
- `learning_rate`: 2e-05
- `num_train_epochs`: 4
- `lr_scheduler_type`: cosine
- `warmup_ratio`: 0.1
- `bf16`: True
- `tf32`: True
- `load_best_model_at_end`: True
- `optim`: adamw_torch_fused
- `batch_sampler`: no_duplicates

#### All Hyperparameters
<details><summary>Click to expand</summary>

- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: epoch
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 32
- `per_device_eval_batch_size`: 16
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 16
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 2e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 4
- `max_steps`: -1
- `lr_scheduler_type`: cosine
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.1
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: True
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: True
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: True
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch_fused
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: False
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`: 
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `eval_use_gather_object`: False
- `batch_sampler`: no_duplicates
- `multi_dataset_batch_sampler`: proportional

</details>

### Training Logs
| Epoch      | Step   | Training Loss | dim_128_cosine_map@100 | dim_256_cosine_map@100 | dim_512_cosine_map@100 | dim_64_cosine_map@100 | dim_768_cosine_map@100 |
|:----------:|:------:|:-------------:|:----------------------:|:----------------------:|:----------------------:|:---------------------:|:----------------------:|
| 0.8122     | 10     | 1.5176        | -                      | -                      | -                      | -                     | -                      |
| 0.9746     | 12     | -             | 0.7500                 | 0.7642                 | 0.7680                 | 0.7079                | 0.7708                 |
| 1.6244     | 20     | 0.6868        | -                      | -                      | -                      | -                     | -                      |
| 1.9492     | 24     | -             | 0.7657                 | 0.7746                 | 0.7784                 | 0.7323                | 0.7816                 |
| 2.4365     | 30     | 0.4738        | -                      | -                      | -                      | -                     | -                      |
| 2.9239     | 36     | -             | 0.7691                 | 0.7780                 | 0.7790                 | 0.7402                | 0.7796                 |
| 3.2487     | 40     | 0.3934        | -                      | -                      | -                      | -                     | -                      |
| **3.8985** | **48** | **-**         | **0.7697**             | **0.7785**             | **0.7809**             | **0.7404**            | **0.7808**             |

* The bold row denotes the saved checkpoint.

### Framework Versions
- Python: 3.11.9
- Sentence Transformers: 3.0.1
- Transformers: 4.44.2
- PyTorch: 2.4.0+cu121
- Accelerate: 0.33.0
- Datasets: 2.21.0
- Tokenizers: 0.19.1

## Citation

### BibTeX

#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
```

#### MatryoshkaLoss
```bibtex
@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning}, 
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}
```

#### MultipleNegativesRankingLoss
```bibtex
@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply}, 
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->