sentence-transformers safetensors bert sentence-similarity feature-extraction generated_from_trainer dataset_size:6300 loss:MatryoshkaLoss loss:MultipleNegativesRankingLoss en arxiv:1908.10084 arxiv:2205.13147 arxiv:1705.00652 base_model:BAAI/bge-base-en-v1.5 base_model:finetune:BAAI/bge-base-en-v1.5 license:apache-2.0 autotrain_compatible text-embeddings-inference endpoints_compatible region:us
---
base_model: BAAI/bge-base-en-v1.5
language:
- en
library_name: sentence-transformers
license: apache-2.0
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:6300
- loss:MatryoshkaLoss
- loss:MultipleNegativesRankingLoss
widget:
- source_sentence: The example provided showcases a typical title or header used in
    a document, referred to as 'Item 8 Financial Statements and Supplementary Data
    39'.
  sentences:
  - What was the total unrecognized compensation expense related to nonvested restricted
    stock unit awards as of August 26, 2023?
  - What is the title illustration in the sample sentence?
  - What kind of information does Note 16 in the Consolidated Financial Statements
    cover in a Form 10-K?
- source_sentence: In 5,682 of our domestic stores as well as the vast majority of
    our stores in Mexico and Brazil, we had a commercial sales program that provided
    commercial credit and prompt delivery of parts and other products to local, regional,
    and national repair garages, dealers, service stations, fleet owners and other
    accounts.
  sentences:
  - What factors does the company consider in the valuation allowance assessment for
    deferred tax assets?
  - What is the primary focus of AutoZone's commercial sales program?
  - How much were the net cash flows from investing activities in 2023 and 2022?
- source_sentence: Revenues were $307.4 billion, an increase of 9% year over year.
  sentences:
  - What percentage increase in revenues did Alphabet Inc. report in 2023 compared
    to the previous year?
  - What are Level 1, Level 2, and Level 3 measurements in the fair value hierarchy?
  - What is the role of Kroger’s retail media business?
- source_sentence: Goodwill has been assigned to the reporting units based on prior
    business combinations related to the brands.
  sentences:
  - Where can one find the consolidated financial statements mentioned in an Annual
    Report on Form 10-K?
  - What type of revenue does 'Non-advertising revenue' include?
  - How is goodwill assigned to reporting units?
- source_sentence: We paid dividends on common shares and preferred shares of $8,136
    in 2023, compared with $9,859 in 2022.
  sentences:
  - How much were the dividends on common shares and preferred shares paid by AT&T
    Inc. in 2023?
  - What was the gross unrecognized tax benefit at the beginning of the year 2021?
  - What was the net change in Commercial and other receivables in 2023 compared to
    2022?
---

# BGE base Financial Matryoshka

This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) on the json dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) <!-- at revision a5beb1e3e68b9ab74eb54cfd186867f64f240e1a -->
- **Maximum Sequence Length:** 512 tokens
- **Output Dimensionality:** 768 dimensions
- **Similarity Function:** Cosine Similarity
- **Training Dataset:**
    - json
- **Language:** en
- **License:** apache-2.0

### Model Sources

- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)

### Full Model Architecture

```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)
```

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("wu-tz/bge-base-financial-matryoshka")
# Run inference
sentences = [
    'We paid dividends on common shares and preferred shares of $8,136 in 2023, compared with $9,859 in 2022.',
    'How much were the dividends on common shares and preferred shares paid by AT&T Inc. in 2023?',
    'What was the gross unrecognized tax benefit at the beginning of the year 2021?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```

<!--
### Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details>
-->

<!--
### Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details>
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Dataset

#### json

* Dataset: json
* Size: 6,300 training samples
* Columns: <code>positive</code> and <code>anchor</code>
* Approximate statistics based on the first 1000 samples:
  |         | positive                                                                           | anchor                                                                            |
  |:--------|:-----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|
  | type    | string                                                                             | string                                                                            |
  | details | <ul><li>min: 9 tokens</li><li>mean: 44.77 tokens</li><li>max: 215 tokens</li></ul> | <ul><li>min: 8 tokens</li><li>mean: 20.76 tokens</li><li>max: 43 tokens</li></ul> |
* Samples:
  | positive                                                                                                                                                                                                                                                                                                                                                                                                        | anchor                                                                                                                                 |
  |:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------|
  | <code>During the year ended December 31, 2023, Chipotle Mexican Grill, Inc. incurred a stock-based compensation expense of $126.7 million.</code>                                                                                                                                                                                                                                                               | <code>How much stock-based compensation expense did Chipotle Mexican Grill, Inc. incur during the year ended December 31, 2023?</code> |
  | <code>Depreciation and amortization totaled $4,856 as recorded in the financial statements.</code>                                                                                                                                                                                                                                                                                                              | <code>How much did depreciation and amortization total to in the financial statements?</code>                                          |
  | <code>On a segment basis, we expect: •Ford Pro EBIT of $8 billion to $9 billion driven by continued growth and favorable mix, partially offset by moderated pricing •Ford Blue EBIT of $7 billion to $7.5 billion, reflecting a balanced market equation, including the impact of our all-new F-150 launch; we also expect costs to be flat as we offset higher labor and product cost with efficiencies</code> | <code>What are the anticipated 2024 EBIT contributions for the Ford Pro and Ford Blue segments?</code>                                 |
* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
  ```json
  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          768,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  ```

### Framework Versions
- Python: 3.11.0rc1
- Sentence Transformers: 3.3.1
- Transformers: 4.41.2
- PyTorch: 2.1.2+cu121
- Accelerate: 0.31.0
- Datasets: 2.19.1
- Tokenizers: 0.19.0

## Citation

### BibTeX

#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
```

#### MatryoshkaLoss
```bibtex
@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}
```

#### MultipleNegativesRankingLoss
```bibtex
@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->