sentence-transformers safetensors mpnet sentence-similarity feature-extraction generated_from_trainer dataset_size:53224 loss:MultipleNegativesRankingLoss en dataset:sentence-transformers/coliee dataset:bwang0911/legal_qa_v1 dataset:bwang0911/law_stackexchange dataset:bwang0911/legal_lens_nli dataset:bwang0911/cuad_qa dataset:bwang0911/privacy_qa dataset:bwang0911/legal_case_summarization dataset:bwang0911/aus_legal_qa arxiv:1908.10084 arxiv:1705.00652 base_model:sentence-transformers/all-mpnet-base-v2 base_model:finetune:sentence-transformers/all-mpnet-base-v2 model-index autotrain_compatible endpoints_compatible region:us
---
base_model: sentence-transformers/all-mpnet-base-v2
datasets:
- sentence-transformers/coliee
- bwang0911/legal_qa_v1
- bwang0911/law_stackexchange
- bwang0911/legal_lens_nli
- bwang0911/cuad_qa
- bwang0911/privacy_qa
- bwang0911/legal_case_summarization
- bwang0911/aus_legal_qa
language:
- en
library_name: sentence-transformers
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:53224
- loss:MultipleNegativesRankingLoss
widget:
- source_sentence: ' A juridical person may not be a partner of a civil law union. '
  sentences:
  - '

    Article 34

    An not-for-profit association or foundation that is involved in academic activities,
    art, charity, worship, religion, or any other matter of public interest may be
    established as a juridical person with the permission of the competent government
    agency..

    '
  - '

    Article 192

    A person that commences the possession of movables peacefully and openly by a
    transactional act acquires the rights that are exercised with respect to the movables
    immediately if the person possesses it in good faith and without negligence..

    '
  - '

    Article 550

    Gifts not in writing may be cancelled by either party;provided, however, that
    this does not apply to a portion of the gift for which performance has been completed..

    '
- source_sentence: are there any legal systems in the world where the judiciary and
    the legislature are the same? it's well known that the limited lawmaking power
    that a judiciary has comes from interpretation of statutes but are there any systems
    where the judiciary and the legislature are one and the same and have broad lawmaking
    powers ?
  sentences:
  - 'Short Answer Is it illegal for US citizens to travel to North Korea? Yes (but
    see the &quot;fine print&quot; below). Long Answer There is: a US travel ban to
    North Korea for American citizens, as of July 2017. Now, Americans wishing to
    travel to North Korea must obtain a Special Validation Passport from the US Department
    of State, only issued under very specific circumstances, such as for journalists
    covering the region or for humanitarian aid workers. The Biden administration
    extended the ban, initially established by the Trump administration, on traveling
    to North Korea on a U.S. passport absent special approval: The ban makes it illegal
    to use a U.S. passport for travel to, from or through North Korea, also known
    as the Democratic People''s Republic of Korea, or the DPRK, unless the document
    has been specially validated. Such validations are granted by the State Department
    only in the case of compelling national interest. The U.S. State Department confirms
    that this ban is still in place. It states that: Travel to, in, or through North
    Korea on a U.S. passport without this special validation may justify revocation
    of your passport for misuse under 22 C.F.R. § 51.62(a)(3) and may subject you
    to felony prosecution under 18 U.S.C. § 1544 or other applicable laws. The maximum
    criminal penalty if you use a U.S. passport to go to North Korea and then return
    and a charged with a crime under 18 U.S.C. § 1544 are quite serious. You could
    be sent to prison for up to ten years for a first or second offense, or up to
    fifteen years if you have two prior convictions under this statute, and/or fined,
    even if you weren''t a terrorist or drug dealer, although the actual sentence
    would probably be milder, if you were charged with a crime at all. The criminal
    statute reads as follows (with the pertinent parts in bold): Whoever willfully
    and knowingly uses, or attempts to use, any passport issued or designed for the
    use of another; or Whoever willfully and knowingly uses or attempts to use any
    passport in violation of the conditions or restrictions therein contained, or
    of the rules prescribed pursuant to the laws regulating the issuance of passports;
    or Whoever willfully and knowingly furnishes, disposes of, or delivers a passport
    to any person, for use by another than the person for whose use it was originally
    issued and designed— Shall be fined under this title, imprisoned not more than
    25 years (if the offense was committed to facilitate an act of international terrorism
    (as defined in section 2331 of this title)), 20 years (if the offense was committed
    to facilitate a drug trafficking crime (as defined in section 929(a) of this title)),
    10 years (in the case of the first or second such offense, if the offense was
    not committed to facilitate such an act of international terrorism or a drug trafficking
    crime), or 15 years (in the case of any other offense), or both. There are also
    many other North Korean sanctions (and keep in mind that North Korea is legally
    an &quot;enemy&quot; of the United States with which the U.S. is officially still
    at war and does not have diplomatic relations). The most recent of those, from
    2017, prohibits ships and aircraft owned by a &quot;foreign person&quot; which
    have been in North Korean in the last 180 days from entering the United States.
    The ban does not prohibit a dual citizen from traveling to North Korea on a passport
    from the person''s other country of citizenship, nor does it prohibit U.S. citizens
    from entering North Korea without using a passport (although entering North Korea
    without a passport or visa probably violates North Korean law). Of course, North
    Korea also regulates entry of people into North Korea under North Korean immigration
    laws. I do not know whether or not it is legal under North Korean law for people
    to enter it with a U.S. passport. But, given that the only U.S. citizen to enter
    North Korea without a special U.S. visa authorizing the trip in the last seven
    years was arrested immediately after crossing into North Korea this week, it would
    appear that this is illegal under North Korean law as well.'
  - Historically, this was true in the Icelandic Commonwealth in the Middle Ages,
    and in some democratic Greek city-states in the classical era. Similarly, in non-democratic
    feudal regimes, the lord or monarch was both the law giver and sitting in court
    was also the arbiter of all disputes arising under the lord's own laws. In places
    like Saudi Arabia where the monarchy's power is more than symbolic, the system
    still works this way to a significant extent. The practical reality in most one
    party Communist states is similar. In the United Kingdom, historically, the Appellate
    committee of the House of Lords (staffed by a subset of aristocrats usually appointed
    for life by the Prime Minister to the post) was the highest court of appeal of
    other courts in the British Commonwealth (with the Judicial committee of the Privy
    Council handling final appeals from outside Britain), and it was also a court
    of original jurisdiction for certain criminal cases against other aristocrats
    to satisfy the Magna Carta's notion that one is entitled to a jury of one's peers.
    Top level general purpose legislatures rarely serve as courts at the highest level,
    except in very isolated political matters. A good example of narrow quasi-judicial
    legislative power is the power of the Congress in the U.S., to be the ultimate
    judge for Congressional election disputes and of some Presidential election disputes.
    Congress also has quasi-judicial jurisdiction over impeachments of government
    employees whether or not they are elected, and over expulsions for cause of its
    own members and over other ethical sanctions of its own members. Many other legislatures
    have some sort of quansi-judicial impeachment and/or explusion power exercised
    as a whole by by some committee within it. It is common in the United States for
    administrative agencies, within their narrow area of competence to exercise both
    quasi-legislative power to enact regulations with a broad mandate in a subject
    area, and also to have quasi-judicial power in that same subject area. The Securities
    and Exchange Commission, the National Labor Relations Board, the Internal Revenue
    Service, the Environmental Protection Agency, and the Merit System Protection
    Board, for example, all operate in this fashion to some extent. Likewise, it is
    very common at the local government level for a city council and its planning
    board to carry out both legislative roles and quasi-judicial role when disputes
    come up regarding its land use regulations. Similarly, school boards routinely
    both establish employment regulations and other school rules, and serve in a quasi-judicial
    role with respect employee discipline or termination, and with respect to student
    discipline. This dual role is also common for the boards of other public institutions
    like hospitals and state colleges, and for private non-profit organizations. A
    recent example in that kind of situation is Colorado's State School Board which
    both exercises legislative power over when charter schools (i.e. public schools
    not under the direct supervision of any elected local school board) may be formed,
    and has the ultimate and final judicial review role over decisions by local school
    boards to grant or deny school charters.
  - It isn't explicitly prohibited so long as the amount claimed is in the aggregate
    less than $20,000. But, it would probably be better to file separately. First,
    very simple single party, single transaction cases are what small claims court
    is designed to do, and going against the flow often creates unforeseen confusion
    for the judge in the Justice Court who isn't a sophisticated civil litigation
    expert. The Justices of the Peace who preside over Justice Courts that handle
    small claims cases in Texas often aren't and don't have to be lawyers or even
    high school graduates. Second, if you sue as a group, and one of your group is
    the lead person handling the case (and that person isn't a lawyer), the lead person
    is at grave risk of being found to be practicing law without a license by taking
    actions in a lawsuit on behalf of your fellow plaintiffs.
- source_sentence: 'Q: Myself & spouse have lived in home 27 years. If I file for
    divorce will he be made to sell home in St Pete and split the. Home in his name
    only. Do not trust him! '
  sentences:
  - A:As long as the house is a marital asset, which it sounds like it is, the court
    will order the equity to be divided as part of equitable distribution. You will
    each have the opportunity to buy the other out or else the property will be sold.
    Speak with a local family lawyer for more specific advice.
  - A:Hi there, good evening. In federal cases, such as yours in the Central District
    Court of California, once a case is e-filed, the documents, including the complaint
    and summons, can typically be downloaded from PACER. This is a common practice
    and allows for easy access to filed documents. A "conformed copy" of a document
    is essentially a copy that includes all signatures and official stamps, making
    it identical to the original. These copies are often required in situations where
    you need to submit a document that is as valid as the original, such as for certain
    legal or official proceedings. They ensure that the document you're using is a
    true and complete representation of the original filed document.
  - A:First off, they read him his rights once. Secondly, not reading your rights
    does not mean not guilty automatically, If Miranda is violated it only suppresses
    any statements made. He was not charged with underage drinking, curfew, truancy,
    or running away. He was charged with a criminal offense, DUI. Time to lawyer up.
    If he is convicted of DUI, he loses his license for a minimum of two years and
    then must have a hearing with the Secretary of State.
- source_sentence: how private is this app?
  sentences:
  - We will not use this information for anything other than providing the Service
    for which the information was supplied.
  - ), and College Board program participants may provide information regarding study
    habits and test scores (e.g., the number of hours studied, modules or tests taken,
    scores earned, etc.
  - We share your information with Service Providers who process data on our behalf,
    such as credit card processors and customer management systems.
- source_sentence: does this app may share my location anonymous?
  sentences:
  - You may opt out of certain ad targeting and retargeting services by visiting the
    Digital Advertising Alliances opt-out page, or the Network Advertising Initiatives
    opt-out page.
  - 'Delivery of location services will involve reference to one or more of the following:
    (a) the coordinates (latitude/longitude) of your location; (b) look-up of your
    country of location by reference to your IP address against public sources; and/or
    (c) your location settings on your Apple device or Android device, or similar
    device identifier/settings.'
  - We may collect usage information about your use of our Service, such as the number
    of problems you have attempted, the number of videos you have viewed, and the
    amount of time spent to complete a problem.
model-index:
- name: SentenceTransformer based on sentence-transformers/all-mpnet-base-v2
  results:
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: mteb/AILA casedocs
      type: mteb/AILA_casedocs
    metrics:
    - type: cosine_accuracy@1
      value: 0.24
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.4
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.44
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.5
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.24
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.2
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.14400000000000002
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.096
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.06261421911421912
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.1773951048951049
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.21672843822843824
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.28030419580419585
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.23571318760075094
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.32385714285714284
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.19099315576955767
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: mteb/AILA statutes
      type: mteb/AILA_statutes
    metrics:
    - type: cosine_accuracy@1
      value: 0.24
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.52
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.72
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.8
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.24
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.20666666666666667
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.19999999999999996
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.144
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.068
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.16066666666666665
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.25033333333333335
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.35100000000000003
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.2945290400206784
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.4145238095238095
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.23863257355862635
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: mteb/legalbench consumer contracts qa
      type: mteb/legalbench_consumer_contracts_qa
    metrics:
    - type: cosine_accuracy@1
      value: 0.48737373737373735
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.6515151515151515
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.73989898989899
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.8560606060606061
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.48737373737373735
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.21717171717171713
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.14797979797979796
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.0856060606060606
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.48737373737373735
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.6515151515151515
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.73989898989899
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.8560606060606061
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.6575720798646046
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.5956780102613435
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.6021553873830202
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: mteb/legalbench corporate lobbying
      type: mteb/legalbench_corporate_lobbying
    metrics:
    - type: cosine_accuracy@1
      value: 0.788235294117647
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.9205882352941176
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.9382352941176471
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.9588235294117647
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.788235294117647
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.3068627450980392
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.1876470588235294
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.09588235294117646
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.788235294117647
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.9205882352941176
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.9382352941176471
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.9588235294117647
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.8823720261303867
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.8569596171802053
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.8589677781368958
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: mteb/legal summarization
      type: mteb/legal_summarization
    metrics:
    - type: cosine_accuracy@1
      value: 0.4788732394366197
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.6373239436619719
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.721830985915493
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.8204225352112676
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.4788732394366197
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.23474178403755866
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.16830985915492958
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.1028169014084507
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.4233891988293397
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.5632004146088653
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.6415233827205657
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.7539452624839948
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.602922176130265
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.5816705790297337
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.5513678334926079
      name: Cosine Map@100
---

# SentenceTransformer based on sentence-transformers/all-mpnet-base-v2

This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2) on the [coliee](https://huggingface.co/datasets/sentence-transformers/coliee), [legal_qa](https://huggingface.co/datasets/bwang0911/legal_qa_v1), [law_stack](https://huggingface.co/datasets/bwang0911/law_stackexchange), [legal_lens](https://huggingface.co/datasets/bwang0911/legal_lens_nli), [cuad_qa](https://huggingface.co/datasets/bwang0911/cuad_qa), [privacy_qa](https://huggingface.co/datasets/bwang0911/privacy_qa), [legal_sum](https://huggingface.co/datasets/bwang0911/legal_case_summarization) and [aus_legal_qa](https://huggingface.co/datasets/bwang0911/aus_legal_qa) datasets. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2) <!-- at revision 9a3225965996d404b775526de6dbfe85d3368642 -->
- **Maximum Sequence Length:** 192 tokens
- **Output Dimensionality:** 768 dimensions
- **Similarity Function:** Cosine Similarity
- **Training Datasets:**
    - [coliee](https://huggingface.co/datasets/sentence-transformers/coliee)
    - [legal_qa](https://huggingface.co/datasets/bwang0911/legal_qa_v1)
    - [law_stack](https://huggingface.co/datasets/bwang0911/law_stackexchange)
    - [legal_lens](https://huggingface.co/datasets/bwang0911/legal_lens_nli)
    - [cuad_qa](https://huggingface.co/datasets/bwang0911/cuad_qa)
    - [privacy_qa](https://huggingface.co/datasets/bwang0911/privacy_qa)
    - [legal_sum](https://huggingface.co/datasets/bwang0911/legal_case_summarization)
    - [aus_legal_qa](https://huggingface.co/datasets/bwang0911/aus_legal_qa)
- **Language:** en
<!-- - **License:** Unknown -->

### Model Sources

- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)

### Full Model Architecture

```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 192, 'do_lower_case': False}) with Transformer model: MPNetModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)
```

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("bwang0911/jev2-legal")
# Run inference
sentences = [
    'does this app may share my location anonymous?',
    'Delivery of location services will involve reference to one or more of the following: (a) the coordinates (latitude/longitude) of your location; (b) look-up of your country of location by reference to your IP address against public sources; and/or (c) your location settings on your Apple device or Android device, or similar device identifier/settings.',
    'We may collect usage information about your use of our Service, such as the number of problems you have attempted, the number of videos you have viewed, and the amount of time spent to complete a problem.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```

<!--
### Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details>
-->

<!--
### Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details>
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

## Evaluation

### Metrics

#### Information Retrieval

* Datasets: `mteb/AILA_casedocs`, `mteb/AILA_statutes`, `mteb/legalbench_consumer_contracts_qa`, `mteb/legalbench_corporate_lobbying` and `mteb/legal_summarization`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)

| Metric              | mteb/AILA_casedocs | mteb/AILA_statutes | mteb/legalbench_consumer_contracts_qa | mteb/legalbench_corporate_lobbying | mteb/legal_summarization |
|:--------------------|:-------------------|:-------------------|:--------------------------------------|:-----------------------------------|:-------------------------|
| cosine_accuracy@1   | 0.24               | 0.24               | 0.4874                                | 0.7882                             | 0.4789                   |
| cosine_accuracy@3   | 0.4                | 0.52               | 0.6515                                | 0.9206                             | 0.6373                   |
| cosine_accuracy@5   | 0.44               | 0.72               | 0.7399                                | 0.9382                             | 0.7218                   |
| cosine_accuracy@10  | 0.5                | 0.8                | 0.8561                                | 0.9588                             | 0.8204                   |
| cosine_precision@1  | 0.24               | 0.24               | 0.4874                                | 0.7882                             | 0.4789                   |
| cosine_precision@3  | 0.2                | 0.2067             | 0.2172                                | 0.3069                             | 0.2347                   |
| cosine_precision@5  | 0.144              | 0.2                | 0.148                                 | 0.1876                             | 0.1683                   |
| cosine_precision@10 | 0.096              | 0.144              | 0.0856                                | 0.0959                             | 0.1028                   |
| cosine_recall@1     | 0.0626             | 0.068              | 0.4874                                | 0.7882                             | 0.4234                   |
| cosine_recall@3     | 0.1774             | 0.1607             | 0.6515                                | 0.9206                             | 0.5632                   |
| cosine_recall@5     | 0.2167             | 0.2503             | 0.7399                                | 0.9382                             | 0.6415                   |
| cosine_recall@10    | 0.2803             | 0.351              | 0.8561                                | 0.9588                             | 0.7539                   |
| **cosine_ndcg@10**  | **0.2357**         | **0.2945**         | **0.6576**                            | **0.8824**                         | **0.6029**               |
| cosine_mrr@10       | 0.3239             | 0.4145             | 0.5957                                | 0.857                              | 0.5817                   |
| cosine_map@100      | 0.191              | 0.2386             | 0.6022                                | 0.859                              | 0.5514                   |

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Datasets

#### coliee

* Dataset: [coliee](https://huggingface.co/datasets/sentence-transformers/coliee) at [d90012e](https://huggingface.co/datasets/sentence-transformers/coliee/tree/d90012e1f3a0d7103713bb2ce7faed1636a10090)
* Size: 9,260 training samples
* Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>
* Approximate statistics based on the first 1000 samples:
  |         | anchor                                                                             | positive                                                                            | negative                                                                             |
  |:--------|:-----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|
  | type    | string                                                                             | string                                                                              | string                                                                               |
  | details | <ul><li>min: 11 tokens</li><li>mean: 41.76 tokens</li><li>max: 99 tokens</li></ul> | <ul><li>min: 25 tokens</li><li>mean: 119.1 tokens</li><li>max: 192 tokens</li></ul> | <ul><li>min: 14 tokens</li><li>mean: 113.91 tokens</li><li>max: 192 tokens</li></ul> |
* Samples:
  | anchor                                                                                                             | positive                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   | negative                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
  |:-------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
  | <code><br>A compulsory auction is also a sale, so warranty is imposed the same as for an ordinary sale.<br></code> | <code><br>Article 568<br>(1) The successful bidder at an auction based on the provisions of the Civil Execution Act and other laws (hereinafter referred to as an "auction" in this Article) may cancel the contract or demand a reduction of the price against the obligor pursuant to the provisions of Articles 541 and 542 and the provisions of Article 563<br>(including as applied mutatis mutandis pursuant to Article 565).<br>(2) In the cases referred to in the preceding paragraph, if the obligor is insolvent, the successful bidder may demand total or partial reimbursement of the proceeds against the obligees that received the distribution of the proceeds.<br>(3) In the cases set forth in the preceding two paragraphs, if obligors knew of the absence of the object or right and did not disclose the same, or if obligees knew of the absence but demanded an auction, the successful bidder may claim compensation for loss or damage against those persons.<br>(4) The provisions of the preceding three paragraphs do not apply ...</code> | <code><br>Article 575<br>(1) If the subject matter of a sale which has not yet been delivered bears fruits, the fruits vest in the seller.<br>(2) The buyer bears the obligation to pay interest on the price beginning from the day of delivery;provided, however, that if a due date is provided for the payment of the price, it is not necessary to pay the interest until that due date arrives..<br></code>                                                                                      |
  | <code><br>A compulsory auction is also a sale, so warranty is imposed the same as for an ordinary sale.<br></code> | <code><br>Article 568<br>(1) The successful bidder at an auction based on the provisions of the Civil Execution Act and other laws (hereinafter referred to as an "auction" in this Article) may cancel the contract or demand a reduction of the price against the obligor pursuant to the provisions of Articles 541 and 542 and the provisions of Article 563<br>(including as applied mutatis mutandis pursuant to Article 565).<br>(2) In the cases referred to in the preceding paragraph, if the obligor is insolvent, the successful bidder may demand total or partial reimbursement of the proceeds against the obligees that received the distribution of the proceeds.<br>(3) In the cases set forth in the preceding two paragraphs, if obligors knew of the absence of the object or right and did not disclose the same, or if obligees knew of the absence but demanded an auction, the successful bidder may claim compensation for loss or damage against those persons.<br>(4) The provisions of the preceding three paragraphs do not apply ...</code> | <code><br>Article 596<br>The provisions of Article 551<br>apply mutatis mutandis to loans for use.<br>Article 551<br>(1) The donor is presumed to have promised to deliver or transfer the thing or right that is the subject matter of the gift, while maintaining its condition as of the time when it is specified as the subject matter of the gift.<br>(2) With respect to gifts with burden, the donor provides the same warranty as that of a seller, to the extent of that burden..<br></code> |
  | <code><br>A compulsory auction is also a sale, so warranty is imposed the same as for an ordinary sale.<br></code> | <code><br>Article 568<br>(1) The successful bidder at an auction based on the provisions of the Civil Execution Act and other laws (hereinafter referred to as an "auction" in this Article) may cancel the contract or demand a reduction of the price against the obligor pursuant to the provisions of Articles 541 and 542 and the provisions of Article 563<br>(including as applied mutatis mutandis pursuant to Article 565).<br>(2) In the cases referred to in the preceding paragraph, if the obligor is insolvent, the successful bidder may demand total or partial reimbursement of the proceeds against the obligees that received the distribution of the proceeds.<br>(3) In the cases set forth in the preceding two paragraphs, if obligors knew of the absence of the object or right and did not disclose the same, or if obligees knew of the absence but demanded an auction, the successful bidder may claim compensation for loss or damage against those persons.<br>(4) The provisions of the preceding three paragraphs do not apply ...</code> | <code><br>Article 520<br>If a claim and obligation becomes vested in the same person, such claim is extinguished;provided, however, that this does not apply if such a claim is the subject matter of the right of a third party..<br></code>                                                                                                                                                                                                                                                          |
* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
  ```json
  {
      "scale": 50.0,
      "similarity_fct": "cos_sim"
  }
  ```

#### legal_qa

* Dataset: [legal_qa](https://huggingface.co/datasets/bwang0911/legal_qa_v1) at [bbe3790](https://huggingface.co/datasets/bwang0911/legal_qa_v1/tree/bbe3790626658e8e020de978d186c8902647b635)
* Size: 3,742 training samples
* Columns: <code>anchor</code> and <code>positive</code>
* Approximate statistics based on the first 1000 samples:
  |         | anchor                                                                               | positive                                                                            |
  |:--------|:-------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
  | type    | string                                                                               | string                                                                              |
  | details | <ul><li>min: 13 tokens</li><li>mean: 108.12 tokens</li><li>max: 192 tokens</li></ul> | <ul><li>min: 6 tokens</li><li>mean: 130.94 tokens</li><li>max: 192 tokens</li></ul> |
* Samples:
  | anchor                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | positive                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
  |:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
  | <code>Q: I was wondering if a pain management office is acting illegally/did an illegal action.. I was discharged as a patient from a pain management office after them telling me that a previous pain management specialist I saw administered a steroid shot wrong and I told them in the portal that I spoke to lawyers for advice but no lawsuit/case was created. It was maybe 1-2 months after I was discharged that I no longer have access to my patient portal with them. Every time I try to login I enter my credentials, wait a few seconds, and then I get re-directed back to the original screen where I have various options to login. I know I can speak to the office directly and ask them about what specifically is going on, talk to other lawyers if this is a violation of my rights, etc. but I was just wondering if anyone on this site would know if this action is in fact illegal. </code>         | <code>A:In Kentucky, your situation raises questions about patient rights and medical records access. If you were discharged from a pain management office and subsequently lost access to your patient portal, it's important to understand your rights regarding medical records. Under the Health Insurance Portability and Accountability Act (HIPAA), you have the right to access your own medical records. A healthcare provider cannot deny you access to your records, even if you were discharged from their practice. The issue with the patient portal could be a technical glitch or a deliberate action, but either way, it needs to be addressed. You should contact the pain management office directly to inquire about the issue with the patient portal. They are obligated to provide you with a way to access your medical records, which may include providing paper copies or access through a different electronic means. If the office is unresponsive or refuses to provide access to your records, consider speakin...</code> |
  | <code>Q: My houseboat was moved while I was at work and was asked to leave the property after 16 months without any notices.. Management told me that he didn’t like those kind of people that came to visit me. I went on vacation and when I came back my electrical cord was missing and my water hose was cut in pieces. He than got of hold of me and said I have your boat and that he was putting a lien on it for back rent. I never received any notices from management. He than told me that he wanted me to leave the property and that he would hold my boat until i found another parking spot for it. I finally found a parking spot a year later and I find out that he liened my houseboat and was selling it and told me to sue him and gave me his lawyers contact. What options do I have. I now am homeless and I lost my job and seeing myself falling as time goes. Any help would be appreciated. </code> | <code>A:If your houseboat was moved without your permission and you were asked to leave the property after 16 months without any notices, and the management has placed a lien on your houseboat and is attempting to sell it, you may have legal options to challenge their actions. First, you may want to consult with an attorney who specializes in property law or landlord-tenant law to review your case and provide guidance on your legal rights and options. It may also be helpful to gather any evidence you have, such as receipts or documentation of payments or communications with the management, to support your case. Some potential legal claims you may have against the management include breach of contract, conversion, or wrongful eviction. These claims may allow you to seek damages or other forms of relief, such as the return of your houseboat or compensation for any losses you have suffered. It's important to act quickly, as there may be deadlines or other requirements for filing a legal claim. ...</code> |
  | <code>Q: Wondering if I could sue a dealership if I’ve had to replace my transmission 2 times within a year. I’ve had to replace my transmission now 2 different times in 10 months of owning my 2018 Ford Expedition. My first replacement was $5,200 and then 1 1/2 months later the transmission went out and now they are working on it again </code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         | <code>A:Yes, you can sue a dealership if you have had to replace your transmission 2 times within a year. Whether you will be successful depends on the facts and your presentation. You don't mention anything about any written agreements. Allowing an attorney to evaluate, organize and draft your conciliation or (small claims) complaint (up to $15,000) would be a wise investment. A well polished complaint will not only provide the judge a solid foundation to rule in your favor, but a detailed outline on which to base your oral argument. It would provide you a distinct advantage over the opposing party. The fees can be reasonable depending on the attorney.</code>                                                                                                                                                                                                                                                                                                                                                             |
* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
  ```json
  {
      "scale": 50.0,
      "similarity_fct": "cos_sim"
  }
  ```

#### law_stack

* Dataset: [law_stack](https://huggingface.co/datasets/bwang0911/law_stackexchange) at [b19a371](https://huggingface.co/datasets/bwang0911/law_stackexchange/tree/b19a37105babf2f9b5e3aa93dbc65037fbdfd0e0)
* Size: 13,000 training samples
* Columns: <code>anchor</code> and <code>positive</code>
* Approximate statistics based on the first 1000 samples:
  |         | anchor                                                                               | positive                                                                             |
  |:--------|:-------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|
  | type    | string                                                                               | string                                                                               |
  | details | <ul><li>min: 27 tokens</li><li>mean: 141.93 tokens</li><li>max: 192 tokens</li></ul> | <ul><li>min: 15 tokens</li><li>mean: 166.18 tokens</li><li>max: 192 tokens</li></ul> |
* Samples:
  | anchor                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   | positive                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
  |:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
  | <code>Why is drunk driving causing accident punished so much worse than just drunk driving? When people drink and drive and then cause an accident especially where if someone dies they get years and years in prison but just the act of drunk driving is punished way more lenient. Shouldn't the 2, drunk driving and drunk driving then causing accident be similarly punished? I feel like a lot of times it's luck whether an accident happens.</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | <code>Moral luck You have raised the issue of moral luck, a long recognized problem in criminal theory. The classic expositions of this issue are by Thomas Nagel, in his chapter, &quot;Moral Luck&quot; (1979) and Bernard Williams, &quot;Moral Luck&quot; (1976). Specifically, you are describing what they call outcome luck, or consequential luck. Driving while intoxicated vs. driving while intoxicated and causing death is not the only example where moral luck results in a distinction in punishment. Other examples are: dangerous driving vs. dangerous driving that causes death a successful offence vs. an attempted offence (generally resulting in a maximum sentence less than that of the successful offence) Nagel writes: If someone has had too much to drink and his car swerves on to the sidewalk, he can count himself morally lucky if there are no pedestrians in its path. If there were, he would be to blame for their deaths, and would probably be prosecuted for manslaughter. But if he hurts no one,...</code> |
  | <code>Question Concerning Responding to Employer of Minor Daughter Paid Under Minimum Wage My high school daughter worked for about a year for an employer who owns a tutoring company in our town. Due to friction between my daughter and the employer, my daughter recently quit but she realized that she was being underpaid for much of this year (2023) because the minimum wage here in California is currently $15.50 for 2023 but she was still getting paid $14.00, the California minimum wage for 2022, when she recently quit (in August 2023). Now according to my daughter there is a provision in California law which allows employers to pay only 85% of minimum wage for new workers with no prior experience for their first 160 hours of work. For 2023, that &quot;new worker&quot; wage level would work out to $13.17 per hour, but my daughter said that she exceeded her first 160 hours of work for the employer back in March 2023. So, basically, my daughter was being paid under the proper CA 2023 minimum wa...</code> | <code>Read the terms It’s quite likely that, if you took this to court, the employer would be liable to pay your daughter interest on the underpayment and possibly be fined by the state for failing to follow the law. The terms probably are offering to pay the back pay with no interest and your daughter agreeing to confidentiality about the breach. Probably - I haven’t read them. In other words, they’re asking her to sign a contract saying she gets $XXX now, and can’t make any further claims against them. Such releases are commonplace when setting a dispute and there’s probably nothing underhanded going on. Probably - I haven’t read them. Because minors can void contracts if they are not in their interest, they want you, as her legal guardian, to also sign so that can’t happen. A relatively prudent precaution on their part. The alternative is to not sign the document and they presumably won’t pay. It will then be up to you whether to sue them which will cost you money, possibly more than you ...</code> |
  | <code>Can Hawaii secede from the U.S. through legal means? Can Hawaii secede from the U.S. through legal means or is it forbidden by U.S. law? I am asking, because I doubt the U.S. would accept the result of a referendum that rules that the Hawaiians want to secede from the U.S. just like Russia or China wouldn't accept it.</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             | <code>Currently, there is no legal means for a state to secede form the U.S. A quick Google search yields So you want to secede from the U.S.: A four-step guide - The Washington Post: &quot;When the Confederate states seceded in 1861 and were then defeated in the Civil War, the argument is that they demonstrated that you can't secede from the Union. The 1869 Supreme Court case TEXAS v. WHITE ET AL (Legal Information Institute) determined that the secession was never actually a real thing in the eyes of the federal government. The Confederate States of America wasn't an independent country any more than your house is its own country simply because you say it is. 'The Constitution, in all its provisions,' the justices wrote, 'looks to an indestructible Union composed of indestructible States.'&quot; Also from that Post piece: In 2006, Justice Antonin Scalia was asked by screenwriter Dan Turkewitz if the idea of Maine seceding from the country made sense as a possible plot point. Scalia, perhap...</code> |
* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
  ```json
  {
      "scale": 50.0,
      "similarity_fct": "cos_sim"
  }
  ```

#### legal_lens

* Dataset: [legal_lens](https://huggingface.co/datasets/bwang0911/legal_lens_nli) at [a4c8193](https://huggingface.co/datasets/bwang0911/legal_lens_nli/tree/a4c8193930720698fdce36b394957bda75ba8863)
* Size: 107 training samples
* Columns: <code>anchor</code> and <code>positive</code>
* Approximate statistics based on the first 107 samples:
  |         | anchor                                                                                | positive                                                                            |
  |:--------|:--------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
  | type    | string                                                                                | string                                                                              |
  | details | <ul><li>min: 107 tokens</li><li>mean: 164.29 tokens</li><li>max: 192 tokens</li></ul> | <ul><li>min: 27 tokens</li><li>mean: 78.31 tokens</li><li>max: 192 tokens</li></ul> |
* Samples:
  | anchor                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               | positive                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
  |:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
  | <code>DEFENDANT has agreed to a $72,250 settlement to resolve claims that it violated Illinois' Biometric Information Privacy Act (BIPA) by scanning the hands and fingerprints of its employees without obtaining prior consent. The settlement benefits employees in Illinois who had their fingers or hands scanned by a timekeeping system between Dec. 31, 2015, and Dec. 31, 2020, without first signing a consent form. Although DEFENDANT has not admitted any wrongdoing, it agreed to the settlement to resolve the BIPA allegations. Class members can receive an equal share of the net settlement fund, estimated to be around $352.75 per worker. The deadline for exclusion and objection is May 15, 2023, and the final approval hearing for the BIPA settlement is scheduled for June 15, 2023. No claim form is required to benefit from the settlement.</code>                                                    | <code>Y'know, it's funny how things turn out sometimes. I was working this job, right? Nothing fancy, just clocking in and out, doing my thing. They had this newfangled tech system for logging our hours, some sort of hand scan thing. Thought it was pretty cool, not gonna lie. High-tech, right? Made me feel like I was in a sci-fi movie or something.<br><br>But then, things started to get a bit weird. I mean, I didn't notice anything at first, but after a while, it kinda felt off, y'know? Like, I couldn't shake the feeling that something was not right. I mean, it's just a hand scanner, right? What could possibly go wrong?<br><br>And then, outta nowhere, I heard this rumor going around about some sort of issue with the hand scanner. Something about not getting the proper consents or something. It was all a bit hush-hush, and no one was really talking about it openly. But you know how these things go, word gets around.<br><br>So here I am, just trying to do my job, and suddenly I'm in the middle of some sort of t...</code> |
  | <code>DEFENDANT has agreed to pay $6.5 million to settle a nationwide class action lawsuit, which accused the company of making telemarketing calls in violation of the Telephone Consumer Protection Act (TCPA). The lawsuit, filed in 2015, alleged that the company made calls using a prerecorded message to cell phones, residential lines, and numbers on the National Do-Not-Call Registry to obtain new clients. The settlement includes cash payments of between $100 and $150 to eligible members of the class action, as well as covering class administration costs, plaintiffs’ attorneys’ fees and litigation costs up to $2,210,566. A $25,000 court-approved service award will also be given to the plaintiff. DEFENDANT will also implement policies and procedures to prevent future violations. Despite the settlement, DEFENDANT does not admit to any wrongdoing and continues to deny the allegations.</code> | <code>Unsolicited calls? They're a real nuisance. It seems my number made it onto a list somewhere, and now my phone won't stop ringing with offers I never asked for.</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
  | <code>DEFENDANT has agreed to a $8.5 million class action settlement over allegations that its stores used misleading price tags. The lawsuit alleged that the use of the phrase “Compare At” on price tags was deceptive, as it misled customers about the actual price of comparable items at other stores. Customers who purchased items from DEFENDANT's stores in California between July 17, 2011 and Dec. 6, 2017, can file a claim to receive either a merchandise store credit or cash from the settlement. DEFENDANT has also agreed to change pricing disclosures on its website and in its California stores to comply with the state's price comparison advertising regulations. The settlement was granted preliminary approval on Dec. 5, 2017, and class members have until April 9, 2018 to object or opt out.</code>                                                                                               | <code>Got a bargain at my favourite shop, but the "Compare At" tags sure had me thinking other places were pricier. Hmm, interesting marketing strategy!</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
  ```json
  {
      "scale": 50.0,
      "similarity_fct": "cos_sim"
  }
  ```

#### cuad_qa

* Dataset: [cuad_qa](https://huggingface.co/datasets/bwang0911/cuad_qa) at [333b657](https://huggingface.co/datasets/bwang0911/cuad_qa/tree/333b657309dda78d2bcda86742127c6568d9f1c1)
* Size: 11,180 training samples
* Columns: <code>anchor</code> and <code>positive</code>
* Approximate statistics based on the first 1000 samples:
  |         | anchor                                                                              | positive                                                                          |
  |:--------|:------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|
  | type    | string                                                                              | string                                                                            |
  | details | <ul><li>min: 33 tokens</li><li>mean: 51.31 tokens</li><li>max: 105 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 57.1 tokens</li><li>max: 192 tokens</li></ul> |
* Samples:
  | anchor                                                                                                                                                                        | positive                                      |
  |:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------|
  | <code>Highlight the parts (if any) of this contract related to "Document Name" that should be reviewed by a lawyer. Details: The name of the contract</code>                  | <code>DISTRIBUTOR AGREEMENT</code>            |
  | <code>Highlight the parts (if any) of this contract related to "Parties" that should be reviewed by a lawyer. Details: The two or more parties who signed the contract</code> | <code>Distributor</code>                      |
  | <code>Highlight the parts (if any) of this contract related to "Parties" that should be reviewed by a lawyer. Details: The two or more parties who signed the contract</code> | <code>Electric City of Illinois L.L.C.</code> |
* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
  ```json
  {
      "scale": 50.0,
      "similarity_fct": "cos_sim"
  }
  ```

#### privacy_qa

* Dataset: [privacy_qa](https://huggingface.co/datasets/bwang0911/privacy_qa) at [cd59571](https://huggingface.co/datasets/bwang0911/privacy_qa/tree/cd59571b4424c8ad8585dc615bae3b4f23b7da38)
* Size: 6,038 training samples
* Columns: <code>anchor</code> and <code>positive</code>
* Approximate statistics based on the first 1000 samples:
  |         | anchor                                                                            | positive                                                                           |
  |:--------|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
  | type    | string                                                                            | string                                                                             |
  | details | <ul><li>min: 6 tokens</li><li>mean: 11.54 tokens</li><li>max: 30 tokens</li></ul> | <ul><li>min: 4 tokens</li><li>mean: 30.25 tokens</li><li>max: 143 tokens</li></ul> |
* Samples:
  | anchor                                                                                  | positive                                                                                                                                                                                                                                                                                                    |
  |:----------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
  | <code>do you share my location with other people</code>                                 | <code>To use our Applications, you must have an account with a healthcare provider who uses Epic's software and your use of our Applications is also subject to your healthcare provider's privacy policy.</code>                                                                                           |
  | <code>are you selling my data</code>                                                    | <code>Interest Based Advertising Khan Academy does not display any targeted advertising on our Service.</code>                                                                                                                                                                                              |
  | <code>will the data collected from my usage of the app be sold to third parties?</code> | <code>Additionally, if you choose to participate in our member-to-member communications programs, other TripAdvisor members may contact you by using TripAdvisor as an intermediary; however, TripAdvisor will not share your email address with any other members nor display it any public manner.</code> |
* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
  ```json
  {
      "scale": 50.0,
      "similarity_fct": "cos_sim"
  }
  ```

#### legal_sum

* Dataset: [legal_sum](https://huggingface.co/datasets/bwang0911/legal_case_summarization) at [667db49](https://huggingface.co/datasets/bwang0911/legal_case_summarization/tree/667db49d7a2152de6ab0c7e6e44f07fc3b36d2d1)
* Size: 7,773 training samples
* Columns: <code>anchor</code> and <code>positive</code>
* Approximate statistics based on the first 1000 samples:
  |         | anchor                                                                               | positive                                                                             |
  |:--------|:-------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|
  | type    | string                                                                               | string                                                                               |
  | details | <ul><li>min: 192 tokens</li><li>mean: 192.0 tokens</li><li>max: 192 tokens</li></ul> | <ul><li>min: 63 tokens</li><li>mean: 191.26 tokens</li><li>max: 192 tokens</li></ul> |
* Samples:
  | anchor                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           | positive                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
  |:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
  | <code>Appeal No. LXVI of 1949.<br>Appeal from the High Court of judicature, Bombay, in a reference under section 66 of the Indian Income tax Act, 1022.<br>K.M. Munshi (N. P. Nathvani, with him), for the appel lant. ' M.C. Setalvad, Attorney General for India (H. J. Umrigar, with him), for the respondent. 1950.<br>May 26.<br>The judgment of the Court was delivered by MEHR CHAND MAHAJAN J.<br>This is an appeal against a judgment of the High Court of Judicature at Bombay in an income tax matter and it raises the question whether munici pal property tax and urban immoveable property tax payable under the relevant Bombay Acts are allowable deductions under section 9 (1) (iv) of the Indian Income tax Act.<br>The assessee company is an investment company deriving its income from properties in the city of Bombay.<br>For the assessment year 1940 41 the net income of the assessee under the head "property" was computed by the Income tax Officer in the sum of Rs. 6,21,764 after deducting from gross rents certain payments.<br>T...</code> | <code>The charge created in respect of municipal property tax by section 212 of the City of Bombay Municipal Act, 1888, is an "annual charge not being a capital charge" within the mean ing of section 9 (1) (iv) of the Indian Income tax Act, 199.2, and the amount of such charge should therefore be deducted in computing the income from such property for the purposes of section 9 of the Indian Income tax Act.<br>The charge in respect of urban immoveable property tax created by the Bombay Finance Act, 1939 is similar in character and the amount of such charge should also be deducted.<br>The expression "capital charge" in s.9(1) (iv) means a charge created for a capital sum,that is to say, a charge created to. ' secure the discharge of a liability of a capi tal nature; and an "annual charge" means a charge to secure an annual liabili ty. 554<br></code>                                                                                                                                                                    |
  | <code>Civil Appeal No.94 of 1949.<br>107 834 Appeal from a judgment and decree of the High Court of Judi cature at Patna in Appeal from Appellate Decree No. 97 of 1946 (Mannohar Lall and Mukherji JJ.) dated 23rd Decem ber, 1947, confirming the judgment of the District Judge of Purulia in Appeal No. 159 of 1944.<br>S.P. Sinha (P. K. Bose, with him) for the appel lant.<br>N.C. Chatterjee and Panchanan Ghosh (Chandra Narayan Naik, with them) for the respondent. 1950.<br>December 1.<br>The Judgment of the Court was deliv ered by PATANJALI SASTRI J.<br>This appeal arises out of a suit brought by the respondent in the court of the Subordinate Judge, Dhanbad, for recovery of arrears of royalty and cess from the appellant and another alleged to be due under a compromise decree passed on the 6th March, 1923, in a previ ous suit between the predecessors in interest of the par ties.<br>The only plea which is material for the purpose of this appeal is that the compromise decree not having been registered was inadmissible in...</code>    | <code>An agreement for a lease, which a lease is by the Indian declared to include, must be a document which effects an actual demise and operates as a lease.<br>It must create present and immediate interest in land.<br>Where a litigation between two persons A and B who claimed to be tenants under C was settled by a compromise decree the effect of which was to create a perpetual underlease between A and B which was to take effect only on condition that A paid Rs. 8,000 to C within a fixed period: Held, that such a contingent agreement was not "a lease" within el.<br>(a) of section 17 (t) of the Indian , and even though it was covered by cl.<br>(b) of the said sec tion it was exempt from registration under el.<br>(vi) of subs.<br>(2) of section 17.<br>Hemanta Kumari Debi vs Midnapur Zamindari Co. (I P.C.) relied on.<br></code>                                                                                                                                                                                          |
  | <code>iminal Appeal No. 40 of 1951, 127 Appeal from the Judgment and Order dated the 1st June, 1951, of the High Court of Judicature in Assam (Thadani C.J. and Ram Labhaya J.,) in Criminal Reference No. I of 1951, arising out of Judgment and Order dated the 15th November, 1950, of the Court of the Additional District Magistrate, Lakhimpur, in Case No. 1126C of 1950.<br>Jindra Lal for the appellant.<br>Nuruddin Ahmed for the respondent.<br>October 23.<br>The Judgment of the Court was delivered by CHANDRASEKHARA AIYAR J.<br>Rameshwar Bhartia, the appellant, is a shopkeeper in Assam.<br>He was prosecuted for storing paddy without a licence in excess of the quantity permitted by the Assam Food Grains Control Order, 1947.<br>He admitted storage and possession of 550 maunds of paddy, but pleaded that he did not know that any licence was necessary.<br>The 'Additional District Magistrate recorded a plea of guilty, but imposed him a fine of Rs. 50 only, as he considered his ignorance of the provisions of the Food Grains Con...</code> | <code>The question whether a Magistrate is "personally interested" in a ease within the meaning of section 556, Criminal Procedure Code, has essentially to be decided the facts of each case.<br>Where an officer as a District Magistrate exercising his powers under section 7(1) of the Essential Supplies (Temporary Powers) Act, 1946, sanctioned the prosecution of a person for violation of sections 3 and 7 of the Assam Food Grains Control Order, 1947, and the same officer as Additional District Magistrate tried and convicted the accused, and it was contended that as the officer had given sanction for prosecution he was "personally interested" in the case within the meaning of section 656, Criminal Procedure Code, and the trial and conviction were therefore illegal: Held, that bymerely giving sanction for prosecution he did not become personally interested" in the case and the trial and conviction were not illegal.<br>In both cases of sanction and direction to prosecute, an application of the mind is n...</code> |
* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
  ```json
  {
      "scale": 50.0,
      "similarity_fct": "cos_sim"
  }
  ```

#### aus_legal_qa

* Dataset: [aus_legal_qa](https://huggingface.co/datasets/bwang0911/aus_legal_qa) at [0628f4a](https://huggingface.co/datasets/bwang0911/aus_legal_qa/tree/0628f4a78023fa5cde0000b786e3f57a53d29453)
* Size: 2,124 training samples
* Columns: <code>anchor</code> and <code>positive</code>
* Approximate statistics based on the first 1000 samples:
  |         | anchor                                                                              | positive                                                                             |
  |:--------|:------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|
  | type    | string                                                                              | string                                                                               |
  | details | <ul><li>min: 12 tokens</li><li>mean: 38.68 tokens</li><li>max: 106 tokens</li></ul> | <ul><li>min: 21 tokens</li><li>mean: 111.75 tokens</li><li>max: 192 tokens</li></ul> |
* Samples:
  | anchor                                                                                                                                                                                     | positive                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
  |:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
  | <code>In the case of Nasr v NRMA Insurance [2006] NSWSC 1018, why was the plaintiff's appeal lodged out of time?</code>                                                                    | <code>In Nasr v NRMA Insurance [2006] NSWSC 1018, the plaintiff's appeal was lodged out of time because the summons was filed on 8 June 2006, seven months after the decision of the Local Court was made on 4 October 2005. No explanation was provided for this delay.</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
  | <code>In the case of R v NGUYEN [2001] NSWCCA 334, what was the relationship between the Appellant and Mr Nguyen, and what activities of Mr Nguyen did the Appellant testify about?</code> | <code>In the case of R v NGUYEN [2001] NSWCCA 334, the Appellant testified that Mr Nguyen was her cousin and that she had allowed him to live in her flat for about 4 or 5 days. She stated that she had heard that Mr Nguyen was selling heroin and that she had seen him hand over a small foil to a third person, an event that made her feel surprised, upset, and angry. Despite her protests, Mr Nguyen allegedly continued to sell heroin from the flat. The Appellant also mentioned seeing other customers in the flat and a friend of Mr Nguyen's cutting foil in the lounge-room. Despite her complaints to her boyfriend and an aunt, she took no further steps to prevent these activities, citing reasons such as their close familial relationship and her reluctance to involve the police.</code> |
  | <code>In the case of Moore v Scenic Tours Pty Ltd [2015] NSWSC 237, what was the court's decision regarding the motion to restrain a firm from acting?</code>                              | <code>In the case of Moore v Scenic Tours Pty Ltd [2015] NSWSC 237, the court decided to dismiss the motion to restrain a firm from acting. The court found that the plaintiff was entitled to a solicitor of their choice and it was not in the interest of justice to deprive the plaintiff of their choice of solicitor.</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
  ```json
  {
      "scale": 50.0,
      "similarity_fct": "cos_sim"
  }
  ```

### Training Hyperparameters
#### Non-Default Hyperparameters

- `eval_strategy`: steps
- `per_device_train_batch_size`: 64
- `learning_rate`: 1e-06
- `num_train_epochs`: 2
- `warmup_ratio`: 0.1
- `fp16`: True
- `batch_sampler`: no_duplicates

#### All Hyperparameters
<details><summary>Click to expand</summary>

- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: steps
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 64
- `per_device_eval_batch_size`: 8
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 1e-06
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 2
- `max_steps`: -1
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.1
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: False
- `fp16`: True
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: False
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: False
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`: 
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `use_liger_kernel`: False
- `eval_use_gather_object`: False
- `prompts`: None
- `batch_sampler`: no_duplicates
- `multi_dataset_batch_sampler`: proportional

</details>

### Training Logs
| Epoch  | Step | Training Loss | mteb/AILA_casedocs_cosine_ndcg@10 | mteb/AILA_statutes_cosine_ndcg@10 | mteb/legalbench_consumer_contracts_qa_cosine_ndcg@10 | mteb/legalbench_corporate_lobbying_cosine_ndcg@10 | mteb/legal_summarization_cosine_ndcg@10 |
|:------:|:----:|:-------------:|:---------------------------------:|:---------------------------------:|:----------------------------------------------------:|:-------------------------------------------------:|:---------------------------------------:|
| 0      | 0    | -             | 0.1704                            | 0.2351                            | 0.6781                                               | 0.8793                                            | 0.5766                                  |
| 0.1196 | 100  | -             | 0.1709                            | 0.2434                            | 0.6791                                               | 0.8834                                            | 0.5820                                  |
| 0.2392 | 200  | -             | 0.2164                            | 0.2702                            | 0.6808                                               | 0.8832                                            | 0.6015                                  |
| 0.3589 | 300  | -             | 0.2221                            | 0.2707                            | 0.6739                                               | 0.8855                                            | 0.6089                                  |
| 0.4785 | 400  | -             | 0.2170                            | 0.2705                            | 0.6681                                               | 0.8857                                            | 0.6149                                  |
| 0.5981 | 500  | 2.757         | 0.2138                            | 0.2644                            | 0.6711                                               | 0.8830                                            | 0.6116                                  |
| 0.7177 | 600  | -             | 0.2124                            | 0.2725                            | 0.6671                                               | 0.8861                                            | 0.6142                                  |
| 0.8373 | 700  | -             | 0.2235                            | 0.2919                            | 0.6656                                               | 0.8856                                            | 0.6112                                  |
| 0.9569 | 800  | -             | 0.2258                            | 0.2902                            | 0.6632                                               | 0.8848                                            | 0.6128                                  |
| 1.0766 | 900  | -             | 0.2220                            | 0.2999                            | 0.6597                                               | 0.8865                                            | 0.6120                                  |
| 1.1962 | 1000 | 1.6406        | 0.2264                            | 0.3015                            | 0.6582                                               | 0.8870                                            | 0.6106                                  |
| 1.3158 | 1100 | -             | 0.2266                            | 0.2996                            | 0.6576                                               | 0.8859                                            | 0.6097                                  |
| 1.4354 | 1200 | -             | 0.2337                            | 0.2944                            | 0.6581                                               | 0.8863                                            | 0.6066                                  |
| 1.5550 | 1300 | -             | 0.2343                            | 0.2928                            | 0.6572                                               | 0.8829                                            | 0.6064                                  |
| 1.6746 | 1400 | -             | 0.2342                            | 0.2920                            | 0.6566                                               | 0.8822                                            | 0.6041                                  |
| 1.7943 | 1500 | 1.6345        | 0.2358                            | 0.2947                            | 0.6575                                               | 0.8824                                            | 0.6026                                  |
| 1.9139 | 1600 | -             | 0.2357                            | 0.2945                            | 0.6576                                               | 0.8824                                            | 0.6029                                  |


### Framework Versions
- Python: 3.10.12
- Sentence Transformers: 3.3.0
- Transformers: 4.45.2
- PyTorch: 2.5.1+cu124
- Accelerate: 1.1.0
- Datasets: 3.1.0
- Tokenizers: 0.20.3

## Citation

### BibTeX

#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
```

#### MultipleNegativesRankingLoss
```bibtex
@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->