sentence-transformers safetensors mpnet sentence-similarity feature-extraction generated_from_trainer dataset_size:98928 loss:MultipleNegativesRankingLoss arxiv:1908.10084 arxiv:1705.00652 base_model:sentence-transformers/multi-qa-mpnet-base-dot-v1 base_model:finetune:sentence-transformers/multi-qa-mpnet-base-dot-v1 model-index autotrain_compatible endpoints_compatible region:us
---
base_model: sentence-transformers/multi-qa-mpnet-base-dot-v1
datasets: []
language: []
library_name: sentence-transformers
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
- dot_accuracy@1
- dot_accuracy@3
- dot_accuracy@5
- dot_accuracy@10
- dot_precision@1
- dot_precision@3
- dot_precision@5
- dot_precision@10
- dot_recall@1
- dot_recall@3
- dot_recall@5
- dot_recall@10
- dot_ndcg@10
- dot_mrr@10
- dot_map@100
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:98928
- loss:MultipleNegativesRankingLoss
widget:
- source_sentence: Abnormal wakefulness, loss of self-awareness, involuntary trance
  sentences:
  - 'See also: Neonatal lupus erythematosus'
  - 'Scalp-ear-nipple syndrome, as its name suggests, is a condition characterized
    by abnormalities of the scalp, ears, and nipples. Less frequently, affected individuals
    have problems affecting other parts of the body. The features of this disorder
    can vary even within the same family.


    Babies with scalp-ear-nipple syndrome are born with a condition called aplasia
    cutis congenita, which involves patchy abnormal areas (lesions) on the scalp.
    These lesions are firm, raised, hairless nodules that resemble open wounds or
    ulcers at birth, but that heal during childhood.


    The external ears of people with scalp-ear-nipple syndrome may be small, cup-shaped,
    folded over, or otherwise mildly misshapen. Hearing is generally normal. Affected
    individuals also have nipples that are underdeveloped (hypothelia) or absent (athelia).
    In some cases the underlying breast tissue is absent as well (amastia).'
  - "Abnormal state of wakefulness or altered state of consciousness\n\nFor other\
    \ uses, see Trance (disambiguation).\n\nThis article needs additional citations\
    \ for verification. Please help improve this article by adding citations to reliable\
    \ sources. Unsourced material may be challenged and removed.  \nFind sources:\
    \ \"Trance\" – news · newspapers · books · scholar · JSTOR (August 2010) (Learn\
    \ how and when to remove this template message)  \n  \nDissociative trance  \n\
    The Oracle at Delphi was famous for her  divinatory trances throughout the ancient\
    \ Mediterranean world. Oil painting, John Collier, 1891  \nSpecialtyPsychiatry\
    \  \n  \nTrance is an abnormal state of wakefulness in which a person is not self-aware\
    \ and is either altogether unresponsive to external stimuli (but nevertheless\
    \ capable of pursuing and realizing an aim) or is selectively responsive in following\
    \ the directions of the person (if any) who has induced the trance. Trance states\
    \ may occur involuntarily and unbidden."
- source_sentence: respiratory infections, recurrent infections, primary immunodeficiency
  sentences:
  - 'A number sign (#) is used with this entry because of evidence that autosomal
    dominant common variable immunodeficiency-12 (CVID12) is caused by heterozygous
    mutation in the NFKB1 gene (164011) on chromosome 4q24.


    Description


    Common variable immunodeficiency-12 is an autosomal dominant primary immunodeficiency
    characterized by recurrent infections, mainly respiratory, associated with hypogammaglobulinemia.
    The disorder shows a highly variable age at onset and highly variable disease
    severity, even within the same family. Some patients have features of autoimmunity
    (summary by Fliegauf et al., 2015).


    For a general description and a discussion of genetic heterogeneity of common
    variable immunodeficiency, see CVID1 (607594).


    Clinical Features'
  - Kyasanura forest disease (KFD), caused by the KFD virus, is an arbovirus characterized
    by an initial fever, headache and myalgia that can progress to a hemorrhagic disease
    and that in some cases is followed by a second phase characterized by neurological
    manifestations.
  - 'A number sign (#) is used with this entry because of evidence that X-linked syndromic
    mental retardation-33 (MRXS33) is caused by mutation in the TAF1 gene (313650)
    on chromosome Xq13.


    Description


    X-linked syndromic mental retardation-33 is an X-linked recessive neurodevelopmental
    disorder characterized by delayed psychomotor development, intellectual disability,
    and characteristic facial features (summary by O''Rawe et al., 2015).


    Clinical Features'
- source_sentence: Common variable immunodeficiency, recurrent infections, impaired
    antibody production
  sentences:
  - 'A number sign (#) is used with this entry because this form of common variable
    immunodeficiency (CVID), referred to here as CVID5, is caused by homozygous mutation
    in the CD20 gene (MS4A1; 112210) on chromosome 11q13.


    For a general description and a discussion of genetic heterogeneity of common
    variable immunodeficiency, see CVID1 (607594).


    Clinical Features'
  - 'A number sign (#) is used with this entry because of evidence that the Stanescu
    type of spondyloepiphyseal dysplasia (SEDSTN) is caused by heterozygous mutation
    in the COL2A1 gene (120140) on chromosome 12q13.


    Description'
  - '##  Description


    Macular dystrophies are inherited retinal dystrophies in which various forms of
    deposits, pigmentary changes, and atrophic lesions are observed in the macula
    lutea, the cone-rich region of the central retina. Vitelliform macular dystrophies
    (VMDs) form a subset of macular dystrophies characterized by round yellow deposits,
    usually at the center of the macula and containing lipofuscin, a chemically heterogeneous
    pigment visualized by autofluorescence imaging of the fundus (summary by Manes
    et al., 2013). In contrast to typical VMD (see 153700), patients with atypical
    VMD may exhibit normal electrooculography, even when severe loss of vision is
    present, and fluorescein angiography is thus the most reliable test for identifying
    affected individuals (Hittner et al., 1984).


    ### Genetic Heterogeneity of Vitelliform Macular Dystrophy'
- source_sentence: Growth retardation, hearing impairment, joint hypermobility, sacral
    caudal remnant
  sentences:
  - 'A number sign (#) is used with this entry because Bruck syndrome-2 (BRKS2) is
    caused by homozygous mutation in the PLOD2 gene (601865), which encodes telopeptide
    lysyl hydroxylase, on chromosome 3q24.


    For a phenotypic description and a discussion of genetic heterogeneity of Bruck
    syndrome, see Bruck syndrome-1 (259450).


    Clinical Features


    Ha-Vinh et al. (2004) described a child with Bruck syndrome who was the offspring
    of healthy nonconsanguineous Turkish parents. At birth, pterygia were present
    at the left elbow and at both knees, and extension of these joints was limited.
    Contractures were also present at the wrists, and there were bilateral clubfeet.
    Bilateral inguinal hernias were present. A fracture of the left arm was recognized
    immediately after birth, and the boy had 2 more fractures in the first 3 months
    of life. His urine contained high levels of hydroxyproline but low levels of collagen
    crosslinks degradation products.'
  - '## Summary


    ### Clinical characteristics.


    Thrombocytopenia absent radius (TAR) syndrome is characterized by bilateral absence
    of the radii with the presence of both thumbs and thrombocytopenia (<50 platelets/nL)
    that is generally transient. Thrombocytopenia may be congenital or may develop
    within the first few weeks to months of life; in general, thrombocytopenic episodes
    decrease with age. Cow''s milk allergy is common and can be associated with exacerbation
    of thrombocytopenia. Other anomalies of the skeleton (upper and lower limbs, ribs,
    and vertebrae), heart, and genitourinary system (renal anomalies and agenesis
    of uterus, cervix, and upper part of the vagina) can occur.


    ### Diagnosis/testing.'
  - A rare multiple congenital anomalies/dysmorphic syndrome characterized by global
    developmental delay, intellectual disability, growth retardation, hearing impairment,
    characteristic facial dysmorphology (including prominent supraorbital ridges,
    downslanting palpebral fissures, deep-set eyes, long face, sagging cheeks, anteverted
    nares, and pointed chin), generalized hypotonia, joint hypermobility, gluteal
    crease with sacral caudal remnant and sacral dimple, and variable neurological
    features. Various ophthalmic, cutaneous, musculoskeletal, gastrointestinal, and
    cardiovascular anomalies have also been described.
- source_sentence: ear malformations, nipple abnormalities, dental anomalies
  sentences:
  - "This article is an orphan, as no other articles link to it. Please introduce\
    \ links to this page from related articles; try the Find link tool for suggestions.\
    \ (July 2016)  \n  \nInguinal lymphadenopathy  \nInguinal lymphadenopathy  \n\
    \  \nInguinal lymphadenopathy causes swollen lymph nodes in the groin area. It\
    \ can be a symptom of infective or neoplastic processes. Infective aetiologies\
    \ include Tuberculosis, HIV, non-specific or reactive lymphadenopathy to recent\
    \ lower limb infection or groin infections. Another notable infectious cause is\
    \ Lymphogranuloma venereum, which is a sexually transmitted infection of the lymphatic\
    \ system. Neoplastic aetiologies include lymphoma, leukaemia and metastatic disease\
    \ from primary tumours in the lower limb, external genitalia or perianal region\
    \ and melanoma.\n\n## References[edit]\n\n  * Ferrer R (October 1998). \"Lymphadenopathy:\
    \ differential diagnosis and evaluation\". Am Fam Physician. 58 (6): 1313–20.\
    \ PMID 9803196.\n\n## Further reading[edit]"
  - 'A number sign (#) is used with this entry because scalp-ear-nipple syndrome (SENS)
    is caused by heterozygous mutation in the KCTD1 gene (613420) on chromosome 18q11.


    Description


    Scalp-ear-nipple syndrome is characterized by aplasia cutis congenita of the scalp,
    breast anomalies that range from hypothelia or athelia to amastia, and minor anomalies
    of the external ears. Less frequent clinical characteristics include nail dystrophy,
    dental anomalies, cutaneous syndactyly of the digits, and renal malformations.
    Penetrance appears to be high, although there is substantial variable expressivity
    within families (Marneros et al., 2013).


    Clinical Features'
  - Familial multiple meningioma is a rare, benign neoplasm of the central nervous
    system characterized by the development of multiple or, rarely, solitary meningiomas
    in two or more blood relatives, without other apparent syndromic manifestations.
    Depending on the localization, growth rate and size of the tumors, patients can
    present with subtle, gradually worsening or abrupt and severe neurological compromise
    or can be completely asymptomatic.
model-index:
- name: SentenceTransformer based on sentence-transformers/multi-qa-mpnet-base-dot-v1
  results:
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: Unknown
      type: unknown
    metrics:
    - type: cosine_accuracy@1
      value: 0.18070477009024494
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.5426514825956167
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.7380747743876236
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.8160721959604641
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.18070477009024494
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.1808838275318722
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.1476149548775247
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.08160721959604642
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.18070477009024494
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.5426514825956167
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.7380747743876236
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.8160721959604641
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.49469594615283
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.39074511770043246
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.3952600557331103
      name: Cosine Map@100
    - type: dot_accuracy@1
      value: 0.18274602492479589
      name: Dot Accuracy@1
    - type: dot_accuracy@3
      value: 0.5412548345509239
      name: Dot Accuracy@3
    - type: dot_accuracy@5
      value: 0.7430167597765364
      name: Dot Accuracy@5
    - type: dot_accuracy@10
      value: 0.8167168027503223
      name: Dot Accuracy@10
    - type: dot_precision@1
      value: 0.18274602492479589
      name: Dot Precision@1
    - type: dot_precision@3
      value: 0.18041827818364134
      name: Dot Precision@3
    - type: dot_precision@5
      value: 0.1486033519553073
      name: Dot Precision@5
    - type: dot_precision@10
      value: 0.08167168027503223
      name: Dot Precision@10
    - type: dot_recall@1
      value: 0.18274602492479589
      name: Dot Recall@1
    - type: dot_recall@3
      value: 0.5412548345509239
      name: Dot Recall@3
    - type: dot_recall@5
      value: 0.7430167597765364
      name: Dot Recall@5
    - type: dot_recall@10
      value: 0.8167168027503223
      name: Dot Recall@10
    - type: dot_ndcg@10
      value: 0.4956715454485796
      name: Dot Ndcg@10
    - type: dot_mrr@10
      value: 0.391808804169147
      name: Dot Mrr@10
    - type: dot_map@100
      value: 0.39626188359327835
      name: Dot Map@100
---

# SentenceTransformer based on sentence-transformers/multi-qa-mpnet-base-dot-v1

This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/multi-qa-mpnet-base-dot-v1](https://huggingface.co/sentence-transformers/multi-qa-mpnet-base-dot-v1). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [sentence-transformers/multi-qa-mpnet-base-dot-v1](https://huggingface.co/sentence-transformers/multi-qa-mpnet-base-dot-v1) <!-- at revision 3af7c6da5b3e1bea796ef6c97fe237538cbe6e7f -->
- **Maximum Sequence Length:** 512 tokens
- **Output Dimensionality:** 768 tokens
- **Similarity Function:** Dot Product
<!-- - **Training Dataset:** Unknown -->
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->

### Model Sources

- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)

### Full Model Architecture

```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: MPNetModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)
```

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'ear malformations, nipple abnormalities, dental anomalies',
    'A number sign (#) is used with this entry because scalp-ear-nipple syndrome (SENS) is caused by heterozygous mutation in the KCTD1 gene (613420) on chromosome 18q11.\n\nDescription\n\nScalp-ear-nipple syndrome is characterized by aplasia cutis congenita of the scalp, breast anomalies that range from hypothelia or athelia to amastia, and minor anomalies of the external ears. Less frequent clinical characteristics include nail dystrophy, dental anomalies, cutaneous syndactyly of the digits, and renal malformations. Penetrance appears to be high, although there is substantial variable expressivity within families (Marneros et al., 2013).\n\nClinical Features',
    'This article is an orphan, as no other articles link to it. Please introduce links to this page from related articles; try the Find link tool for suggestions. (July 2016)  \n  \nInguinal lymphadenopathy  \nInguinal lymphadenopathy  \n  \nInguinal lymphadenopathy causes swollen lymph nodes in the groin area. It can be a symptom of infective or neoplastic processes. Infective aetiologies include Tuberculosis, HIV, non-specific or reactive lymphadenopathy to recent lower limb infection or groin infections. Another notable infectious cause is Lymphogranuloma venereum, which is a sexually transmitted infection of the lymphatic system. Neoplastic aetiologies include lymphoma, leukaemia and metastatic disease from primary tumours in the lower limb, external genitalia or perianal region and melanoma.\n\n## References[edit]\n\n  * Ferrer R (October 1998). "Lymphadenopathy: differential diagnosis and evaluation". Am Fam Physician. 58 (6): 1313–20. PMID 9803196.\n\n## Further reading[edit]',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```

<!--
### Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details>
-->

<!--
### Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details>
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

## Evaluation

### Metrics

#### Information Retrieval

* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)

| Metric              | Value      |
|:--------------------|:-----------|
| cosine_accuracy@1   | 0.1807     |
| cosine_accuracy@3   | 0.5427     |
| cosine_accuracy@5   | 0.7381     |
| cosine_accuracy@10  | 0.8161     |
| cosine_precision@1  | 0.1807     |
| cosine_precision@3  | 0.1809     |
| cosine_precision@5  | 0.1476     |
| cosine_precision@10 | 0.0816     |
| cosine_recall@1     | 0.1807     |
| cosine_recall@3     | 0.5427     |
| cosine_recall@5     | 0.7381     |
| cosine_recall@10    | 0.8161     |
| cosine_ndcg@10      | 0.4947     |
| cosine_mrr@10       | 0.3907     |
| cosine_map@100      | 0.3953     |
| dot_accuracy@1      | 0.1827     |
| dot_accuracy@3      | 0.5413     |
| dot_accuracy@5      | 0.743      |
| dot_accuracy@10     | 0.8167     |
| dot_precision@1     | 0.1827     |
| dot_precision@3     | 0.1804     |
| dot_precision@5     | 0.1486     |
| dot_precision@10    | 0.0817     |
| dot_recall@1        | 0.1827     |
| dot_recall@3        | 0.5413     |
| dot_recall@5        | 0.743      |
| dot_recall@10       | 0.8167     |
| dot_ndcg@10         | 0.4957     |
| dot_mrr@10          | 0.3918     |
| **dot_map@100**     | **0.3963** |

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Dataset

#### Unnamed Dataset


* Size: 98,928 training samples
* Columns: <code>queries</code> and <code>chunks</code>
* Approximate statistics based on the first 1000 samples:
  |         | queries                                                                          | chunks                                                                              |
  |:--------|:---------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
  | type    | string                                                                           | string                                                                              |
  | details | <ul><li>min: 7 tokens</li><li>mean: 17.4 tokens</li><li>max: 76 tokens</li></ul> | <ul><li>min: 5 tokens</li><li>mean: 159.93 tokens</li><li>max: 334 tokens</li></ul> |
* Samples:
  | queries                                                                  | chunks                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
  |:-------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
  | <code>fever, malaise, headaches, lymphadenopathy</code>                  | <code>A rare, acquired, self-limiting, infectious disease due to the mite-borne bacteria Rickettsia akari characterized by an asymptomatic, 0.5 to 2 cm in diameter papulovesicle that typically ulcerates and forms an eschar, followed by a generalized papulovesicular rash associating variable constitutional symptoms, such as localized lymphadenopathy, fever, malaise, and headaches. Additonal symptoms may include diaphoresis, myalgia and, less frequently, rhinorrhea, pharyngitis, nausea, vomiting, splenomegaly, conjunctival hyperemia, and abdominal pain. Systemic symtoms resolve within 6-10 days.</code> |
  | <code>rash, papulovesicular, generalized, constitutional symptoms</code> | <code>A rare, acquired, self-limiting, infectious disease due to the mite-borne bacteria Rickettsia akari characterized by an asymptomatic, 0.5 to 2 cm in diameter papulovesicle that typically ulcerates and forms an eschar, followed by a generalized papulovesicular rash associating variable constitutional symptoms, such as localized lymphadenopathy, fever, malaise, and headaches. Additonal symptoms may include diaphoresis, myalgia and, less frequently, rhinorrhea, pharyngitis, nausea, vomiting, splenomegaly, conjunctival hyperemia, and abdominal pain. Systemic symtoms resolve within 6-10 days.</code> |
  | <code>myalgia, diaphoresis, nausea, vomiting</code>                      | <code>A rare, acquired, self-limiting, infectious disease due to the mite-borne bacteria Rickettsia akari characterized by an asymptomatic, 0.5 to 2 cm in diameter papulovesicle that typically ulcerates and forms an eschar, followed by a generalized papulovesicular rash associating variable constitutional symptoms, such as localized lymphadenopathy, fever, malaise, and headaches. Additonal symptoms may include diaphoresis, myalgia and, less frequently, rhinorrhea, pharyngitis, nausea, vomiting, splenomegaly, conjunctival hyperemia, and abdominal pain. Systemic symtoms resolve within 6-10 days.</code> |
* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
  ```json
  {
      "scale": 1,
      "similarity_fct": "dot_score"
  }
  ```

### Evaluation Dataset

#### Unnamed Dataset


* Size: 9,308 evaluation samples
* Columns: <code>queries</code> and <code>chunks</code>
* Approximate statistics based on the first 1000 samples:
  |         | queries                                                                          | chunks                                                                              |
  |:--------|:---------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
  | type    | string                                                                           | string                                                                              |
  | details | <ul><li>min: 7 tokens</li><li>mean: 17.8 tokens</li><li>max: 48 tokens</li></ul> | <ul><li>min: 4 tokens</li><li>mean: 166.19 tokens</li><li>max: 299 tokens</li></ul> |
* Samples:
  | queries                                                                                                | chunks                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
  |:-------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
  | <code>facial features, overgrowth, learning disabilities, delayed development</code>                   | <code>Sotos syndrome is a condition characterized mainly by distinctive facial features; overgrowth in childhood; and learning disabilities or delayed development. Facial features may include a long, narrow face; a high forehead; flushed (reddened) cheeks; a small, pointed chin; and down-slanting palpebral fissures. Affected infants and children tend to grow quickly; they are significantly taller than their siblings and peers and have a large head. Other signs and symptoms may include intellectual disability; behavioral problems; problems with speech and language; and/or weak muscle tone (hypotonia). Sotos syndrome is usually caused by a mutation in the NSD1 gene and is inherited in an autosomal dominant manner. About 95% of cases are due to a new mutation in the affected person and occur sporadically (are not inherited).</code> |
  | <code>long face, high forehead, flushed cheeks, small chin, down-slanting palpebral fissures</code>    | <code>Sotos syndrome is a condition characterized mainly by distinctive facial features; overgrowth in childhood; and learning disabilities or delayed development. Facial features may include a long, narrow face; a high forehead; flushed (reddened) cheeks; a small, pointed chin; and down-slanting palpebral fissures. Affected infants and children tend to grow quickly; they are significantly taller than their siblings and peers and have a large head. Other signs and symptoms may include intellectual disability; behavioral problems; problems with speech and language; and/or weak muscle tone (hypotonia). Sotos syndrome is usually caused by a mutation in the NSD1 gene and is inherited in an autosomal dominant manner. About 95% of cases are due to a new mutation in the affected person and occur sporadically (are not inherited).</code> |
  | <code>intellectual disability, behavioral problems, speech and language difficulties, hypotonia</code> | <code>Sotos syndrome is a condition characterized mainly by distinctive facial features; overgrowth in childhood; and learning disabilities or delayed development. Facial features may include a long, narrow face; a high forehead; flushed (reddened) cheeks; a small, pointed chin; and down-slanting palpebral fissures. Affected infants and children tend to grow quickly; they are significantly taller than their siblings and peers and have a large head. Other signs and symptoms may include intellectual disability; behavioral problems; problems with speech and language; and/or weak muscle tone (hypotonia). Sotos syndrome is usually caused by a mutation in the NSD1 gene and is inherited in an autosomal dominant manner. About 95% of cases are due to a new mutation in the affected person and occur sporadically (are not inherited).</code> |
* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
  ```json
  {
      "scale": 1,
      "similarity_fct": "dot_score"
  }
  ```

### Training Hyperparameters
#### Non-Default Hyperparameters

- `eval_strategy`: steps
- `per_device_train_batch_size`: 32
- `per_device_eval_batch_size`: 32
- `learning_rate`: 2e-05
- `num_train_epochs`: 25
- `warmup_ratio`: 0.1
- `fp16`: True
- `load_best_model_at_end`: True
- `eval_on_start`: True
- `batch_sampler`: no_duplicates

#### All Hyperparameters
<details><summary>Click to expand</summary>

- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: steps
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 32
- `per_device_eval_batch_size`: 32
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 2e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 25
- `max_steps`: -1
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.1
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: False
- `fp16`: True
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: True
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: True
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: False
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`: 
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: True
- `eval_use_gather_object`: False
- `batch_sampler`: no_duplicates
- `multi_dataset_batch_sampler`: proportional

</details>

### Training Logs
<details><summary>Click to expand</summary>

| Epoch      | Step     | Training Loss | loss      | dot_map@100 |
|:----------:|:--------:|:-------------:|:---------:|:-----------:|
| 0          | 0        | -             | 1.8701    | 0.2095      |
| 0.1295     | 100      | 1.5494        | -         | -           |
| 0.2591     | 200      | 0.9993        | -         | -           |
| 0.3886     | 300      | 0.7225        | -         | -           |
| 0.5181     | 400      | 0.6533        | -         | -           |
| 0.6477     | 500      | 0.6618        | 0.5939    | 0.3722      |
| 0.7772     | 600      | 0.6454        | -         | -           |
| 0.9067     | 700      | 0.5568        | -         | -           |
| 1.0363     | 800      | 0.5435        | -         | -           |
| 1.1658     | 900      | 0.499         | -         | -           |
| 1.2953     | 1000     | 0.5386        | 0.4768    | 0.3842      |
| 1.4249     | 1100     | 0.5077        | -         | -           |
| 1.5544     | 1200     | 0.4929        | -         | -           |
| 1.6839     | 1300     | 0.5194        | -         | -           |
| 1.8135     | 1400     | 0.5157        | -         | -           |
| 1.9430     | 1500     | 0.4337        | 0.4455    | 0.3894      |
| 2.0725     | 1600     | 0.4373        | -         | -           |
| 2.2021     | 1700     | 0.4569        | -         | -           |
| 2.3316     | 1800     | 0.4084        | -         | -           |
| 2.4611     | 1900     | 0.42          | -         | -           |
| 2.5907     | 2000     | 0.4112        | 0.4578    | 0.3886      |
| 2.7202     | 2100     | 0.4498        | -         | -           |
| 2.8497     | 2200     | 0.415         | -         | -           |
| 2.9793     | 2300     | 0.3734        | -         | -           |
| 3.1088     | 2400     | 0.3359        | -         | -           |
| 3.2383     | 2500     | 0.3923        | 0.4339    | 0.3929      |
| 3.3679     | 2600     | 0.3345        | -         | -           |
| 3.4974     | 2700     | 0.3324        | -         | -           |
| 3.6269     | 2800     | 0.3574        | -         | -           |
| 3.7565     | 2900     | 0.4078        | -         | -           |
| 3.8860     | 3000     | 0.3221        | 0.4293    | 0.3904      |
| 4.0155     | 3100     | 0.2895        | -         | -           |
| 4.1451     | 3200     | 0.2821        | -         | -           |
| 4.2746     | 3300     | 0.3192        | -         | -           |
| 4.4041     | 3400     | 0.28          | -         | -           |
| 4.5337     | 3500     | 0.2716        | 0.4486    | 0.3885      |
| 4.6632     | 3600     | 0.3147        | -         | -           |
| 4.7927     | 3700     | 0.3565        | -         | -           |
| 4.9223     | 3800     | 0.2465        | -         | -           |
| 5.0518     | 3900     | 0.2436        | -         | -           |
| 5.1813     | 4000     | 0.2297        | 0.4486    | 0.3917      |
| 5.3109     | 4100     | 0.2538        | -         | -           |
| 5.4404     | 4200     | 0.2448        | -         | -           |
| 5.5699     | 4300     | 0.2433        | -         | -           |
| 5.6995     | 4400     | 0.3017        | -         | -           |
| 5.8290     | 4500     | 0.2958        | 0.4737    | 0.3934      |
| 5.9585     | 4600     | 0.2142        | -         | -           |
| 6.0881     | 4700     | 0.1939        | -         | -           |
| 6.2176     | 4800     | 0.2449        | -         | -           |
| 6.3472     | 4900     | 0.2026        | -         | -           |
| 6.4767     | 5000     | 0.2006        | 0.4901    | 0.3895      |
| 6.6062     | 5100     | 0.2118        | -         | -           |
| 6.7358     | 5200     | 0.3064        | -         | -           |
| 6.8653     | 5300     | 0.2276        | -         | -           |
| 6.9948     | 5400     | 0.1809        | -         | -           |
| 7.1244     | 5500     | 0.1782        | 0.4992    | 0.3915      |
| 7.2539     | 5600     | 0.2211        | -         | -           |
| 7.3834     | 5700     | 0.1728        | -         | -           |
| 7.5130     | 5800     | 0.1651        | -         | -           |
| 7.6425     | 5900     | 0.2158        | -         | -           |
| 7.7720     | 6000     | 0.2864        | 0.5113    | 0.3892      |
| 7.9016     | 6100     | 0.179         | -         | -           |
| 8.0311     | 6200     | 0.1677        | -         | -           |
| 8.1606     | 6300     | 0.1517        | -         | -           |
| 8.2902     | 6400     | 0.1851        | -         | -           |
| 8.4197     | 6500     | 0.1646        | 0.5030    | 0.3933      |
| 8.5492     | 6600     | 0.1608        | -         | -           |
| 8.6788     | 6700     | 0.217         | -         | -           |
| 8.8083     | 6800     | 0.2357        | -         | -           |
| 8.9378     | 6900     | 0.1404        | -         | -           |
| 9.0674     | 7000     | 0.1465        | 0.5153    | 0.3877      |
| 9.1969     | 7100     | 0.1791        | -         | -           |
| 9.3264     | 7200     | 0.1261        | -         | -           |
| 9.4560     | 7300     | 0.1406        | -         | -           |
| 9.5855     | 7400     | 0.1626        | -         | -           |
| 9.7150     | 7500     | 0.223         | 0.5326    | 0.3939      |
| 9.8446     | 7600     | 0.1806        | -         | -           |
| 9.9741     | 7700     | 0.1289        | -         | -           |
| 10.1036    | 7800     | 0.1269        | -         | -           |
| 10.2332    | 7900     | 0.1609        | -         | -           |
| 10.3627    | 8000     | 0.1279        | 0.5113    | 0.3933      |
| 10.4922    | 8100     | 0.1264        | -         | -           |
| 10.6218    | 8200     | 0.1453        | -         | -           |
| 10.7513    | 8300     | 0.2227        | -         | -           |
| 10.8808    | 8400     | 0.1314        | -         | -           |
| 11.0104    | 8500     | 0.1192        | 0.5444    | 0.3925      |
| 11.1399    | 8600     | 0.1164        | -         | -           |
| 11.2694    | 8700     | 0.1418        | -         | -           |
| 11.3990    | 8800     | 0.1202        | -         | -           |
| 11.5285    | 8900     | 0.1152        | -         | -           |
| **11.658** | **9000** | **0.1454**    | **0.529** | **0.3963**  |
| 11.7876    | 9100     | 0.1952        | -         | -           |
| 11.9171    | 9200     | 0.1079        | -         | -           |
| 12.0466    | 9300     | 0.1139        | -         | -           |
| 12.1762    | 9400     | 0.1067        | -         | -           |
| 12.3057    | 9500     | 0.1219        | 0.5257    | 0.3938      |
| 12.4352    | 9600     | 0.119         | -         | -           |
| 12.5648    | 9700     | 0.1195        | -         | -           |
| 12.6943    | 9800     | 0.158         | -         | -           |
| 12.8238    | 9900     | 0.156         | -         | -           |
| 12.9534    | 10000    | 0.0974        | 0.5434    | 0.3934      |
| 13.0829    | 10100    | 0.0928        | -         | -           |
| 13.2124    | 10200    | 0.1266        | -         | -           |
| 13.3420    | 10300    | 0.0964        | -         | -           |
| 13.4715    | 10400    | 0.1007        | -         | -           |
| 13.6010    | 10500    | 0.112         | 0.5789    | 0.3893      |
| 13.7306    | 10600    | 0.1699        | -         | -           |
| 13.8601    | 10700    | 0.1084        | -         | -           |
| 13.9896    | 10800    | 0.0967        | -         | -           |
| 14.1192    | 10900    | 0.0856        | -         | -           |
| 14.2487    | 11000    | 0.1142        | 0.5252    | 0.3933      |
| 14.3782    | 11100    | 0.0891        | -         | -           |
| 14.5078    | 11200    | 0.0911        | -         | -           |
| 14.6373    | 11300    | 0.1128        | -         | -           |
| 14.7668    | 11400    | 0.1686        | -         | -           |
| 14.8964    | 11500    | 0.0874        | 0.5874    | 0.3945      |
| 15.0259    | 11600    | 0.0909        | -         | -           |
| 15.1554    | 11700    | 0.0778        | -         | -           |
| 15.2850    | 11800    | 0.1055        | -         | -           |
| 15.4145    | 11900    | 0.0872        | -         | -           |
| 15.5440    | 12000    | 0.0884        | 0.5894    | 0.3934      |
| 15.6736    | 12100    | 0.1101        | -         | -           |
| 15.8031    | 12200    | 0.1354        | -         | -           |
| 15.9326    | 12300    | 0.0762        | -         | -           |
| 16.0622    | 12400    | 0.0782        | -         | -           |
| 16.1917    | 12500    | 0.0936        | 0.5589    | 0.3919      |
| 16.3212    | 12600    | 0.072         | -         | -           |
| 16.4508    | 12700    | 0.0806        | -         | -           |
| 16.5803    | 12800    | 0.0929        | -         | -           |
| 16.7098    | 12900    | 0.1215        | -         | -           |
| 16.8394    | 13000    | 0.1039        | 0.6025    | 0.3926      |
| 16.9689    | 13100    | 0.0738        | -         | -           |
| 17.0984    | 13200    | 0.0651        | -         | -           |
| 17.2280    | 13300    | 0.0943        | -         | -           |
| 17.3575    | 13400    | 0.0678        | -         | -           |
| 17.4870    | 13500    | 0.077         | 0.6002    | 0.3941      |
| 17.6166    | 13600    | 0.0839        | -         | -           |
| 17.7461    | 13700    | 0.1268        | -         | -           |
| 17.8756    | 13800    | 0.0764        | -         | -           |
| 18.0052    | 13900    | 0.0686        | -         | -           |
| 18.1347    | 14000    | 0.0697        | 0.5898    | 0.3913      |
| 18.2642    | 14100    | 0.0871        | -         | -           |
| 18.3938    | 14200    | 0.0699        | -         | -           |
| 18.5233    | 14300    | 0.0611        | -         | -           |
| 18.6528    | 14400    | 0.0872        | -         | -           |
| 18.7824    | 14500    | 0.1281        | 0.6087    | 0.3927      |
| 18.9119    | 14600    | 0.0583        | -         | -           |
| 19.0415    | 14700    | 0.0658        | -         | -           |
| 19.1710    | 14800    | 0.0595        | -         | -           |
| 19.3005    | 14900    | 0.0816        | -         | -           |
| 19.4301    | 15000    | 0.0699        | 0.6078    | 0.3965      |
| 19.5596    | 15100    | 0.0729        | -         | -           |
| 19.6891    | 15200    | 0.0908        | -         | -           |
| 19.8187    | 15300    | 0.0978        | -         | -           |
| 19.9482    | 15400    | 0.0585        | -         | -           |
| 20.0777    | 15500    | 0.0557        | 0.5861    | 0.3925      |
| 20.2073    | 15600    | 0.0787        | -         | -           |
| 20.3368    | 15700    | 0.061         | -         | -           |
| 20.4663    | 15800    | 0.0638        | -         | -           |
| 20.5959    | 15900    | 0.0656        | -         | -           |
| 20.7254    | 16000    | 0.1003        | 0.6032    | 0.3923      |
| 20.8549    | 16100    | 0.0718        | -         | -           |
| 20.9845    | 16200    | 0.0625        | -         | -           |
| 21.1140    | 16300    | 0.0532        | -         | -           |
| 21.2435    | 16400    | 0.0739        | -         | -           |
| 21.3731    | 16500    | 0.0552        | 0.6080    | 0.3942      |
| 21.5026    | 16600    | 0.0588        | -         | -           |
| 21.6321    | 16700    | 0.0716        | -         | -           |
| 21.7617    | 16800    | 0.1078        | -         | -           |
| 21.8912    | 16900    | 0.0559        | -         | -           |
| 22.0207    | 17000    | 0.0596        | 0.6044    | 0.3922      |
| 22.1503    | 17100    | 0.0512        | -         | -           |
| 22.2798    | 17200    | 0.0716        | -         | -           |
| 22.4093    | 17300    | 0.0574        | -         | -           |
| 22.5389    | 17400    | 0.058         | -         | -           |
| 22.6684    | 17500    | 0.07          | 0.6117    | 0.3942      |
| 22.7979    | 17600    | 0.0965        | -         | -           |
| 22.9275    | 17700    | 0.0507        | -         | -           |
| 23.0570    | 17800    | 0.0498        | -         | -           |
| 23.1865    | 17900    | 0.0524        | -         | -           |
| 23.3161    | 18000    | 0.0656        | 0.5936    | 0.3936      |
| 23.4456    | 18100    | 0.057         | -         | -           |
| 23.5751    | 18200    | 0.0619        | -         | -           |
| 23.7047    | 18300    | 0.0785        | -         | -           |
| 23.8342    | 18400    | 0.0729        | -         | -           |
| 23.9637    | 18500    | 0.0541        | 0.6174    | 0.3979      |
| 24.0933    | 18600    | 0.0456        | -         | -           |
| 24.2228    | 18700    | 0.0696        | -         | -           |
| 24.3523    | 18800    | 0.048         | -         | -           |
| 24.4819    | 18900    | 0.0547        | -         | -           |
| 24.6114    | 19000    | 0.0553        | 0.6146    | 0.3962      |
| 24.7409    | 19100    | 0.0936        | -         | -           |
| 24.8705    | 19200    | 0.0579        | -         | -           |
| 25.0       | 19300    | 0.0498        | 0.5290    | 0.3963      |

* The bold row denotes the saved checkpoint.
</details>

### Framework Versions
- Python: 3.11.9
- Sentence Transformers: 3.0.1
- Transformers: 4.43.3
- PyTorch: 2.3.1+cu121
- Accelerate: 0.30.1
- Datasets: 2.19.2
- Tokenizers: 0.19.1

## Citation

### BibTeX

#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
```

#### MultipleNegativesRankingLoss
```bibtex
@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply}, 
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->