transformers pytorch bert feature-extraction text-embeddings-inference endpoints_compatible region:us
---
{}
---
This BiEncoder is designed with the specific purpose of normalizing annotations generated by the `datummd/NCBI_BC5CDR_disease` model to their corresponding Medical Subject Headings (MeSH) term, excluding branches C and F.
Please download the `mesh_embeddings` file under Files and versions. These embeddings correspond to the [CLS] embeddings for MeSH terms. Load and set the file as the `mesh_ordered_dictionary`.

This code snippet demonstrates the processing of text data using various pipelines and transformers. It involves extracting disease-related information from a corpus of text and then normalizing the annotations. 

## Corpus
Here's the text corpus used in the code:

```python
corpus = [
    'They support the hypothesis that short-term aerobic exercise interventions can act as a buffer against depression and perceived stress in university students after 6 weeks of aerobic exercise of low to moderate intensity.',
    'RESULTS: Vigorous physical activity, self-reported stress, a type A behavior pattern, and less frequent intakes of green-yellow vegetables were significantly associated with an increased risk of amyotrophic lateral sclerosis, whereas smoking and drinking habits were not.',
    'Benfotiamine decreased oxidative stress and inflammation, two major characteristics of neurodegenerative diseases, in a neuroblastoma cell line (Neuro2a) and an immortalized brain microglial cell line (BV2).',
    'Vitamin C Attenuates Oxidative Stress and Behavioral Abnormalities Triggered by Fipronil and Pyriproxyfen Insecticide Chronic Exposure on Zebrafish Juveniles.',
    'Synthesis suggests beneficial associations between green space exposure and reduced stress, positive mood, fewer depressive symptoms, better emotional well-being, improved mental health and behavior, and decreased psychological distress in adolescents.'
]
```

## Annotation Pipeline
We use the `BertForTokenClassification` model by `datummd/NCBI_BC5CDR_disease` to create an annotation pipeline for disease-related information. Here's how it's done:
```python
from transformers import pipeline

annotation_pipeline = pipeline(
    model="datummd/NCBI_BC5CDR_disease",
    tokenizer="datummd/NCBI_BC5CDR_disease",
    aggregation_strategy='average',
    device=0
)

annotations = annotation_pipeline(corpus)
```

## Extracting Annotations
Annotations with a confidence score greater than 0.75 are extracted and stored in a list. The code snippet for this is as follows:
```python
annotations = [
    {'text': text, 'start': span['start'], 'end': span['end']}
    for text, spans in zip(corpus, annotations)
    for span in spans if span['score'] > 0.75
]
```

## Normalizing Annotations
We utilize this pipeline for mapping the annotations to their respective MeSH term. The embeddings for each MeSH term can be downloaded under "Files and versions" and then used for normalizing the annotations:
```python
import torch

mesh_ordered_dict = torch.load('/path/to/mesh-pipeline/mesh_embeddings.pt')

mesh_pipeline = pipeline(
    model="Vydiant/mesh-pipeline",
    trust_remote_code=True,
    device=0,
    mesh_ordered_dictionary=mesh_ordered_dict
)

normalized_annotations = mesh_pipeline(annotations)
```

The outputs will be the similiarity scores of each MeSH term with respective to the given annotation and the MeSH record with the highest score. 