transformers pytorch safetensors bert text-classification en autotrain_compatible endpoints_compatible region:us
---
language:
- en
library_name: transformers
metrics:
- f1
- accuracy
- recall
- precision
pipeline_tag: text-classification
widget:
- text: The past 25 years have seen a strong increase in the number of publications
    related to criticality in different areas of neuroscience. The potential of criticality
    to explain various brain properties, including optimal information processing,
    has made it an increasingly exciting area of investigation for neuroscientists.
    Recent reviews on this topic, sometimes termed brain criticality, make brief mention
    of clinical applications of these findings to several neurological disorders such
    as epilepsy, neurodegenerative disease, and neonatal hypoxia. Other clinicallyrelevant
    domains - including anesthesia, sleep medicine, developmental-behavioral pediatrics,
    and psychiatry - are seldom discussed in review papers of brain criticality. Thorough
    assessments of these application areas and their relevance for clinicians have
    also yet to be published. In this scoping review, studies of brain criticality
    involving human data of all ages are evaluated for their current and future clinical
    relevance. To make the results of these studies understandable to a more clinical
    audience, a review of the key concepts behind criticality (e.g., phase transitions,
    long-range temporal correlation, self-organized criticality, power laws, branching
    processes) precedes the discussion of human clinical studies. Open questions and
    forthcoming areas of investigation are also considered.
---

# MIReAD Neuro

This model is a fine-tuned version of [arazd/MIReAD](https://huggingface.co/arazd/MIReAD) on a dataset of Neuroscience papers from 200 journals collected from various sources for a journal classification task.
It achieves the following results on the evaluation set:

- Loss: 2.7117
- Accuracy: 0.4011
- F1: 0.3962
- Precision: 0.4066
- Recall: 0.3999


## Model description

This model was trained on a journal classification task.

## Intended uses & limitations

The intended use of this model is to create abstract embeddings for semantic similarity search for neuroscience-related articles. 

## Model Usage

To load the model:

```py
from transformers import BertForSequenceClassification, AutoTokenizer
model_path = "biodatlab/MIReAD-Neuro"
model = BertForSequenceClassification.from_pretrained(model_path)
tokenizer = AutoTokenizer.from_pretrained(model_path)
```

To create embeddings and for classification:

```py
# sample abstract & title text
title = "Why Brain Criticality Is Clinically Relevant: A Scoping Review."
abstract = "The past 25 years have seen a strong increase in the number of publications related to criticality in different areas of neuroscience..."
text = title + tokenizer.sep_token + abstract
tokens = tokenizer(
    text,
    max_length=512,
    padding=True,
    truncation=True,
    return_tensors="pt"
)

# to generate an embedding from a given title and abstract
with torch.no_grad():
  output = model.bert(**tokens)
  embedding = output.last_hidden_state[:, 0, :]

# to classify (200 journals) a given title and abstract
output = model(**tokens)
class = output.logits 
```

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 16
- num_epochs: 6