readme: ---
  language: en
  tags:
  - textual-entailment
  - nli
  - pytorch
  datasets:
  - mnli
  license: mit
  widget :
  - text: "EpCAM is overexpressed in breast cancer. </s></s> EpCAM is downregulated in breast cancer."
  ---
  
  # BiomedNLP-PubMedBERT finetuned on textual entailment (NLI)
  
  The [microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract-fulltext](https://huggingface.co/microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract-fulltext?text=%5BMASK%5D+is+a+tumor+suppressor+gene) finetuned on the MNLI dataset. It should be useful in textual entailment tasks involving biomedical corpora.
  
  ## Usage 
  Given two sentences (a premise and a hypothesis), the model outputs the logits of entailment, neutral or contradiction. 
  
  You can test the model using the HuggingFace model widget on the side:
  - Input two sentences (premise and hypothesis) one after the other.
  - The model returns the probabilities of 3 labels: entailment(LABEL:0), neutral(LABEL:1) and contradiction(LABEL:2) respectively.
  
  To use the model locally on your machine: 
  ```python
  # import torch
  # device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
  
  from transformers import AutoTokenizer, AutoModelForSequenceClassification
    
  tokenizer = AutoTokenizer.from_pretrained("lighteternal/BiomedNLP-PubMedBERT-base-uncased-abstract-fulltext-finetuned-mnli")
  model = AutoModelForSequenceClassification.from_pretrained("lighteternal/BiomedNLP-PubMedBERT-base-uncased-abstract-fulltext-finetuned-mnli")
  
  premise = 'EpCAM is overexpressed in breast cancer'
  hypothesis = 'EpCAM is downregulated in breast cancer.'
  
  # run through model pre-trained on MNLI
  x = tokenizer.encode(premise, hypothesis, return_tensors='pt',
                       truncation_strategy='only_first')
  logits = model(x)[0]
  
  probs = logits.softmax(dim=1)
  print('Probabilities for entailment, neutral, contradiction \n', np.around(probs.cpu().
                                                                             detach().numpy(),3))
  # Probabilities for entailment, neutral, contradiction 
  # 0.001 0.001 0.998
  
  ```
  
  ## Metrics
  Evaluation on classification accuracy (entailment, contradiction, neutral) on MNLI test set:
  
  | Metric | Value |
  | --- | --- |
  | Accuracy | 0.8338|
  
  See Training Metrics tab for detailed info.