readme: ---
  language: 
  - en
  thumbnail: https://cogcomp.seas.upenn.edu/images/logo.png
  tags:
  - text-classification
  - bart
  - xsum
  license: cc-by-sa-4.0
  datasets:
  - xsum
  widget:
  - text: "<s> Ban Ki-moon was elected for a second term in 2007. </s></s> Ban Ki-Moon was re-elected for a second term by the UN General Assembly, unopposed and unanimously, on 21 June 2011."
  - text: "<s> Ban Ki-moon was elected for a second term in 2011. </s></s> Ban Ki-Moon was re-elected for a second term by the UN General Assembly, unopposed and unanimously, on 21 June 2011."
  
  ---
  
  # bart-faithful-summary-detector
  
  ## Model description
  
  A BART (base) model trained to classify whether a summary is *faithful* to the original article. See our [paper in NAACL'21](https://www.seas.upenn.edu/~sihaoc/static/pdf/CZSR21.pdf) for details. 
  
  ## Usage
  Concatenate a summary and a source document as input (note that the summary needs to be the **first** sentence). 
  
  Here's an example usage (with PyTorch)
  ```python
  from transformers import AutoTokenizer, AutoModelForSequenceClassification
    
  tokenizer = AutoTokenizer.from_pretrained("CogComp/bart-faithful-summary-detector")
  model = AutoModelForSequenceClassification.from_pretrained("CogComp/bart-faithful-summary-detector")
  
  article = "Ban Ki-Moon was re-elected for a second term by the UN General Assembly, unopposed and unanimously, on 21 June 2011."
  
  bad_summary = "Ban Ki-moon was elected for a second term in 2007."
  good_summary = "Ban Ki-moon was elected for a second term in 2011."
  
  bad_pair = tokenizer(text=bad_summary, text_pair=article, return_tensors='pt')
  good_pair = tokenizer(text=good_summary, text_pair=article, return_tensors='pt')
  
  bad_score = model(**bad_pair)
  good_score = model(**good_pair)
  
  print(good_score[0][:, 1] > bad_score[0][:, 1]) # True, label mapping: "0" -> "Hallucinated" "1" -> "Faithful"
  ```
  
  
  
  
  ### BibTeX entry and citation info
  
  ```bibtex
  @inproceedings{CZSR21,
      author = {Sihao Chen and Fan Zhang and Kazoo Sone and Dan Roth},
      title = {{Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection}},
      booktitle = {NAACL},
      year = {2021}
  }
  ```