readme: ---
  language: en
  tags:
  - qa
  - classification
  - question
  - answering
  - SQuAD
  - metric
  - nlg
  - t5-small
  license: mit
  datasets:
  - squad
  - cnndm
  model-index:
  - name: t5-weighter_cnndm-en
    results:
    - task:
        name: Classification
        type: Question Weighter
  widget:
     - text: "a Buckingham Palace guard </s> Who felt on a manhole? </s> This is the embarrassing moment a Buckingham Palace guard slipped and fell on a manhole cover in front of hundreds of shocked tourists as he took up position in his sentry box. [...] The Guard comprises two detachments, one each for Buckingham Palace and St James’s Palace, under the command of the Captain of The Queen’s Guard."
  ---
  # t5-weighter_cnndm-en
  
  ## Model description
  This model is a *Classifier* model based on T5-small, that predicts if a answer / question couple is considered as important fact or not (Is this answer enough relevant to appear in a plausible summary?).
  It is actually a component of [QuestEval](https://github.com/ThomasScialom/QuestEval) metric but can be used independently as it is.
  
  
  ## How to use
  ```python
  from transformers import T5Tokenizer, T5ForConditionalGeneration
  
  tokenizer = T5Tokenizer.from_pretrained("ThomasNLG/t5-weighter_cnndm-en")
  
  model = T5ForConditionalGeneration.from_pretrained("ThomasNLG/t5-weighter_cnndm-en")
  ```
  
  You can play with the model using the inference API, the text input format should follow this template (accordingly to the training stage of the model):
  
  `text_input = "{ANSWER} </s> {QUESTION} </s> {CONTEXT}"`
  
  ## Training data
  The model was trained on synthetic data as described in [Questeval: Summarization asks for fact-based evaluation](https://arxiv.org/abs/2103.12693).
  
  ### Citation info
  
  ```bibtex
  @article{scialom2021questeval,
    title={Questeval: Summarization asks for fact-based evaluation},
    author={Scialom, Thomas and Dray, Paul-Alexis and Gallinari, Patrick and Lamprier, Sylvain and Piwowarski, Benjamin and Staiano, Jacopo and Wang, Alex},
    journal={arXiv preprint arXiv:2103.12693},
    year={2021}
  }
  ```