readme: ---
  license: mit
  datasets:
  - McGill-NLP/FaithDial
  widget:
  - text: "A cardigan is a type of knitted garment (sweater) that has an open front. </s></s> The old version is the regular one, knitted garment that has open front and buttons!"
  model-index:
  - name: roberta-large-faithcritic
    results:
    - task:
        type: text-classification
        name: Faithfulness Critic
      dataset:
        name: 'FaithCritic'
        type: McGill-NLP/FaithDial
      metrics:
      - name: Accuracy
        type: accuracy
        value: 86.51
    - task:
        type: text-classification
        name: Faithfulness Critic
      dataset:
        name: 'MNLI'
        type: multi_nli
      metrics:
      - name: Accuracy
        type: accuracy
        value: 74.720
    - task:
        type: text-classification
        name: Faithfulness Critic
      dataset:
        name: 'BEGIN'
        type: begin
      metrics:
      - name: Accuracy
        type: accuracy
        value: 71.56
  ---
  
  ## Overview
  
  **Model Description:** roberta-large-faithcritic is the [RoBERTa large model](https://huggingface.co/roberta-large) fine-tuned on FaithCritic, a derivative of the [FaithDial](https://huggingface.co/datasets/McGill-NLP/FaithDial) dataset. The objective is to predict whether an utterance is faithful or not, given the source knowledge.
  
  The hyperparameters are provided in [hparams.yml](https://huggingface.co/McGill-NLP/roberta-large-faithcritic/blob/main/hparams.yaml). To know more about how to train a critic model, visit [our repo](https://github.com/McGill-NLP/FaithDial).
  
  ## Usage
  
  ```python
  from transformers import AutoTokenizer, AutoModel
  
  tokenizer = AutoTokenizer.from_pretrained("McGill-NLP/roberta-large-faithcritic")
  model = AutoModel.from_pretrained("McGill-NLP/roberta-large-faithcritic")
  
  knowledge = "A cardigan is a type of knitted garment (sweater) that has an open front."
  response = "The old version is the regular one, knitted garment that has open front and buttons!"
  input = tokenizer(knowledge, response)
  output = model(**input)
  
  ```
  
  
  ## Citation Information
  
  ```bibtex
  @article{dziri2022faithdial,
    title={FaithDial: A Faithful Benchmark for Information-Seeking Dialogue},
    author={Dziri, Nouha and Kamalloo, Ehsan and Milton, Sivan and Zaiane, Osmar and Yu, Mo and Ponti, Edoardo and Reddy, Siva},
    journal={arXiv preprint, arXiv:2204.10757},
    year={2022},
    url={https://arxiv.org/abs/2204.10757}
  }
  ```
  