readme: ---
  license: apache-2.0
  language: en
  library: transformers
  other: distilroberta
  datasets:
  - Short Question Answer Assessment Dataset 
  ---
  
  # DistilRoBERTa base model for Short Question Answer Assessment
  
  ## Model description
  
  The pre-trained model is a distilled version of the [RoBERTa-base model](https://huggingface.co/roberta-base). It follows the same training procedure as [DistilBERT](https://huggingface.co/distilbert-base-uncased).
  The code for the distillation process can be found [here](https://github.com/huggingface/transformers/tree/master/examples/distillation).
  This model is case-sensitive: it makes a difference between english and English.
  
  The model has 6 layers, 768 dimension and 12 heads, totalizing 82M parameters (compared to 125M parameters for RoBERTa-base).
  On average DistilRoBERTa is twice as fast as Roberta-base.
  
  We encourage to check [RoBERTa-base model](https://huggingface.co/roberta-base) to know more about usage, limitations and potential biases.
  
  
  
  This is a classification model that solves Short Question Answer Assessment task, finetuned [pretrained DistilRoBERTa model](https://huggingface.co/distilroberta-base) on 
  [Question Answer Assessment dataset](#)
  
  ## Intended uses & limitations
  
  This can only be used for the kind of questions and answers provided by that are similar to the ones in the dataset of [Banjade et al.](https://aclanthology.org/W16-0520.pdf).
  
  
  ### How to use
  
  You can use this model directly with a :
  
  ```python
  >>> from transformers import pipeline
  >>> classifier = pipeline("text-classification", model="Giyaseddin/distilroberta-base-finetuned-short-answer-assessment", return_all_scores=True)
  >>> context = "To rescue a child who has fallen down a well, rescue workers fasten him to a rope, the other end of which is then reeled in by a machine. The rope pulls the child straight upward at steady speed."
  >>> question = "How does the amount of tension in the rope compare to the downward force of gravity acting on the child?"
  >>> ref_answer = "Since the child is being raised straight upward at a constant speed, the net force on the child is zero and all the forces balance. That means that the tension in the rope balances the downward force of gravity."
  >>> student_answer = "The tension force is higher than the force of gravity."
  >>> 
  >>> body = " [SEP] ".join([context, question, ref_answer, student_answer])
  >>> raw_results = classifier([body])
  >>> raw_results
  [[{'label': 'LABEL_0', 'score': 0.0004029414849355817},
    {'label': 'LABEL_1', 'score': 0.0005476847873069346},
    {'label': 'LABEL_2', 'score': 0.998059093952179},
    {'label': 'LABEL_3', 'score': 0.0009902542224153876}]]
  >>> _LABELS_ID2NAME = {0: "correct", 1: "correct_but_incomplete", 2: "contradictory", 3: "incorrect"}
  >>> results = []
  >>> for result in raw_results:
      	for score in result:
          	results.append([
              	{_LABELS_ID2NAME[int(score["label"][-1:])]: "%.2f" % score["score"]}
          	])
  
  >>> results
  [[{'correct': '0.00'}],
   [{'correct_but_incomplete': '0.00'}],
   [{'contradictory': '1.00'}],
   [{'incorrect': '0.00'}]]
  ```
  
  ### Limitations and bias
  
  Even if the training data used for this model could be characterized as fairly neutral, this model can have biased
  predictions. It also inherits some of
  [the bias of its teacher model](https://huggingface.co/bert-base-uncased#limitations-and-bias).
  
  This bias will also affect all fine-tuned versions of this model.
  
  Also one of the limiations of this model is the length, longer sequences would lead to wrong predictions, due to the pre-processing phase (after concatentating the input sequences, the important student answer might be pruned!)
  
  ## Pre-training data
  
  ## Training data
  
  The RoBERTa model was pretrained on the reunion of five datasets:
  - [BookCorpus](https://yknzhu.wixsite.com/mbweb), a dataset consisting of 11,038 unpublished books;
  - [English Wikipedia](https://en.wikipedia.org/wiki/English_Wikipedia) (excluding lists, tables and headers) ;
  - [CC-News](https://commoncrawl.org/2016/10/news-dataset-available/), a dataset containing 63 millions English news
    articles crawled between September 2016 and February 2019.
  - [OpenWebText](https://github.com/jcpeterson/openwebtext), an opensource recreation of the WebText dataset used to
    train GPT-2,
  - [Stories](https://arxiv.org/abs/1806.02847) a dataset containing a subset of CommonCrawl data filtered to match the
    story-like style of Winograd schemas.
  
  Together theses datasets weight 160GB of text.
  
  ## Fine-tuning data
  
  The annotated dataset consists of 900 students’ short constructed answers and their correctness in the given context. Four qualitative levels of correctness are defined, correct, correct-but-incomplete, contradictory and Incorrect.
  
  
  ## Training procedure
  
  ### Preprocessing
  
  In the preprocessing phase, the following parts are concatenated: _question context_, _question_, _reference_answer_, and _student_answer_ using the separator `[SEP]`.
  This makes the full text as:
  
  ```
  [C