readme: ---
  tags:
  - Question Answering
  metrics:
  - rouge
  model-index:
  - name: question-answering-generative-t5-v1-base-s-q-c
    results: []
  ---
  
  # Question Answering Generative
  The model is intended to be used for Q&A task, given the question & context, the model would attempt to infer the answer text.<br>
  Model is generative (t5-v1-base), fine-tuned from [question-generation-auto-hints-t5-v1-base-s-q-c](https://huggingface.co/consciousAI/question-generation-auto-hints-t5-v1-base-s-q-c) with - **Loss:** 0.6751 & **Rougel:** 0.8022 performance scores.
  
  [Live Demo: Question Answering Encoders vs Generative](https://huggingface.co/spaces/consciousAI/question_answering)
  
  [Encoder based Question Answering V1](https://huggingface.co/consciousAI/question-answering-roberta-base-s/)
  <br>[Encoder based Question Answering V2](https://huggingface.co/consciousAI/question-answering-roberta-base-s-v2/)
  
  Example code:
  
  ```
  from transformers import (
      AutoModelForSeq2SeqLM,
      AutoTokenizer
  )
  
  def _generate(query, context, model, device):
      
      FT_MODEL = AutoModelForSeq2SeqLM.from_pretrained(model).to(device)
      FT_MODEL_TOKENIZER = AutoTokenizer.from_pretrained(model)
      input_text = "question: " + query + "</s> question_context: " + context
      
      input_tokenized = FT_MODEL_TOKENIZER.encode(input_text, return_tensors='pt', truncation=True, padding='max_length', max_length=1024).to(device)
      _tok_count_assessment = FT_MODEL_TOKENIZER.encode(input_text, return_tensors='pt', truncation=True).to(device)
  
      summary_ids = FT_MODEL.generate(input_tokenized, 
                                         max_length=30, 
                                         min_length=5, 
                                         num_beams=2,
                                         early_stopping=True,
                                     )
      output = [FT_MODEL_TOKENIZER.decode(id, clean_up_tokenization_spaces=True, skip_special_tokens=True) for id in summary_ids] 
      
      return str(output[0])
  
  device = [0 if torch.cuda.is_available() else 'cpu'][0]
  _generate(query, context, model="consciousAI/t5-v1-base-s-q-c-multi-task-qgen-v2", device=device)   
  ```
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 0.0003
  - train_batch_size: 3
  - eval_batch_size: 3
  - seed: 42
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - num_epochs: 5
  
  ### Training results
  
  | Training Loss | Epoch | Step  | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum |
  |:-------------:|:-----:|:-----:|:---------------:|:------:|:------:|:------:|:---------:|
  | 0.5479        | 1.0   | 14600 | 0.5104          | 0.7672 | 0.4898 | 0.7666 | 0.7666    |
  | 0.3647        | 2.0   | 29200 | 0.5180          | 0.7862 | 0.4995 | 0.7855 | 0.7858    |
  | 0.2458        | 3.0   | 43800 | 0.5302          | 0.7938 | 0.5039 | 0.7932 | 0.7935    |
  | 0.1532        | 4.0   | 58400 | 0.6024          | 0.7989 | 0.514  | 0.7984 | 0.7984    |
  | 0.0911        | 5.0   | 73000 | 0.6751          | 0.8028 | 0.5168 | 0.8022 | 0.8022    |
  
  
  ### Framework versions
  
  - Transformers 4.23.0.dev0
  - Pytorch 1.12.1+cu113
  - Datasets 2.5.2
  - Tokenizers 0.13.0
  