region:us
---
{}
---
# Georgian Language Question-Answering Model

## Project Overview
This project fine-tunes a pre-trained GPT-2 model for a Georgian language question-answering task. The model generates answers to questions in Georgian using a dataset containing question-answer pairs. The fine-tuned GPT-2 model is evaluated using common NLP metrics such as BLEU and ROUGE. The trained model can be used for automatic question answering in Georgian.

## Key Features
- Fine-tuned GPT-2 model for Georgian question-answering.
- Trained on the MrDimitri/georgian-quora-faq dataset.
- Evaluation using metrics such as BLEU and ROUGE.
- Predictions are generated for test samples and compared against ground truth answers.

## Requirements
- Python 3.6+
- PyTorch 1.9+ (for GPU support)
- Hugging Face transformers and datasets library
- Google Colab (recommended for GPU access)
- SacreBLEU and Rouge-Score libraries for evaluation metrics.

## Dataset
The dataset used for training is `MrDimitri/georgian-quora-faq` from Hugging Face. It contains a collection of question-answer pairs in Georgian, ideal for training a question-answering model.

### Dataset Structure
The dataset consists of two columns:
- **question**: The input question.
- **answer**: The expected answer.

## Model
The model architecture is based on GPT-2, a Causal Language Model. It is a transformer-based model that is fine-tuned for the task of question answering by generating answers in the Georgian language.

### Model Details
- **Pre-trained Model**: gpt2 (GPT-2 for general language modeling).
- **Tokenizer**: The model uses the GPT-2 tokenizer to preprocess input text.
- **Special Tokens**:
  - `pad_token_id`: Padding token ID to handle sequences of different lengths.
  - `eos_token_id`: End-of-sequence token used for sequence generation.

## Training Process

### Data Preprocessing
- Tokenization of input (question) and output (answer) sequences.
- Padding and truncation to a fixed length for model input.

### Model Fine-Tuning
- The model is trained for 100 epochs.
- A batch size of 16 for both training and evaluation.
- Loss function: Cross-entropy loss for sequence generation.

### Training Arguments
- **evaluation_strategy**: Evaluates the model at the end of each epoch.
- **num_train_epochs**: Number of training epochs (100).
- **save_strategy**: Saves the model at the end of each epoch.
- **learning_rate**: Set to 5e-5.
- **gradient_accumulation_steps**: Set to 2 for memory efficiency.

## Evaluation
## Metrics

### BLEU Score
A metric for evaluating text generation in machine translation and question answering tasks.

### ROUGE Scores
- **ROUGE-1**: Measures the overlap of unigrams.
- **ROUGE-2**: Measures the overlap of bigrams.
- **ROUGE-L**: Measures the longest common subsequence overlap.
