region:us
---
{}
---
**Seq2Seq-Based English-to-Georgian Translation Model**



**1. Project Overview**
This project implements a Seq2Seq translation model for converting English text to Georgian. It utilizes an LSTM-based encoder-decoder architecture and evaluates performance using BLEU and ChrF metrics.

**2. Requirements**
The following libraries are required:
•	HuggingFace's datasets for loading the OPUS100 dataset
•	transformers for multilingual tokenization
•	PyTorch for building and training the Seq2Seq model
•	evaluate for BLEU and ChrF metrics
•	sacrebleu for additional BLEU metric evaluation


**3. Dataset**
•	Source: OPUS100 Dataset (English-Georgian translation pair) from HuggingFace.
•	Splits: Training, Validation, and Test datasets.

**4. Preprocessing**
•	Tokenization: Sentences are tokenized using the bert-base-multilingual-cased tokenizer.
•	Truncation and Padding: Applied with a maximum token length of 128.
•	Batch Processing: Data is processed in batches for efficient training and evaluation.

**5. Model Architecture**
The Seq2Seq model is a custom LSTM-based encoder-decoder architecture with the following components:
1.	Embedding Layer: Converts tokens into dense vector representations.
2.	LSTM Encoder: Encodes input sequences into context vectors.
3.	LSTM Decoder: Decodes context vectors into output sequences.
4.	Fully Connected Layer: Maps decoder outputs to the target vocabulary.
Key Parameters:
•	Vocabulary Size: English + Georgian tokens.
•	Hidden Dimension: 256.
•	LSTM Layers: 2.

**6. Training**
•	Optimizer: Adam with a learning rate of 0.001.
•	Loss Function: CrossEntropyLoss.
•	Epochs: 10.
•	Batch Size: 8.
Training includes a teacher forcing ratio of 0.5 to guide the decoder with ground truth inputs during the early stages.

**7. Evaluation**
Performance is evaluated using:
1.	BLEU Score: Measures word-level translation quality.
2.	ChrF Score: Measures character-level translation quality.
Scores are computed after each epoch and saved in CSV files (seq2seq_bleu_scores.csv and seq2seq_chrf_scores.csv) for analysis.

**8. Inference**
The trained model is used for inference by loading saved checkpoints (seq2seq_model_state_dict.pth). Translations are generated by passing English input sequences to the model.

**9. Visualization**
•	Loss Curves: Training and validation loss curves are plotted to analyze convergence and detect overfitting.
•	BLEU Scores: Plotted across epochs to visualize translation quality improvement.
•	ChrF Scores: Plotted across epochs to assess character-level accuracy.

**10. Key Observations**
•	Loss: Gradual reduction in training and validation loss indicates learning progress.
•	BLEU & ChrF Scores: Consistent improvement reflects enhanced translation quality.
•	Convergence: Observed through decreasing validation loss and increasing evaluation scores.

**11. Future Enhancements**
1.	Introduce attention mechanisms for better long-sequence translations.
2.	Explore deeper or bidirectional LSTM architectures.
3.	Compare with Transformer-based architectures like BERT or GPT for improved performance.
