region:us
---
{}
---
# **LSTM Translation with Bidirectional LSTM**

This project implements a translation model using a **Bidirectional LSTM** model. This repository contains the trained LSTM models for English to Haitian Creole translation and viceversa.

## Project Overview
* **Tokenizer**: BART-base tokenizer from Hugging Face's transformers library
* **Model**: Bidirectional Long Short-Term Memory (LSTM) neural network
* **Task**: Translation between English and Haitian Creole and vice vera
* **Languages**: The project uses **Haitian Creole** and **English** for training the model

## Repository Structure
This repository contains the below files:
1. `model_en_to_ht.pth`: The trained LSTM model for English to Haitian translation
2. `model_ht_to_en.pth`: The trained LSTM model for Haitian to English translation
3. `README.md`: This file, explaining the model

## Model Architecture
The LSTM Translator consists of:
* **Embedding Layer**: Converts tokens to vectors
* **Bidirectional LSTM**: Processes sequences in both directions
* **Fully Connected Layer**: Maps to output vocabulary

Key Parameters:
* **Embedding Dimension**: 128
* **Hidden Size**: 256
* **Batch Size**: 128
* **Block Size (Sequence Length)**: 16
* **Dropout**: 0.2
* **Learning Rate**: 1e-3
* **Number of Epochs**: 10

### Translation Metrics
Both BLEU and ChrF scores were tracked during training to measure the model's performance

#### BLEU Scores
* **LSTM (Ht->En)**: Starts high at 0.55, stabilizes around 0.15
* **LSTM (En->Ht)**: Shows consistent performance near 0

#### ChrF Scores
* **LSTM (En->Ht)**: Stabilizes around 2.5-3.0
* **LSTM (Ht->En)**: Gradually decreases and stabilizes around 2.5

### Training and Validation Losses
The training and validation losses were recorded throughout the training process. The model shows:
* Stable training behavior
* Better performance in Haitian to English direction
* Consistent validation metrics
* Moderate overall translation quality

## Dataset
* **Training Set**: 16,000 sentence pairs
* **Validation Set**: 4,000 sentence pairs
* **Data Format**: JSON with parallel text pairs
* **Tokenization**: BERT tokenizer from Hugging Face

## Limitations
1. Fixed sequence length
2. No attention mechanism
3. Memory constraints with long sequences
4. Basic architecture compared to modern standards
