Update README.md

README.md

@@ -16,7 +16,7 @@ This repository contains a custom-trained Hindi Causal Language Model designed f
 ## Model Description
 - **Model Size:** 113M (YAH !!! Its very small)
 
-- **Architecture:** Custom Transformer (12 layers, hidden=768, 16 heads, ffn=3072, act=geglu, norm=rmsnorm) based on the `HindiCausalLM` class with Hindi-specific optimizations:
+- **Architecture:** Custom Transformer (12 layers, hidden=768, 16 heads, ffn=3072, act=swiglu, norm=rmsnorm) based on the `HindiCausalLM` class with Hindi-specific optimizations:
   - Multi-resolution attention to capture both character-level and word-level patterns
   - Morphology-aware feed-forward layers
   - Script-mix processing for Hindi-English code-mixing
@@ -32,7 +32,7 @@ This repository contains a custom-trained Hindi Causal Language Model designed f
   - IndicGLUE (30K samples)
   - Hindi literature (5K passages)
 - **Tokenizer:** SentencePiece trained on Hindi text with vocab size of 16,000
-- **Training Details:** Trained on 4xL4 24GB VRAM GPUs for 8 hours. 2 epochs, hidden size=768, num_layers=12, block_size=512, batch_size=64, learning_rate=5e-5, geglu activation, rope positional encoding, and rms normalization
+- **Training Details:** Trained on 4xL4 24GB VRAM GPUs for 8 hours. 2 epochs, hidden size=768, num_layers=12, block_size=512, batch_size=64, learning_rate=5e-5, swiglu activation, rope positional encoding, and rms normalization
 
 ## How to Use