Update README.md
Browse files
README.md
CHANGED
@@ -32,7 +32,7 @@ This repository contains a custom-trained Hindi Causal Language Model designed f
|
|
32 |
- IndicGLUE (30K samples)
|
33 |
- Hindi literature (5K passages)
|
34 |
- **Tokenizer:** SentencePiece trained on Hindi text with vocab size of 16,000
|
35 |
-
- **Training Details:** 2 epochs, hidden size=768, num_layers=12, block_size=512, batch_size=64, learning_rate=5e-5, swiglu activation, rope positional encoding, and rms normalization
|
36 |
|
37 |
## How to Use
|
38 |
|
|
|
32 |
- IndicGLUE (30K samples)
|
33 |
- Hindi literature (5K passages)
|
34 |
- **Tokenizer:** SentencePiece trained on Hindi text with vocab size of 16,000
|
35 |
+
- **Training Details:** Trained on 4xL4 24GB VRAM GPUs for 8 hours. 2 epochs, hidden size=768, num_layers=12, block_size=512, batch_size=64, learning_rate=5e-5, swiglu activation, rope positional encoding, and rms normalization
|
36 |
|
37 |
## How to Use
|
38 |
|