Aharshms commited on
Commit
a486ac0
·
verified ·
1 Parent(s): 4309bf8

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -10
README.md CHANGED
@@ -44,17 +44,15 @@ The following hyperparameters were used during training:
44
  - lr_scheduler_warmup_ratio: 0.1
45
  - num_epochs: 3.0
46
 
47
- ### Training results
48
 
49
- ### Performance Comparison on ARCC Dataset
50
-
51
- | Benchmark | Gemma-2-2B | Llama-3.2-3B | Llama-3.1-8B | Sarvam-1 | Hex-1 |
52
- |-----------|------------|--------------|---------------|-----------|--------|
53
- | arcc_hi | 37.57 | 49.13 | 56.17 | 60.00 | 36.68 |
54
- | arcc_ta | 32.78 | 34.70 | 44.78 | 57.04 | 38.65 |
55
- | arcc_te | 30.00 | 34.09 | 43.04 | 59.39 | 37.96 |
56
- | arcc_kn | 29.22 | 36.43 | 44.70 | 57.04 | 38.31 |
57
- | arcc_ml | 29.91 | 33.22 | 46.78 | 58.96 | 29.60 |
58
 
59
 
60
 
 
44
  - lr_scheduler_warmup_ratio: 0.1
45
  - num_epochs: 3.0
46
 
47
+ ### Training results - Multilingual Task Performance Comparison
48
 
49
+ | Language | Hellaswag | ARC-c | ARC-e | MMLU | BoolQ |
50
+ |------------|-----------|--------|--------|--------|--------|
51
+ | Hindi | 47.85 | 36.68 | 52.14 | 46.73 | 57.61 |
52
+ | Tamil | 49.45 | 38.65 | 53.45 | 44.71 | 45.87 |
53
+ | Telugu | 50.84 | 37.96 | 53.36 | 46.85 | 51.89 |
54
+ | Kannada | 52.16 | 38.31 | 53.11 | 46.38 | 52.32 |
55
+ | Malayalam | 46.32 | 29.60 | 40.86 | 43.63 | 46.69 |
 
 
56
 
57
 
58