Update README.md

README.md

@@ -12,3 +12,18 @@ A monolingual Slovak language model.
 
 Model was trained on a collection of Slovak web pages from various sources.
 
+## Training parameters
+
+We used 4 x A100 40GB GPU for 14 hours.
+
+- Effective batch size: 192
+- Sequence length 512
+- Training Steps 120 000.
+- warmup_steps 1000 
+- optimizer adamw 
+- Per device batch size 48
+- mixed_precision bf16
+- weight decay 0.01
+- gradient clipping 1.0 
+- learning_rate 1e-5
+- scheduler cosine