This is a SmolLM2-135M-Instruct model fine-tuned on the Icelandic portion of Fineweb-2. It is intended for my research and has not been evaluated more broadly yet.

Training:

  • 1 Epoch
  • Learning rate: 5e-4
  • LR scheduler: Cosine
  • Warmup ratio: 0.05
  • Batch size: 1
  • 4 A100 (40GB) GPUs
  • Gradient accumulation steps: 64
  • Effective batch size: 256
  • Max. context length: 8192 tokens
Downloads last month
14
Safetensors
Model size
135M params
Tensor type
F32
·
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API: The model has no library tag.

Model tree for jekunz/smollm-135m-cpt-fineweb-icelandic

Finetuned
(101)
this model

Dataset used to train jekunz/smollm-135m-cpt-fineweb-icelandic

Collection including jekunz/smollm-135m-cpt-fineweb-icelandic