safetensors llama text-generation conversational dataset:voidful/mixed_inst_multi_chat base_model:HuggingFaceTB/SmolLM2-360M base_model:finetune:HuggingFaceTB/SmolLM2-360M license:apache-2.0 region:us
---
base_model:
- HuggingFaceTB/SmolLM2-360M
datasets:
- voidful/mixed_inst_multi_chat
license: apache-2.0
pipeline_tag: text-generation
---


Model Summary

SmolLM2 is a family of compact language models available in three size: 135M, 360M, and 1.7B parameters. They are capable of solving a wide range of tasks while being lightweight enough to run on-device.

SmolLM2 demonstrates significant advances over its predecessor SmolLM1, particularly in instruction following, knowledge, reasoning. The 360M model was trained on 4 trillion tokens using a diverse dataset combination: FineWeb-Edu, DCLM, The Stack, along with new filtered datasets we curated and will release soon. We developed the instruct version through supervised fine-tuning (SFT) using a combination of public datasets and our own curated datasets. We then applied Direct Preference Optimization (DPO) using UltraFeedback.

The instruct model additionally supports tasks such as text rewriting, summarization and function calling thanks to datasets developed by Argilla such as Synth-APIGen-v0.1.

For more details refer to: https://github.com/huggingface/smollm. You will find pre-training, post-training, evaluation and local inference code.