Hugging Face's logo

MiniLLM

community
https://github.com/microsoft/LMOps/tree/main/minillm
t1101675
Activity Feed

AI & ML interests

Training efficient language models (MiniLLM, MiniPLM)

Recent Activity

t1101675  updated a model 14 days ago
MiniLLM/MiniLLM-gpt2-340M
t1101675  new activity about 1 month ago
MiniLLM/MiniLLM-gpt2-340M:Adding `safetensors` variant of this model
t1101675  new activity about 1 month ago
MiniLLM/SFT-gpt2-120M:Adding `safetensors` variant of this model
View all activity

Organization Card
Community About org cards

Training Small Language Models with Knowledge Distillation

Official pre-trained models and baselines in

  • MiniLLM: Knowledge distillation of LLMs during instruction tuning.
  • MiniPLM: Knowledge distillation of LLMs during pre-training.

Collections 2