title: SlimLM: An Efficient Small Language Model for On-Device Document
    Assistance
summary: While small language models (SLMs) show promises for mobile deployment, their
  real-world performance and applications on smartphones remains underexplored.
  We present SlimLM, a series of SLMs optimized for document assistance tasks on
  mobile devices. Through extensive experiments on a Samsung Galaxy S24, we
  identify the optimal trade-offs between model size (ranging from 125M to 7B
  parameters), context length, and inference time for efficient on-device
  processing. SlimLM is pre-trained on SlimPajama-627B and fine-tuned on
  DocAssist, our constructed dataset for summarization, question answering and
  suggestion tasks. Our smallest model demonstrates efficient performance on S24,
  while larger variants offer enhanced capabilities within mobile constraints. We
  evaluate SlimLM against existing SLMs, showing comparable or superior
  performance and offering a benchmark for future research in on-device language
  models. We also provide an Android application, offering practical insights
  into SLM deployment. Our findings provide valuable insights and illuminate the
  capabilities of running advanced language models on high-end smartphones,
  potentially reducing server costs and enhancing privacy through on-device
  processing.
paper_authors._id: 673c3346955070e4cfcac54f
paper_authors.name: Thang M. Pham
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-19T15:25:53.374Z
paper_authors._id: 600e4c4010a64b1c9f81ea94
paper_authors.avatarUrl: /avatars/d043adb5a89023eb6ef01c2ec457e8ca.svg
paper_authors.fullname: Thang M. Pham
paper_authors.type: user
paper_authors.user: pmthangk09
paper_authors._id: 673c3346955070e4cfcac550
paper_authors.name: Phat T. Nguyen
paper_authors._id: 673c3346955070e4cfcac551
paper_authors.name: Seunghyun Yoon
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-20T10:09:04.313Z
paper_authors._id: 62deee6401dd68486922d9a6
paper_authors.avatarUrl: /avatars/242350bb05ee452c2fb2de4fa8213aa7.svg
paper_authors.fullname: David Seunghyun Yoon
paper_authors.type: user
paper_authors.user: mysmilesh
paper_authors._id: 673c3346955070e4cfcac552
paper_authors.name: Viet Dac Lai
paper_authors._id: 673c3346955070e4cfcac553
paper_authors.name: Franck Dernoncourt
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-11-19T06:43:47.566Z
paper_authors._id: 62c5947524171688a9feb992
paper_authors.avatarUrl: /avatars/5a151713b9eae8dc566f5957acee3475.svg
paper_authors.fullname: Franck Dernoncourt
paper_authors.type: user
paper_authors.user: Franck-Dernoncourt
paper_authors._id: 673c3346955070e4cfcac554
paper_authors.name: Trung Bui
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-20T10:09:02.636Z
paper_authors._id: 6036eb46413a78f892e733a0
paper_authors.avatarUrl: /avatars/b15925f89631323c176ecbd0eab99feb.svg
paper_authors.fullname: Trung Bui
paper_authors.type: user
paper_authors.user: Trung
paper_title: SlimLM: An Efficient Small Language Model for On-Device Document
    Assistance
paper_summary: While small language models (SLMs) show promises for mobile deployment, their
  real-world performance and applications on smartphones remains underexplored.
  We present SlimLM, a series of SLMs optimized for document assistance tasks on
  mobile devices. Through extensive experiments on a Samsung Galaxy S24, we
  identify the optimal trade-offs between model size (ranging from 125M to 7B
  parameters), context length, and inference time for efficient on-device
  processing. SlimLM is pre-trained on SlimPajama-627B and fine-tuned on
  DocAssist, our constructed dataset for summarization, question answering and
  suggestion tasks. Our smallest model demonstrates efficient performance on S24,
  while larger variants offer enhanced capabilities within mobile constraints. We
  evaluate SlimLM against existing SLMs, showing comparable or superior
  performance and offering a benchmark for future research in on-device language
  models. We also provide an Android application, offering practical insights
  into SLM deployment. Our findings provide valuable insights and illuminate the
  capabilities of running advanced language models on high-end smartphones,
  potentially reducing server costs and enhancing privacy through on-device
  processing.
paper_ai_keywords: small language models (SLMs) document assistance tasks optimal trade-offs model size context length inference time efficient on-device processing pre-trained SlimPajama-627B fine-tuned DocAssist summarization question answering suggestion tasks benchmark on-device language models Android application on-device processing