title: Slamming: Training a Speech Language Model on One GPU in a Day
summary: We introduce Slam, a recipe for training high-quality Speech Language Models
  (SLMs) on a single academic GPU in 24 hours. We do so through empirical
  analysis of model initialisation and architecture, synthetic training data,
  preference optimisation with synthetic data and tweaking all other components.
  We empirically demonstrate that this training recipe also scales well with more
  compute getting results on par with leading SLMs in a fraction of the compute
  cost. We hope these insights will make SLM training and research more
  accessible. In the context of SLM scaling laws, our results far outperform
  predicted compute optimal performance, giving an optimistic view to SLM
  feasibility. See code, data, models, samples at -
  https://pages.cs.huji.ac.il/adiyoss-lab/slamming .
paper_authors._id: 67bd3972f077ddf1f98bacda
paper_authors.name: Gallil Maimon
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-25T09:39:36.258Z
paper_authors._id: 66b9bc2dacdbc1d0b39c3b50
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/hwR0pVfP_E8XjimXIxDOU.jpeg
paper_authors.fullname: Gallil Maimon
paper_authors.type: user
paper_authors.user: gallilmaimon
paper_authors._id: 67bd3972f077ddf1f98bacdb
paper_authors.name: Avishai Elmakies
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-25T09:39:33.712Z
paper_authors._id: 644662145004f2cb3af08b27
paper_authors.avatarUrl: /avatars/5f2af24c7410a5db46374d0b84fb479d.svg
paper_authors.fullname: Avishai Elmakies
paper_authors.type: user
paper_authors.user: avishai-elmakies
paper_authors._id: 67bd3972f077ddf1f98bacdc
paper_authors.name: Yossi Adi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-25T16:02:07.243Z
paper_authors._id: 6481e135578646b5c2386728
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6481e135578646b5c2386728/SPva4iNw0pORiCXD45cx9.jpeg
paper_authors.fullname: Yossi Adi
paper_authors.type: user
paper_authors.user: adiyoss
paper_title: Slamming: Training a Speech Language Model on One GPU in a Day
paper_summary: We introduce Slam, a recipe for training high-quality Speech Language Models
  (SLMs) on a single academic GPU in 24 hours. We do so through empirical
  analysis of model initialisation and architecture, synthetic training data,
  preference optimisation with synthetic data and tweaking all other components.
  We empirically demonstrate that this training recipe also scales well with more
  compute getting results on par with leading SLMs in a fraction of the compute
  cost. We hope these insights will make SLM training and research more
  accessible. In the context of SLM scaling laws, our results far outperform
  predicted compute optimal performance, giving an optimistic view to SLM
  feasibility. See code, data, models, samples at -
  https://pages.cs.huji.ac.il/adiyoss-lab/slamming .
paper_ai_summary: A method for training high-quality Speech Language Models on a single GPU within 24 hours, demonstrating cost-effectiveness and outperforming predicted computational scaling laws.
paper_ai_keywords: Slam Speech Language Models model initialization architecture synthetic training data preference optimization