title: Learning the Latent Rules of a Game from Data: A Chess Story
summary: We demonstrate that small pretrained foundational generative language models
  with millions of parameters can learn the latent rules of a process from data
  associated with the process. Inspired by Stefan Zweig's novella
  "Schachnovelle," also known as "The Royal Game" in English, we show that 28M
  and 125M parameter pretrained foundational small language models (SLMs) can be
  instruction fine-tuned with 1,000-to-1,000,000 examples to learn the rules of
  chess, propose legal moves, and accurately solve chess problems. We also
  explore the impact of successive language model fine-tuning epochs on improved
  outcomes and demonstrate reductions in model hallucinations by increasing the
  number of instruction fine-tuning examples.
paper_authors._id: 66ff3f5c1e307d098692d137
paper_authors.name: Ben Fauber
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-04T01:05:37.180Z
paper_authors._id: 6331bab56eae0bb0a01968ea
paper_authors.avatarUrl: /avatars/20848394b32d9f2d0037ccfd1050e7bb.svg
paper_authors.fullname: BF
paper_authors.type: user
paper_authors.user: BFauber
paper_title: Learning the Latent Rules of a Game from Data: A Chess Story
paper_summary: We demonstrate that small pretrained foundational generative language models
  with millions of parameters can learn the latent rules of a process from data
  associated with the process. Inspired by Stefan Zweig's novella
  "Schachnovelle," also known as "The Royal Game" in English, we show that 28M
  and 125M parameter pretrained foundational small language models (SLMs) can be
  instruction fine-tuned with 1,000-to-1,000,000 examples to learn the rules of
  chess, propose legal moves, and accurately solve chess problems. We also
  explore the impact of successive language model fine-tuning epochs on improved
  outcomes and demonstrate reductions in model hallucinations by increasing the
  number of instruction fine-tuning examples.
paper_ai_summary: Pretrained small language models can be fine-tuned to learn and propose legal chess moves through instruction fine-tuning with varying numbers of examples.
paper_ai_keywords: pretrained foundational generative language models SLMs instruction fine-tuning model hallucinations