title: Think Inside the JSON: Reinforcement Strategy for Strict LLM Schema
    Adherence
summary: In this paper, we address the challenge of enforcing strict schema adherence
  in large language model (LLM) generation by leveraging LLM reasoning
  capabilities. Building on the DeepSeek R1 reinforcement learning framework, our
  approach trains structured reasoning skills of a 1.5B parameter model through a
  novel pipeline that combines synthetic reasoning dataset construction with
  custom reward functions under Group Relative Policy Optimization (GRPO).
  Specifically, we first perform R1 reinforcement learning on a 20K sample
  unstructured-to-structured dataset, mirroring the original DeepSeek R1 methods,
  to establish core reasoning abilities. Subsequently, we performed supervised
  fine-tuning on a separate 10K reasoning sample dataset, focusing on refining
  schema adherence for downstream tasks. Despite the relatively modest training
  scope, requiring approximately 20 hours on an 8xH100 GPU cluster for GRPO
  training and 3 hours on 1xA100 for SFT, our model demonstrates robust
  performance in enforcing schema consistency. We compare our ThinkJSON approach
  against the original DeepSeek R1 (671B), distilled versions of DeepSeek R1
  (Qwen-1.5B and Qwen-7B), and Gemini 2.0 Flash (70B), showcasing its
  effectiveness in real-world applications. Our results underscore the practical
  utility of a resource-efficient framework for schema-constrained text
  generation.
paper_authors._id: 67bbe0520aabd5d571a723e7
paper_authors.name: Bhavik Agarwal
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-24T16:38:48.442Z
paper_authors._id: 6739c3675d150a0c7e3c0014
paper_authors.avatarUrl: /avatars/87a0d9d39b0c854c467a3cdd46fa0ce1.svg
paper_authors.fullname: Bhavik Agarwal
paper_authors.type: user
paper_authors.user: bhaviktheslider
paper_authors._id: 67bbe0520aabd5d571a723e8
paper_authors.name: Ishan Joshi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-24T15:55:31.357Z
paper_authors._id: 647e85b6e4d52fe0e0210718
paper_authors.avatarUrl: /avatars/ee9ed51b64486b898ce0b58b22db32d5.svg
paper_authors.fullname: Ishan Joshi
paper_authors.type: user
paper_authors.user: IshanJoshi
paper_authors._id: 67bbe0520aabd5d571a723e9
paper_authors.name: Viktoria Rojkova
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-24T15:55:40.802Z
paper_authors._id: 62c1b81e8b647bdc24f78027
paper_authors.avatarUrl: /avatars/b994d31a0c4161f715bb2153c0f0a83f.svg
paper_authors.fullname: Viktoria Rojkova
paper_authors.type: user
paper_authors.user: vrojkova
paper_title: Think Inside the JSON: Reinforcement Strategy for Strict LLM Schema
    Adherence
paper_summary: In this paper, we address the challenge of enforcing strict schema adherence
  in large language model (LLM) generation by leveraging LLM reasoning
  capabilities. Building on the DeepSeek R1 reinforcement learning framework, our
  approach trains structured reasoning skills of a 1.5B parameter model through a
  novel pipeline that combines synthetic reasoning dataset construction with
  custom reward functions under Group Relative Policy Optimization (GRPO).
  Specifically, we first perform R1 reinforcement learning on a 20K sample
  unstructured-to-structured dataset, mirroring the original DeepSeek R1 methods,
  to establish core reasoning abilities. Subsequently, we performed supervised
  fine-tuning on a separate 10K reasoning sample dataset, focusing on refining
  schema adherence for downstream tasks. Despite the relatively modest training
  scope, requiring approximately 20 hours on an 8xH100 GPU cluster for GRPO
  training and 3 hours on 1xA100 for SFT, our model demonstrates robust
  performance in enforcing schema consistency. We compare our ThinkJSON approach
  against the original DeepSeek R1 (671B), distilled versions of DeepSeek R1
  (Qwen-1.5B and Qwen-7B), and Gemini 2.0 Flash (70B), showcasing its
  effectiveness in real-world applications. Our results underscore the practical
  utility of a resource-efficient framework for schema-constrained text
  generation.
paper_ai_keywords: reinforcement learning DeepSeek R1 Group Relative Policy Optimization (GRPO) synthetic reasoning dataset custom reward functions supervised fine-tuning (SFT) schema adherence schema consistency ThinkJSON Qwen-1.5B Qwen-7B Gemini 2.0 Flash