title: How Instruction and Reasoning Data shape Post-Training: Data Quality
    through the Lens of Layer-wise Gradients
summary: As the post-training of large language models (LLMs) advances from
  instruction-following to complex reasoning tasks, understanding how different
  data affect finetuning dynamics remains largely unexplored. In this paper, we
  present a spectral analysis of layer-wise gradients induced by low/high-quality
  instruction and reasoning data for LLM post-training. Our analysis reveals that
  widely-studied metrics for data evaluation, e.g., IFD, InsTag, Difficulty, and
  Reward, can be explained and unified by spectral properties computed from
  gradients' singular value decomposition (SVD). Specifically, higher-quality
  data are usually associated with lower nuclear norms and higher effective
  ranks. Notably, effective rank exhibits better robustness and resolution than
  nuclear norm in capturing subtle quality differences. For example, reasoning
  data achieves substantially higher effective ranks than instruction data,
  implying richer gradient structures on more complex tasks. Our experiments also
  highlight that models within the same family share similar gradient patterns
  regardless of their sizes, whereas different model families diverge
  significantly. Providing a unified view on the effects of data quality across
  instruction and reasoning data, this work illuminates the interplay between
  data quality and training stability, shedding novel insights into developing
  better data exploration strategies for post-training.
paper_authors._id: 67ff114a3026f8abc4bf7e43
paper_authors.name: Ming Li
paper_authors._id: 67ff114a3026f8abc4bf7e44
paper_authors.name: Yanhong Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-17T08:37:48.885Z
paper_authors._id: 642082a1b286e8c464f5d45d
paper_authors.avatarUrl: /avatars/7d87f6f81991b2c4bae788560d7c1352.svg
paper_authors.fullname: Yanhong Li
paper_authors.type: user
paper_authors.user: yanhong-li
paper_authors._id: 67ff114a3026f8abc4bf7e45
paper_authors.name: Ziyue Li
paper_authors._id: 67ff114a3026f8abc4bf7e46
paper_authors.name: Tianyi Zhou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-16T08:46:19.086Z
paper_authors._id: 647f5af5b0e96764589f3b2a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/VJ4cDyjp5M3V5WmI5gPIU.jpeg
paper_authors.fullname: Tianyi Zhou
paper_authors.type: user
paper_authors.user: zhoutianyi
paper_title: How Instruction and Reasoning Data shape Post-Training: Data Quality
    through the Lens of Layer-wise Gradients
paper_summary: As the post-training of large language models (LLMs) advances from
  instruction-following to complex reasoning tasks, understanding how different
  data affect finetuning dynamics remains largely unexplored. In this paper, we
  present a spectral analysis of layer-wise gradients induced by low/high-quality
  instruction and reasoning data for LLM post-training. Our analysis reveals that
  widely-studied metrics for data evaluation, e.g., IFD, InsTag, Difficulty, and
  Reward, can be explained and unified by spectral properties computed from
  gradients' singular value decomposition (SVD). Specifically, higher-quality
  data are usually associated with lower nuclear norms and higher effective
  ranks. Notably, effective rank exhibits better robustness and resolution than
  nuclear norm in capturing subtle quality differences. For example, reasoning
  data achieves substantially higher effective ranks than instruction data,
  implying richer gradient structures on more complex tasks. Our experiments also
  highlight that models within the same family share similar gradient patterns
  regardless of their sizes, whereas different model families diverge
  significantly. Providing a unified view on the effects of data quality across
  instruction and reasoning data, this work illuminates the interplay between
  data quality and training stability, shedding novel insights into developing
  better data exploration strategies for post-training.
paper_ai_summary: A spectral analysis of gradients during the post-training of large language models reveals the impact of data quality using metrics like nuclear norms and effective ranks, providing insights into improving data exploration strategies.
paper_ai_keywords: spectral analysis layer-wise gradients singular value decomposition SVD nuclear norms effective ranks gradient structures