SuperWriter: Reflection-Driven Long-Form Generation with Large Language
  Models
Long-form text generation remains a significant challenge for large language
models (LLMs), particularly in maintaining coherence, ensuring logical
consistency, and preserving text quality as sequence length increases. To
address these limitations, we propose SuperWriter-Agent, an agent-based
framework designed to enhance the quality and consistency of long-form text
generation. SuperWriter-Agent introduces explicit structured thinking-through
planning and refinement stages into the generation pipeline, guiding the model
to follow a more deliberate and cognitively grounded process akin to that of a
professional writer. Based on this framework, we construct a supervised
fine-tuning dataset to train a 7B SuperWriter-LM. We further develop a
hierarchical Direct Preference Optimization (DPO) procedure that uses Monte
Carlo Tree Search (MCTS) to propagate final quality assessments and optimize
each generation step accordingly. Empirical results across diverse benchmarks
demonstrate that SuperWriter-LM achieves state-of-the-art performance,
surpassing even larger-scale baseline models in both automatic evaluation and
human evaluation. Furthermore, comprehensive ablation studies demonstrate the
effectiveness of hierarchical DPO and underscore the value of incorporating
structured thinking steps to improve the quality of long-form text generation.
6840fefb3098ab525906d852
Yuhao Wu
6840fefb3098ab525906d853
Yushi Bai
admin_assigned
2025-06-05T12:46:26.352Z
64ed568ccf6118a9379a61b8
/avatars/6d040cbcb4a9b624cbe64c9d01cd5c88.svg
Yushi Bai
user
bys0318
6840fefb3098ab525906d854
Zhiqiang Hu
claimed_verified
2025-06-05T07:27:09.305Z
637f228152229c63921119c3
https://cdn-avatars.huggingface.co/v1/production/uploads/637f228152229c63921119c3/acwXorra1r9_7i3KlBFjS.jpeg
Zhiqiang Hu
user
Zhiqiang007
6840fefb3098ab525906d855
Juanzi Li
6840fefb3098ab525906d856
Roy Ka-Wei Lee
admin_assigned
2025-06-05T12:46:33.575Z
64b94cc0e3d41dbd6974ae45
/avatars/5edb9d9465addfceccef04c4465a34e6.svg
Roy Ka-Wei lee
user
sroylee
SuperWriter: Reflection-Driven Long-Form Generation with Large Language
  Models
Long-form text generation remains a significant challenge for large language
models (LLMs), particularly in maintaining coherence, ensuring logical
consistency, and preserving text quality as sequence length increases. To
address these limitations, we propose SuperWriter-Agent, an agent-based
framework designed to enhance the quality and consistency of long-form text
generation. SuperWriter-Agent introduces explicit structured thinking-through
planning and refinement stages into the generation pipeline, guiding the model
to follow a more deliberate and cognitively grounded process akin to that of a
professional writer. Based on this framework, we construct a supervised
fine-tuning dataset to train a 7B SuperWriter-LM. We further develop a
hierarchical Direct Preference Optimization (DPO) procedure that uses Monte
Carlo Tree Search (MCTS) to propagate final quality assessments and optimize
each generation step accordingly. Empirical results across diverse benchmarks
demonstrate that SuperWriter-LM achieves state-of-the-art performance,
surpassing even larger-scale baseline models in both automatic evaluation and
human evaluation. Furthermore, comprehensive ablation studies demonstrate the
effectiveness of hierarchical DPO and underscore the value of incorporating
structured thinking steps to improve the quality of long-form text generation.
SuperWriter-Agent enhances long-form text generation by integrating structured planning and refinement, achieving top performance with a 7B model and hierarchical Direct Preference Optimization.
agent-based framework structured thinking-through planning refinement stages SuperWriter-Agent SuperWriter-LM hierarchical Direct Preference Optimization Monte Carlo Tree Search DPO automatic evaluation human evaluation ablation studies