title: WritingBench: A Comprehensive Benchmark for Generative Writing
summary: Recent advancements in large language models (LLMs) have significantly
  enhanced text generation capabilities, yet evaluating their performance in
  generative writing remains a challenge. Existing benchmarks primarily focus on
  generic text generation or limited in writing tasks, failing to capture the
  diverse requirements of high-quality written contents across various domains.
  To bridge this gap, we present WritingBench, a comprehensive benchmark designed
  to evaluate LLMs across 6 core writing domains and 100 subdomains, encompassing
  creative, persuasive, informative, and technical writing. We further propose a
  query-dependent evaluation framework that empowers LLMs to dynamically generate
  instance-specific assessment criteria. This framework is complemented by a
  fine-tuned critic model for criteria-aware scoring, enabling evaluations in
  style, format and length. The framework's validity is further demonstrated by
  its data curation capability, which enables 7B-parameter models to approach
  state-of-the-art (SOTA) performance. We open-source the benchmark, along with
  evaluation tools and modular framework components, to advance the development
  of LLMs in writing.
paper_authors._id: 67cfebe18a4265f3656a50aa
paper_authors.name: Yuning Wu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-11T08:24:29.900Z
paper_authors._id: 642d430a7f9efee76b8713c0
paper_authors.avatarUrl: /avatars/4981f166a6df8e2ea60cd4c41c2f44d4.svg
paper_authors.fullname: YuningWu
paper_authors.type: user
paper_authors.user: AQuarterMile
paper_authors._id: 67cfebe18a4265f3656a50ab
paper_authors.name: Jiahao Mei
paper_authors._id: 67cfebe18a4265f3656a50ac
paper_authors.name: Ming Yan
paper_authors._id: 67cfebe18a4265f3656a50ad
paper_authors.name: Chenliang Li
paper_authors._id: 67cfebe18a4265f3656a50ae
paper_authors.name: SHaopeng Lai
paper_authors._id: 67cfebe18a4265f3656a50af
paper_authors.name: Yuran Ren
paper_authors._id: 67cfebe18a4265f3656a50b0
paper_authors.name: Zijia Wang
paper_authors._id: 67cfebe18a4265f3656a50b1
paper_authors.name: Ji Zhang
paper_authors._id: 67cfebe18a4265f3656a50b2
paper_authors.name: Mengyue Wu
paper_authors._id: 67cfebe18a4265f3656a50b3
paper_authors.name: Qin Jin
paper_authors._id: 67cfebe18a4265f3656a50b4
paper_authors.name: Fei Huang
paper_title: WritingBench: A Comprehensive Benchmark for Generative Writing
paper_summary: Recent advancements in large language models (LLMs) have significantly
  enhanced text generation capabilities, yet evaluating their performance in
  generative writing remains a challenge. Existing benchmarks primarily focus on
  generic text generation or limited in writing tasks, failing to capture the
  diverse requirements of high-quality written contents across various domains.
  To bridge this gap, we present WritingBench, a comprehensive benchmark designed
  to evaluate LLMs across 6 core writing domains and 100 subdomains, encompassing
  creative, persuasive, informative, and technical writing. We further propose a
  query-dependent evaluation framework that empowers LLMs to dynamically generate
  instance-specific assessment criteria. This framework is complemented by a
  fine-tuned critic model for criteria-aware scoring, enabling evaluations in
  style, format and length. The framework's validity is further demonstrated by
  its data curation capability, which enables 7B-parameter models to approach
  state-of-the-art (SOTA) performance. We open-source the benchmark, along with
  evaluation tools and modular framework components, to advance the development
  of LLMs in writing.
paper_ai_summary: WritingBench is a comprehensive benchmark that evaluates large language models across diverse writing domains using a query-dependent framework and a fine-tuned critic model for criteria-aware scoring.
paper_ai_keywords: large language models WritingBench generative writing query-dependent evaluation fine-tuned critic model state-of-the-art