title: Weaver: Foundation Models for Creative Writing
summary: This work introduces Weaver, our first family of large language models (LLMs)
  dedicated to content creation. Weaver is pre-trained on a carefully selected
  corpus that focuses on improving the writing capabilities of large language
  models. We then fine-tune Weaver for creative and professional writing purposes
  and align it to the preference of professional writers using a suit of novel
  methods for instruction data synthesis and LLM alignment, making it able to
  produce more human-like texts and follow more diverse instructions for content
  creation. The Weaver family consists of models of Weaver Mini (1.8B), Weaver
  Base (6B), Weaver Pro (14B), and Weaver Ultra (34B) sizes, suitable for
  different applications and can be dynamically dispatched by a routing agent
  according to query complexity to balance response quality and computation cost.
  Evaluation on a carefully curated benchmark for assessing the writing
  capabilities of LLMs shows Weaver models of all sizes outperform generalist
  LLMs several times larger than them. Notably, our most-capable Weaver Ultra
  model surpasses GPT-4, a state-of-the-art generalist LLM, on various writing
  scenarios, demonstrating the advantage of training specialized LLMs for writing
  purposes. Moreover, Weaver natively supports retrieval-augmented generation
  (RAG) and function calling (tool usage). We present various use cases of these
  abilities for improving AI-assisted writing systems, including integration of
  external knowledge bases, tools, or APIs, and providing personalized writing
  assistance. Furthermore, we discuss and summarize a guideline and best
  practices for pre-training and fine-tuning domain-specific LLMs.
paper_authors._id: 65b9b10a2f2022969fcee30c
paper_authors.name: Tiannan Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-31T09:19:31.359Z
paper_authors._id: 632bfaebea6e62428ab0e9c2
paper_authors.avatarUrl: /avatars/344aaf371bbba9aea091b12741c451e5.svg
paper_authors.fullname: Tiannan Wang
paper_authors.type: user
paper_authors.user: WTNswaggy
paper_authors._id: 65b9b10a2f2022969fcee30d
paper_authors.name: Jiamin Chen
paper_authors._id: 65b9b10a2f2022969fcee30e
paper_authors.name: Qingrui Jia
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-31T09:23:05.555Z
paper_authors._id: 6530cb83da81958b0f5ccb03
paper_authors.avatarUrl: /avatars/eaea96dd636a0f05715c546640ee5178.svg
paper_authors.fullname: qingrui jia
paper_authors.type: user
paper_authors.user: CCChris
paper_authors._id: 65b9b10a2f2022969fcee30f
paper_authors.name: Shuai Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-21T14:03:18.220Z
paper_authors._id: 668a17447d3f73951ed2ab84
paper_authors.avatarUrl: /avatars/2a1785af3b1b02e495304d2b118accb0.svg
paper_authors.fullname: Shuai Wang
paper_authors.type: user
paper_authors.user: Shuaiii
paper_authors._id: 65b9b10a2f2022969fcee310
paper_authors.name: Ruoyu Fang
paper_authors._id: 65b9b10a2f2022969fcee311
paper_authors.name: Huilin Wang
paper_authors._id: 65b9b10a2f2022969fcee312
paper_authors.name: Zhaowei Gao
paper_authors._id: 65b9b10a2f2022969fcee313
paper_authors.name: Chunzhao Xie
paper_authors._id: 65b9b10a2f2022969fcee314
paper_authors.name: Chuou Xu
paper_authors._id: 65b9b10a2f2022969fcee315
paper_authors.name: Jihong Dai
paper_authors._id: 65b9b10a2f2022969fcee316
paper_authors.name: Yibin Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-31T09:24:55.106Z
paper_authors._id: 6298932a7e4c5c435c79304b
paper_authors.avatarUrl: /avatars/193fae096996ab95fec2d825f1fee458.svg
paper_authors.fullname: Liu Yibin
paper_authors.type: user
paper_authors.user: sbliu
paper_authors._id: 65b9b10a2f2022969fcee317
paper_authors.name: Jialong Wu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-31T09:25:11.558Z
paper_authors._id: 644a4fbc2166258fccc664bc
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/8k3b44MbhQiWuo6i8BnYl.jpeg
paper_authors.fullname: Jialong Wu
paper_authors.type: user
paper_authors.user: callanwu
paper_authors._id: 65b9b10a2f2022969fcee318
paper_authors.name: Shengwei Ding
paper_authors._id: 65b9b10a2f2022969fcee319
paper_authors.name: Long Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-25T09:32:49.747Z
paper_authors._id: 6365d83ce7a78348d82572b0
paper_authors.avatarUrl: /avatars/d50587902cced2c3640fd5de82ff78dd.svg
paper_authors.fullname: ll
paper_authors.type: user
paper_authors.user: jianghuyihei
paper_authors._id: 65b9b10a2f2022969fcee31a
paper_authors.name: Zhiwei Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-31T09:25:40.794Z
paper_authors._id: 647d2e0d36e109abce3fe29a
paper_authors.avatarUrl: /avatars/8cf816e2a8f7ab6ba84d442c0f6e20bc.svg
paper_authors.fullname: Zhiwei Huang
paper_authors.type: user
paper_authors.user: Avril671366613
paper_authors._id: 65b9b10a2f2022969fcee31b
paper_authors.name: Xinle Deng
paper_authors._id: 65b9b10a2f2022969fcee31c
paper_authors.name: Teng Yu
paper_authors._id: 65b9b10a2f2022969fcee31d
paper_authors.name: Gangan Ma
paper_authors._id: 65b9b10a2f2022969fcee31e
paper_authors.name: Han Xiao
paper_authors._id: 65b9b10a2f2022969fcee31f
paper_authors.name: Zixin Chen
paper_authors._id: 65b9b10a2f2022969fcee320
paper_authors.name: Danjun Xiang
paper_authors._id: 65b9b10a2f2022969fcee321
paper_authors.name: Yunxia Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-31T09:27:24.142Z
paper_authors._id: 64377f91b8ea86e5032b04a2
paper_authors.avatarUrl: /avatars/03ebbcbea86cf71ccb9f9c48b18e2b39.svg
paper_authors.fullname: wang
paper_authors.type: user
paper_authors.user: yunxiawang
paper_authors._id: 65b9b10a2f2022969fcee322
paper_authors.name: Yuanyuan Zhu
paper_authors._id: 65b9b10a2f2022969fcee323
paper_authors.name: Yi Xiao
paper_authors._id: 65b9b10a2f2022969fcee324
paper_authors.name: Jing Wang
paper_authors._id: 65b9b10a2f2022969fcee325
paper_authors.name: Yiru Wang
paper_authors._id: 65b9b10a2f2022969fcee326
paper_authors.name: Siran Ding
paper_authors._id: 65b9b10a2f2022969fcee327
paper_authors.name: Jiayang Huang
paper_authors._id: 65b9b10a2f2022969fcee328
paper_authors.name: Jiayi Xu
paper_authors._id: 65b9b10a2f2022969fcee329
paper_authors.name: Yilihamu Tayier
paper_authors._id: 65b9b10a2f2022969fcee32a
paper_authors.name: Zhenyu Hu
paper_authors._id: 65b9b10a2f2022969fcee32b
paper_authors.name: Yuan Gao
paper_authors._id: 65b9b10a2f2022969fcee32c
paper_authors.name: Chengfeng Zheng
paper_authors._id: 65b9b10a2f2022969fcee32d
paper_authors.name: Yueshu Ye
paper_authors._id: 65b9b10a2f2022969fcee32e
paper_authors.name: Yihang Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-31T09:30:22.220Z
paper_authors._id: 647c95e91f878439e2fef485
paper_authors.avatarUrl: /avatars/b99c04cb0eb299d9595639b06e18c451.svg
paper_authors.fullname: Yihang Li
paper_authors.type: user
paper_authors.user: Aurora1412
paper_authors._id: 65b9b10a2f2022969fcee32f
paper_authors.name: Lei Wan
paper_authors._id: 65b9b10a2f2022969fcee330
paper_authors.name: Xinyue Jiang
paper_authors._id: 65b9b10a2f2022969fcee331
paper_authors.name: Yujie Wang
paper_authors._id: 65b9b10a2f2022969fcee332
paper_authors.name: Siyu Cheng
paper_authors._id: 65b9b10a2f2022969fcee333
paper_authors.name: Zhule Song
paper_authors._id: 65b9b10a2f2022969fcee334
paper_authors.name: Xiangru Tang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-31T09:28:51.110Z
paper_authors._id: 63357c608adfa81faf2ac180
paper_authors.avatarUrl: /avatars/ae0314c644f882251baf59b9134fd36f.svg
paper_authors.fullname: Xiangru Tang
paper_authors.type: user
paper_authors.user: RTT1
paper_authors._id: 65b9b10a2f2022969fcee335
paper_authors.name: Xiaohua Xu
paper_authors._id: 65b9b10a2f2022969fcee336
paper_authors.name: Ningyu Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-31T09:28:30.710Z
paper_authors._id: 620b3bbb0668e435407c8d0a
paper_authors.avatarUrl: /avatars/e0fccbb2577d76088e09f054c35cffbc.svg
paper_authors.fullname: Ningyu Zhang
paper_authors.type: user
paper_authors.user: Ningyu
paper_authors._id: 65b9b10a2f2022969fcee337
paper_authors.name: Huajun Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-31T09:28:12.524Z
paper_authors._id: 64931296137833d7ec7689cd
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64931296137833d7ec7689cd/TBihNdp1ZwIWjhfAWjRr6.jpeg
paper_authors.fullname: Huajun Chen
paper_authors.type: user
paper_authors.user: huajunsir
paper_authors._id: 65b9b10a2f2022969fcee338
paper_authors.name: Yuchen Eleanor Jiang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-31T09:28:01.584Z
paper_authors._id: 62c872ff5a87d76d88fdba75
paper_authors.avatarUrl: /avatars/d4696529d1d135e5e2774eb74776ec58.svg
paper_authors.fullname: Yuchen Eleanor Jiang
paper_authors.type: user
paper_authors.user: eleanorjiang
paper_authors._id: 65b9b10a2f2022969fcee339
paper_authors.name: Wangchunshu Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-31T09:27:49.094Z
paper_authors._id: 628c8598ef14f971b698107f
paper_authors.avatarUrl: /avatars/3a4ad87e6b5f9e836a1160d869df1447.svg
paper_authors.fullname: Zhou
paper_authors.type: user
paper_authors.user: Wangchunshu
paper_title: Weaver: Foundation Models for Creative Writing
paper_summary: This work introduces Weaver, our first family of large language models (LLMs)
  dedicated to content creation. Weaver is pre-trained on a carefully selected
  corpus that focuses on improving the writing capabilities of large language
  models. We then fine-tune Weaver for creative and professional writing purposes
  and align it to the preference of professional writers using a suit of novel
  methods for instruction data synthesis and LLM alignment, making it able to
  produce more human-like texts and follow more diverse instructions for content
  creation. The Weaver family consists of models of Weaver Mini (1.8B), Weaver
  Base (6B), Weaver Pro (14B), and Weaver Ultra (34B) sizes, suitable for
  different applications and can be dynamically dispatched by a routing agent
  according to query complexity to balance response quality and computation cost.
  Evaluation on a carefully curated benchmark for assessing the writing
  capabilities of LLMs shows Weaver models of all sizes outperform generalist
  LLMs several times larger than them. Notably, our most-capable Weaver Ultra
  model surpasses GPT-4, a state-of-the-art generalist LLM, on various writing
  scenarios, demonstrating the advantage of training specialized LLMs for writing
  purposes. Moreover, Weaver natively supports retrieval-augmented generation
  (RAG) and function calling (tool usage). We present various use cases of these
  abilities for improving AI-assisted writing systems, including integration of
  external knowledge bases, tools, or APIs, and providing personalized writing
  assistance. Furthermore, we discuss and summarize a guideline and best
  practices for pre-training and fine-tuning domain-specific LLMs.
paper_ai_keywords: large language models (LLMs) content creation pre-trained fine-tune creative writing professional writing instruction data synthesis LLM alignment human-like texts dynamic dispatch query complexity response quality computation cost evaluation benchmark writing capabilities state-of-the-art generalist LLM retrieval-augmented generation (RAG) function calling (tool usage) retrieval-augmented generation function calling AI-assisted writing systems external knowledge bases tools APIs personalized writing assistance pre-training and fine-tuning domain-specific LLMs large language models (LLMs) content creation pre-trained fine-tune creative writing professional writing instruction data synthesis LLM alignment human-like texts dynamic dispatch query complexity response quality