Weaver: Foundation Models for Creative Writing
This work introduces Weaver, our first family of large language models (LLMs)
dedicated to content creation. Weaver is pre-trained on a carefully selected
corpus that focuses on improving the writing capabilities of large language
models. We then fine-tune Weaver for creative and professional writing purposes
and align it to the preference of professional writers using a suit of novel
methods for instruction data synthesis and LLM alignment, making it able to
produce more human-like texts and follow more diverse instructions for content
creation. The Weaver family consists of models of Weaver Mini (1.8B), Weaver
Base (6B), Weaver Pro (14B), and Weaver Ultra (34B) sizes, suitable for
different applications and can be dynamically dispatched by a routing agent
according to query complexity to balance response quality and computation cost.
Evaluation on a carefully curated benchmark for assessing the writing
capabilities of LLMs shows Weaver models of all sizes outperform generalist
LLMs several times larger than them. Notably, our most-capable Weaver Ultra
model surpasses GPT-4, a state-of-the-art generalist LLM, on various writing
scenarios, demonstrating the advantage of training specialized LLMs for writing
purposes. Moreover, Weaver natively supports retrieval-augmented generation
(RAG) and function calling (tool usage). We present various use cases of these
abilities for improving AI-assisted writing systems, including integration of
external knowledge bases, tools, or APIs, and providing personalized writing
assistance. Furthermore, we discuss and summarize a guideline and best
practices for pre-training and fine-tuning domain-specific LLMs.
65b9b10a2f2022969fcee30c
Tiannan Wang
admin_assigned
2024-01-31T09:19:31.359Z
632bfaebea6e62428ab0e9c2
/avatars/344aaf371bbba9aea091b12741c451e5.svg
Tiannan Wang
user
WTNswaggy
65b9b10a2f2022969fcee30d
Jiamin Chen
65b9b10a2f2022969fcee30e
Qingrui Jia
admin_assigned
2024-01-31T09:23:05.555Z
6530cb83da81958b0f5ccb03
/avatars/eaea96dd636a0f05715c546640ee5178.svg
qingrui jia
user
CCChris
65b9b10a2f2022969fcee30f
Shuai Wang
claimed_verified
2024-10-21T14:03:18.220Z
668a17447d3f73951ed2ab84
/avatars/2a1785af3b1b02e495304d2b118accb0.svg
Shuai Wang
user
Shuaiii
65b9b10a2f2022969fcee310
Ruoyu Fang
65b9b10a2f2022969fcee311
Huilin Wang
65b9b10a2f2022969fcee312
Zhaowei Gao
65b9b10a2f2022969fcee313
Chunzhao Xie
65b9b10a2f2022969fcee314
Chuou Xu
65b9b10a2f2022969fcee315
Jihong Dai
65b9b10a2f2022969fcee316
Yibin Liu
admin_assigned
2024-01-31T09:24:55.106Z
6298932a7e4c5c435c79304b
/avatars/193fae096996ab95fec2d825f1fee458.svg
Liu Yibin
user
sbliu
65b9b10a2f2022969fcee317
Jialong Wu
admin_assigned
2024-01-31T09:25:11.558Z
644a4fbc2166258fccc664bc
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/8k3b44MbhQiWuo6i8BnYl.jpeg
Jialong Wu
user
callanwu
65b9b10a2f2022969fcee318
Shengwei Ding
65b9b10a2f2022969fcee319
Long Li
claimed_verified
2024-10-25T09:32:49.747Z
6365d83ce7a78348d82572b0
/avatars/d50587902cced2c3640fd5de82ff78dd.svg
ll
user
jianghuyihei
65b9b10a2f2022969fcee31a
Zhiwei Huang
admin_assigned
2024-01-31T09:25:40.794Z
647d2e0d36e109abce3fe29a
/avatars/8cf816e2a8f7ab6ba84d442c0f6e20bc.svg
Zhiwei Huang
user
Avril671366613
65b9b10a2f2022969fcee31b
Xinle Deng
65b9b10a2f2022969fcee31c
Teng Yu
65b9b10a2f2022969fcee31d
Gangan Ma
65b9b10a2f2022969fcee31e
Han Xiao
65b9b10a2f2022969fcee31f
Zixin Chen
65b9b10a2f2022969fcee320
Danjun Xiang
65b9b10a2f2022969fcee321
Yunxia Wang
admin_assigned
2024-01-31T09:27:24.142Z
64377f91b8ea86e5032b04a2
/avatars/03ebbcbea86cf71ccb9f9c48b18e2b39.svg
wang
user
yunxiawang
65b9b10a2f2022969fcee322
Yuanyuan Zhu
65b9b10a2f2022969fcee323
Yi Xiao
65b9b10a2f2022969fcee324
Jing Wang
65b9b10a2f2022969fcee325
Yiru Wang
65b9b10a2f2022969fcee326
Siran Ding
65b9b10a2f2022969fcee327
Jiayang Huang
65b9b10a2f2022969fcee328
Jiayi Xu
65b9b10a2f2022969fcee329
Yilihamu Tayier
65b9b10a2f2022969fcee32a
Zhenyu Hu
65b9b10a2f2022969fcee32b
Yuan Gao
65b9b10a2f2022969fcee32c
Chengfeng Zheng
65b9b10a2f2022969fcee32d
Yueshu Ye
65b9b10a2f2022969fcee32e
Yihang Li
admin_assigned
2024-01-31T09:30:22.220Z
647c95e91f878439e2fef485
/avatars/b99c04cb0eb299d9595639b06e18c451.svg
Yihang Li
user
Aurora1412
65b9b10a2f2022969fcee32f
Lei Wan
65b9b10a2f2022969fcee330
Xinyue Jiang
65b9b10a2f2022969fcee331
Yujie Wang
65b9b10a2f2022969fcee332
Siyu Cheng
65b9b10a2f2022969fcee333
Zhule Song
65b9b10a2f2022969fcee334
Xiangru Tang
admin_assigned
2024-01-31T09:28:51.110Z
63357c608adfa81faf2ac180
/avatars/ae0314c644f882251baf59b9134fd36f.svg
Xiangru Tang
user
RTT1
65b9b10a2f2022969fcee335
Xiaohua Xu
65b9b10a2f2022969fcee336
Ningyu Zhang
admin_assigned
2024-01-31T09:28:30.710Z
620b3bbb0668e435407c8d0a
/avatars/e0fccbb2577d76088e09f054c35cffbc.svg
Ningyu Zhang
user
Ningyu
65b9b10a2f2022969fcee337
Huajun Chen
admin_assigned
2024-01-31T09:28:12.524Z
64931296137833d7ec7689cd
https://cdn-avatars.huggingface.co/v1/production/uploads/64931296137833d7ec7689cd/TBihNdp1ZwIWjhfAWjRr6.jpeg
Huajun Chen
user
huajunsir
65b9b10a2f2022969fcee338
Yuchen Eleanor Jiang
admin_assigned
2024-01-31T09:28:01.584Z
62c872ff5a87d76d88fdba75
/avatars/d4696529d1d135e5e2774eb74776ec58.svg
Yuchen Eleanor Jiang
user
eleanorjiang
65b9b10a2f2022969fcee339
Wangchunshu Zhou
admin_assigned
2024-01-31T09:27:49.094Z
628c8598ef14f971b698107f
/avatars/3a4ad87e6b5f9e836a1160d869df1447.svg
Zhou
user
Wangchunshu
Weaver: Foundation Models for Creative Writing
This work introduces Weaver, our first family of large language models (LLMs)
dedicated to content creation. Weaver is pre-trained on a carefully selected
corpus that focuses on improving the writing capabilities of large language
models. We then fine-tune Weaver for creative and professional writing purposes
and align it to the preference of professional writers using a suit of novel
methods for instruction data synthesis and LLM alignment, making it able to
produce more human-like texts and follow more diverse instructions for content
creation. The Weaver family consists of models of Weaver Mini (1.8B), Weaver
Base (6B), Weaver Pro (14B), and Weaver Ultra (34B) sizes, suitable for
different applications and can be dynamically dispatched by a routing agent
according to query complexity to balance response quality and computation cost.
Evaluation on a carefully curated benchmark for assessing the writing
capabilities of LLMs shows Weaver models of all sizes outperform generalist
LLMs several times larger than them. Notably, our most-capable Weaver Ultra
model surpasses GPT-4, a state-of-the-art generalist LLM, on various writing
scenarios, demonstrating the advantage of training specialized LLMs for writing
purposes. Moreover, Weaver natively supports retrieval-augmented generation
(RAG) and function calling (tool usage). We present various use cases of these
abilities for improving AI-assisted writing systems, including integration of
external knowledge bases, tools, or APIs, and providing personalized writing
assistance. Furthermore, we discuss and summarize a guideline and best
practices for pre-training and fine-tuning domain-specific LLMs.
Weaver, a family of specialized large language models, achieves superior writing capabilities through pre-training and fine-tuning methods, surpasses GPT-4 in various writing tasks, and supports retrieval-augmented generation and tool usage.
large language models pre-training fine-tuning instruction data synthesis LLM alignment human-like texts query complexity response quality computation cost benchmark generalist LLMs evaluation retrieval-augmented generation RAG function calling AI-assisted writing systems external knowledge bases tools APIs personalized writing assistance domain-specific LLMs guideline best practices