COIG-P: A High-Quality and Large-Scale Chinese Preference Dataset for
  Alignment with Human Values
Aligning large language models (LLMs) with human preferences has achieved
remarkable success. However, existing Chinese preference datasets are limited
by small scale, narrow domain coverage, and lack of rigorous data validation.
Additionally, the reliance on human annotators for instruction and response
labeling significantly constrains the scalability of human preference datasets.
To address these challenges, we design an LLM-based Chinese preference dataset
annotation pipeline with no human intervention. Specifically, we crawled and
carefully filtered 92k high-quality Chinese queries and employed 15 mainstream
LLMs to generate and score chosen-rejected response pairs. Based on it, we
introduce COIG-P (Chinese Open Instruction Generalist - Preference), a
high-quality, large-scale Chinese preference dataset, comprises 1,009k Chinese
preference pairs spanning 6 diverse domains: Chat, Code, Math, Logic, Novel,
and Role. Building upon COIG-P, to reduce the overhead of using LLMs for
scoring, we trained a 8B-sized Chinese Reward Model (CRM) and meticulously
constructed a Chinese Reward Benchmark (CRBench). Evaluation results based on
AlignBench liu2024alignbenchbenchmarkingchinesealignment show that that
COIG-P significantly outperforms other Chinese preference datasets, and it
brings significant performance improvements ranging from 2% to 12% for the
Qwen2/2.5 and Infinity-Instruct-3M-0625 model series, respectively. The results
on CRBench demonstrate that our CRM has a strong and robust scoring ability. We
apply it to filter chosen-rejected response pairs in a test split of COIG-P,
and our experiments show that it is comparable to GPT-4o in identifying
low-quality samples while maintaining efficiency and cost-effectiveness. Our
codes and data are released in
https://github.com/multimodal-art-projection/COIG-P.
67f630091aed1b4344b57c1b
M-A-P Team
67f630091aed1b4344b57c1c
Siwei Wu
claimed_verified
2025-04-09T14:36:41.051Z
656d97b10bbc114fe64a96c5
/avatars/fd23bae1d85c5b96c42064a5ddcfad41.svg
SiweiWu
user
SiweiWu
67f630091aed1b4344b57c1d
Jincheng Ren
claimed_verified
2025-04-09T09:48:17.939Z
6704ee27386892c420db1938
https://cdn-avatars.huggingface.co/v1/production/uploads/6704ee27386892c420db1938/lb5mtEwYhn47RawkynYPs.jpeg
JinCheng Ren
user
JinChengRen
67f630091aed1b4344b57c1e
Xinrun Du
claimed_verified
2025-04-09T14:36:47.680Z
654907a4a1faff97850c4eff
/avatars/458c90151614bc7f116943b6e67d6b8a.svg
du
user
dododododo
67f630091aed1b4344b57c1f
Shuyue Guo
67f630091aed1b4344b57c20
Xingwei Qu
67f630091aed1b4344b57c21
Yiming Liang
admin_assigned
2025-04-09T16:35:12.067Z
6555e8d8a0c34cd61a6b9ce3
/avatars/71dc562cef4bd42f6b762f036357c800.svg
yimingliang
user
yimingliang
67f630091aed1b4344b57c22
Jie Liu
67f630091aed1b4344b57c23
Yunwen Li
67f630091aed1b4344b57c24
Tianyu Zheng
claimed_verified
2025-04-09T09:48:16.069Z
64ab99dcb76bfd863eba64c1
https://cdn-avatars.huggingface.co/v1/production/uploads/64ab99dcb76bfd863eba64c1/UBXwDPx17X-gl-SzBPvrc.jpeg
TY.Zheng
user
aaabiao
67f630091aed1b4344b57c25
Boyu Feng
claimed_verified
2025-04-09T14:36:53.597Z
67f654ecb88bc093ada9da3f
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/Anh0oLDzzHCeSmEzwAmYd.png
boyuFeng
user
FWORKS
67f630091aed1b4344b57c26
Huaqing Yuan
claimed_verified
2025-04-10T06:45:34.650Z
67f6adf26d55263a1b0d1ea9
/avatars/f8cf3dca6d809518b49f20883b4e24fb.svg
Huaqing Yuan
user
huaqingyuan
67f630091aed1b4344b57c27
Zenith Wang
admin_assigned
2025-04-09T16:35:42.141Z
632c59bbea6e62428ab3b480
https://cdn-avatars.huggingface.co/v1/production/uploads/632c59bbea6e62428ab3b480/XroXmPnGl1fDrSnN1bQ99.jpeg
Zili Wang
user
Zenithwang
67f630091aed1b4344b57c28
Jiaheng Liu
admin_assigned
2025-04-09T16:35:49.044Z
65377c30e48353201e6fdda0
/avatars/a8f803b6f2e598eaee9c52c0d2ddfc16.svg
Jiaheng Liu
user
CheeryLJH
67f630091aed1b4344b57c29
Wenhao Huang
admin_assigned
2025-04-09T16:36:15.438Z
641e5bf65f274a0a92c2f6a2
/avatars/c15a54c51998c0e6367685e8e1737ec9.svg
Wenhao Huang
user
EZ-hwh
67f630091aed1b4344b57c2a
Chenglin Cai
admin_assigned
2025-04-09T16:36:31.116Z
64f9c21b681224dbe49a2280
/avatars/df26cc4b4c6105af2c77392db61e3a27.svg
caichenglin
user
easy4mego
67f630091aed1b4344b57c2b
Haoran Que
admin_assigned
2025-04-09T16:36:48.325Z
676935578263df95771cb486
/avatars/f81098cf991ee8bc2cab653fbddd0913.svg
Haoran Que
user
Quehry2
67f630091aed1b4344b57c2c
Jian Yang
67f630091aed1b4344b57c2d
Yuelin Bai
67f630091aed1b4344b57c2e
Zekun Moore Wang
admin_assigned
2025-04-09T16:37:03.321Z
6149a9e95347647e6bb68882
https://cdn-avatars.huggingface.co/v1/production/uploads/6149a9e95347647e6bb68882/Jddln1FxScCeVgTSCNBpr.png
Zekun Moore Wang
user
ZenMoore
67f630091aed1b4344b57c2f
Zhouliang Yu
claimed_verified
2025-04-09T14:36:43.986Z
62a80fe3ac97233f1625235a
https://cdn-avatars.huggingface.co/v1/production/uploads/62a80fe3ac97233f1625235a/_rGtpqdY7OEBz3pyqb6fE.jpeg
Zhouliang Yu
user
zhouliang
67f630091aed1b4344b57c30
Qunshu Lin
67f630091aed1b4344b57c31
Ding Pan
admin_assigned
2025-04-09T16:37:19.142Z
65d5c5d6c79c53aff7420794
/avatars/f248c4ae2eb022e732b754cfa7235e0a.svg
DingPan
user
DingPan
67f630091aed1b4344b57c32
Yuchen Jiang
admin_assigned
2025-04-09T16:37:26.925Z
64fff4debef9b5946564d490
/avatars/c7856111a030b6b8d96650741471f0ce.svg
Jiang Yuchen
user
YuChenJiang
67f630091aed1b4344b57c33
Tiannan Wang
admin_assigned
2025-04-09T16:37:34.116Z
632bfaebea6e62428ab0e9c2
/avatars/344aaf371bbba9aea091b12741c451e5.svg
Tiannan Wang
user
WTNswaggy
67f630091aed1b4344b57c34
Wangchunshu Zhou
admin_assigned
2025-04-09T16:37:43.878Z
628c8598ef14f971b698107f
/avatars/3a4ad87e6b5f9e836a1160d869df1447.svg
Zhou
user
Wangchunshu
67f630091aed1b4344b57c35
Shenzhi Wang
admin_assigned
2025-04-09T16:37:53.643Z
6486dde1f74857df3f1a5828
https://cdn-avatars.huggingface.co/v1/production/uploads/6486dde1f74857df3f1a5828/FgE80CpalBO5qqArdfwxA.jpeg
Shenzhi Wang
user
shenzhi-wang
67f630091aed1b4344b57c36
Xingyuan Bu
claimed_verified
2025-04-09T14:36:50.729Z
6444e7765691ca69b0d95856
/avatars/4ae5001e7c7dea89c4deaf2b05436857.svg
Xingyuan Bu
user
sefira32
67f630091aed1b4344b57c37
Minghao Liu
claimed_verified
2025-04-09T09:48:19.721Z
6417d9ea8f689506e7148417
https://cdn-avatars.huggingface.co/v1/production/uploads/6417d9ea8f689506e7148417/bAYcruWNw4WvmuQcGgcwC.jpeg
minghao
user
Liam-Liu
67f630091aed1b4344b57c38
Guoyin Wang
admin_assigned
2025-04-09T16:38:01.661Z
6490d4ba1afdee3acd1147f6
/avatars/ae13c7b21fe9ced7541dcd664d1b94ed.svg
Guoyin Wang
user
guoyinwang
67f630091aed1b4344b57c39
Ge Zhang
admin_assigned
2025-04-09T16:38:09.800Z
638efcf4c67af472d316d424
/avatars/97a57859d7d87a3a8f1bb41d32a72bc2.svg
Ge Zhang
user
zhangysk
67f630091aed1b4344b57c3a
Chenghua Lin
admin_assigned
2025-04-09T16:38:19.130Z
6442ba123610a28a4ad5382f
/avatars/97f2d3d2c10058ea41d7bf9087e1f619.svg
Chenghua Lin
user
chenghualin
COIG-P: A High-Quality and Large-Scale Chinese Preference Dataset for
  Alignment with Human Values
Aligning large language models (LLMs) with human preferences has achieved
remarkable success. However, existing Chinese preference datasets are limited
by small scale, narrow domain coverage, and lack of rigorous data validation.
Additionally, the reliance on human annotators for instruction and response
labeling significantly constrains the scalability of human preference datasets.
To address these challenges, we design an LLM-based Chinese preference dataset
annotation pipeline with no human intervention. Specifically, we crawled and
carefully filtered 92k high-quality Chinese queries and employed 15 mainstream
LLMs to generate and score chosen-rejected response pairs. Based on it, we
introduce COIG-P (Chinese Open Instruction Generalist - Preference), a
high-quality, large-scale Chinese preference dataset, comprises 1,009k Chinese
preference pairs spanning 6 diverse domains: Chat, Code, Math, Logic, Novel,
and Role. Building upon COIG-P, to reduce the overhead of using LLMs for
scoring, we trained a 8B-sized Chinese Reward Model (CRM) and meticulously
constructed a Chinese Reward Benchmark (CRBench). Evaluation results based on
AlignBench liu2024alignbenchbenchmarkingchinesealignment show that that
COIG-P significantly outperforms other Chinese preference datasets, and it
brings significant performance improvements ranging from 2% to 12% for the
Qwen2/2.5 and Infinity-Instruct-3M-0625 model series, respectively. The results
on CRBench demonstrate that our CRM has a strong and robust scoring ability. We
apply it to filter chosen-rejected response pairs in a test split of COIG-P,
and our experiments show that it is comparable to GPT-4o in identifying
low-quality samples while maintaining efficiency and cost-effectiveness. Our
codes and data are released in
https://github.com/multimodal-art-projection/COIG-P.
A new Chinese preference dataset and reward model are introduced to scale human preference alignment using LLMs, enhancing performance and cost-effectiveness.
LLMs preference dataset annotation pipeline COIG-P domains Reward Model CRM Reward Benchmark CRBench AlignBench Qwen2 Infinity-Instruct GPT-4