Search, Verify and Feedback: Towards Next Generation Post-training
  Paradigm of Foundation Models via Verifier Engineering
The evolution of machine learning has increasingly prioritized the
development of powerful models and more scalable supervision signals. However,
the emergence of foundation models presents significant challenges in providing
effective supervision signals necessary for further enhancing their
capabilities. Consequently, there is an urgent need to explore novel
supervision signals and technical approaches. In this paper, we propose
verifier engineering, a novel post-training paradigm specifically designed for
the era of foundation models. The core of verifier engineering involves
leveraging a suite of automated verifiers to perform verification tasks and
deliver meaningful feedback to foundation models. We systematically categorize
the verifier engineering process into three essential stages: search, verify,
and feedback, and provide a comprehensive review of state-of-the-art research
developments within each stage. We believe that verifier engineering
constitutes a fundamental pathway toward achieving Artificial General
Intelligence.
673c296853734795380a3dd4
Xinyan Guan
claimed_verified
2024-11-19T13:09:12.237Z
643407dd4b34368fdb0149e8
/avatars/9477b9267d5692a4fe59e30590e9639d.svg
Xinyan Guan
user
xinyan233333
673c296853734795380a3dd5
Yanjiang Liu
claimed_verified
2024-11-20T10:09:05.859Z
643b62ac065961b2252abb7a
/avatars/c7fb4d11f0d795a52bdc771c04a69a20.svg
zuijiang
user
zuijiang
673c296853734795380a3dd6
Xinyu Lu
admin_assigned
2024-11-19T13:10:22.537Z
63115e4daf1fce227a404b16
/avatars/f2e9ee3f1f70bcfabb2b660b241b7aee.svg
Xinyu Lu
user
luxinyu
673c296853734795380a3dd7
Boxi Cao
admin_assigned
2024-11-19T13:10:29.884Z
5fc052241160c47d1d438556
/avatars/f508707fc92f1b42f7897b12b727754c.svg
Boxi Cao
user
Bowieee
673c296853734795380a3dd8
Ben He
673c296853734795380a3dd9
Xianpei Han
admin_assigned
2024-11-19T13:07:58.820Z
65e99a77e71555ed193609cf
/avatars/38ceb127883944677665da967d17dd18.svg
Xianpei Han
user
xphan
673c296853734795380a3dda
Le Sun
673c296853734795380a3ddb
Jie Lou
673c296853734795380a3ddc
Bowen Yu
admin_assigned
2024-11-19T13:08:19.304Z
6438b43ab2ea24b52ebac2b9
/avatars/84133cd719a4b1e2f5c1a74178425f86.svg
Bowen Yu
user
bwy
673c296853734795380a3ddd
Yaojie Lu
admin_assigned
2024-11-19T13:08:11.873Z
6216496a9b34d2fb49144599
https://cdn-avatars.huggingface.co/v1/production/uploads/6216496a9b34d2fb49144599/41CKA_h1Ffj3RzVabSAkm.jpeg
Yaojie Lu
user
luyaojie
673c296853734795380a3dde
Hongyu Lin
admin_assigned
2024-11-19T13:08:05.720Z
6711c702f858a456b4b9f3a4
/avatars/178e9567c3111ab22717c3c0dd003a6a.svg
Hongyu  Lin
user
sanmusunrise
Search, Verify and Feedback: Towards Next Generation Post-training
  Paradigm of Foundation Models via Verifier Engineering
The evolution of machine learning has increasingly prioritized the
development of powerful models and more scalable supervision signals. However,
the emergence of foundation models presents significant challenges in providing
effective supervision signals necessary for further enhancing their
capabilities. Consequently, there is an urgent need to explore novel
supervision signals and technical approaches. In this paper, we propose
verifier engineering, a novel post-training paradigm specifically designed for
the era of foundation models. The core of verifier engineering involves
leveraging a suite of automated verifiers to perform verification tasks and
deliver meaningful feedback to foundation models. We systematically categorize
the verifier engineering process into three essential stages: search, verify,
and feedback, and provide a comprehensive review of state-of-the-art research
developments within each stage. We believe that verifier engineering
constitutes a fundamental pathway toward achieving Artificial General
Intelligence.
Verifier engineering uses automated verification feedback to enhance foundation models, proposing a critical step towards Artificial General Intelligence.
verifier engineering post-training paradigm automated verifiers feedback Artificial General Intelligence