Learning Task Decomposition to Assist Humans in Competitive Programming
When using language models (LMs) to solve complex problems, humans might
struggle to understand the LM-generated solutions and repair the flawed ones.
To assist humans in repairing them, we propose to automatically decompose
complex solutions into multiple simpler pieces that correspond to specific
subtasks. We introduce a novel objective for learning task decomposition,
termed assistive value (AssistV), which measures the feasibility and speed for
humans to repair the decomposed solution. We collect a dataset of human repair
experiences on different decomposed solutions. Utilizing the collected data as
in-context examples, we then learn to critique, refine, and rank decomposed
solutions to improve AssistV. We validate our method under competitive
programming problems: under 177 hours of human study, our method enables
non-experts to solve 33.3\% more problems, speeds them up by 3.3x, and empowers
them to match unassisted experts.
66b63d152174fa1446c304a7
Jiaxin Wen
admin_assigned
2024-08-14T12:58:40.310Z
5fd2bfccaa1f2d0285acf71d
/avatars/1bb42f0af114299d8df43b26f2a296e9.svg
Jiaxin Wen
user
jiaxin-wen
66b63d152174fa1446c304a8
Ruiqi Zhong
admin_assigned
2024-08-14T13:01:55.041Z
5eae5de74e876668a0c37779
/avatars/cf04ab4832c6565ed085dec12fcae188.svg
Ruiqi Zhong
user
ruiqi-zhong
66b63d152174fa1446c304a9
Pei Ke
admin_assigned
2024-08-14T13:02:15.034Z
606c28fff3b1db0eeac26495
/avatars/edad72f14661a06367f360b7fedf3e1b.svg
Pei Ke
user
kepei1106
66b63d152174fa1446c304aa
Zhihong Shao
admin_assigned
2024-08-14T13:02:30.632Z
65db64f8b62d242ed8711701
/avatars/753e9f980eb6786c6b53b2f1becbf745.svg
Zhihong Shao
user
ZhihongShao
66b63d152174fa1446c304ab
Hongning Wang
admin_assigned
2024-08-14T13:02:36.342Z
62ec23fcbd19e355478fc584
/avatars/d7c567ef5f20bb3b9905cb5015d11e12.svg
Hongning Wang
user
howang
66b63d152174fa1446c304ac
Minlie Huang
Learning Task Decomposition to Assist Humans in Competitive Programming
When using language models (LMs) to solve complex problems, humans might
struggle to understand the LM-generated solutions and repair the flawed ones.
To assist humans in repairing them, we propose to automatically decompose
complex solutions into multiple simpler pieces that correspond to specific
subtasks. We introduce a novel objective for learning task decomposition,
termed assistive value (AssistV), which measures the feasibility and speed for
humans to repair the decomposed solution. We collect a dataset of human repair
experiences on different decomposed solutions. Utilizing the collected data as
in-context examples, we then learn to critique, refine, and rank decomposed
solutions to improve AssistV. We validate our method under competitive
programming problems: under 177 hours of human study, our method enables
non-experts to solve 33.3\% more problems, speeds them up by 3.3x, and empowers
them to match unassisted experts.
A novel assistive value metric and framework for decomposing and repairing language model solutions in complex problems improve non-expert performance and efficiency.
language models task decomposition assistive value in-context examples critique refine rank competitive programming