ChemAgent: Self-updating Library in Large Language Models Improves
  Chemical Reasoning
Chemical reasoning usually involves complex, multi-step processes that demand
precise calculations, where even minor errors can lead to cascading failures.
Furthermore, large language models (LLMs) encounter difficulties handling
domain-specific formulas, executing reasoning steps accurately, and integrating
code effectively when tackling chemical reasoning tasks. To address these
challenges, we present ChemAgent, a novel framework designed to improve the
performance of LLMs through a dynamic, self-updating library. This library is
developed by decomposing chemical tasks into sub-tasks and compiling these
sub-tasks into a structured collection that can be referenced for future
queries. Then, when presented with a new problem, ChemAgent retrieves and
refines pertinent information from the library, which we call memory,
facilitating effective task decomposition and the generation of solutions. Our
method designs three types of memory and a library-enhanced reasoning
component, enabling LLMs to improve over time through experience. Experimental
results on four chemical reasoning datasets from SciBench demonstrate that
ChemAgent achieves performance gains of up to 46% (GPT-4), significantly
outperforming existing methods. Our findings suggest substantial potential for
future applications, including tasks such as drug discovery and materials
science. Our code can be found at https://github.com/gersteinlab/chemagent
6785e9ade1226951bc8c83cb
Xiangru Tang
extracted_pending
2025-01-14T04:35:59.001Z
63357c608adfa81faf2ac180
/avatars/ae0314c644f882251baf59b9134fd36f.svg
Xiangru Tang
user
RTT1
6785e9ade1226951bc8c83cc
Tianyu Hu
admin_assigned
2025-01-14T09:25:53.078Z
64915cd1129c5fe554d8a54e
/avatars/ea367103ddb2bfbda0cb05a9e4ca4862.svg
TianYu Hu
user
CamelH
6785e9ade1226951bc8c83cd
Muyang Ye
6785e9ade1226951bc8c83ce
Yanjun Shao
admin_assigned
2025-01-14T09:26:07.193Z
62970df979f193515da13dc0
https://cdn-avatars.huggingface.co/v1/production/uploads/62970df979f193515da13dc0/A-mgKIcgTXRJ54GCHswTq.jpeg
Yanjun Shao
user
super-dainiu
6785e9ade1226951bc8c83cf
Xunjian Yin
6785e9ade1226951bc8c83d0
Siru Ouyang
admin_assigned
2025-01-14T09:44:50.034Z
64bb4eeb5ee4641b32d7db27
/avatars/fe32fe5e5f2914b46f1d05d1a67d4314.svg
Siru Ouyang
user
siruo2
6785e9ade1226951bc8c83d1
Wangchunshu Zhou
admin_assigned
2025-01-14T09:47:31.168Z
628c8598ef14f971b698107f
/avatars/3a4ad87e6b5f9e836a1160d869df1447.svg
Zhou
user
Wangchunshu
6785e9ade1226951bc8c83d2
Pan Lu
admin_assigned
2025-01-14T09:47:40.737Z
60f5f68fa7fd83d025749234
https://cdn-avatars.huggingface.co/v1/production/uploads/60f5f68fa7fd83d025749234/gCeJAZfzaANAcEvI6v5-P.jpeg
Pan Lu
user
lupantech
6785e9ade1226951bc8c83d3
Zhuosheng Zhang
admin_assigned
2025-01-14T09:47:46.326Z
5f82f9f7f0801648bf8844b2
https://cdn-avatars.huggingface.co/v1/production/uploads/1669627733134-5f82f9f7f0801648bf8844b2.jpeg
Zhuosheng Zhang
user
cooelf
6785e9ade1226951bc8c83d4
Yilun Zhao
admin_assigned
2025-01-14T09:48:02.426Z
62f662bcc58915315c4eccea
https://cdn-avatars.huggingface.co/v1/production/uploads/62f662bcc58915315c4eccea/zOAQLONfMP88zr70sxHK-.jpeg
Yilun Zhao
user
yilunzhao
6785e9ade1226951bc8c83d5
Arman Cohan
admin_assigned
2025-01-14T09:48:08.442Z
5f5ba21188f57f65f951f255
https://cdn-avatars.huggingface.co/v1/production/uploads/1599840760465-noauth.png
Arman Cohan
user
armanc
6785e9ade1226951bc8c83d6
Mark Gerstein
ChemAgent: Self-updating Library in Large Language Models Improves
  Chemical Reasoning
Chemical reasoning usually involves complex, multi-step processes that demand
precise calculations, where even minor errors can lead to cascading failures.
Furthermore, large language models (LLMs) encounter difficulties handling
domain-specific formulas, executing reasoning steps accurately, and integrating
code effectively when tackling chemical reasoning tasks. To address these
challenges, we present ChemAgent, a novel framework designed to improve the
performance of LLMs through a dynamic, self-updating library. This library is
developed by decomposing chemical tasks into sub-tasks and compiling these
sub-tasks into a structured collection that can be referenced for future
queries. Then, when presented with a new problem, ChemAgent retrieves and
refines pertinent information from the library, which we call memory,
facilitating effective task decomposition and the generation of solutions. Our
method designs three types of memory and a library-enhanced reasoning
component, enabling LLMs to improve over time through experience. Experimental
results on four chemical reasoning datasets from SciBench demonstrate that
ChemAgent achieves performance gains of up to 46% (GPT-4), significantly
outperforming existing methods. Our findings suggest substantial potential for
future applications, including tasks such as drug discovery and materials
science. Our code can be found at https://github.com/gersteinlab/chemagent
ChemAgent, a framework incorporating a dynamic memory system, enhances LLMs in chemical reasoning tasks by decomposing them into sub-tasks and leveraging a library for improved accuracy and efficiency.
large language models (LLMs) chemical reasoning sub-tasks memory library-enhanced reasoning component SciBench drug discovery materials science