Towards Fully-Automated Materials Discovery via Large-Scale Synthesis
  Dataset and Expert-Level LLM-as-a-Judge
Materials synthesis is vital for innovations such as energy storage,
catalysis, electronics, and biomedical devices. Yet, the process relies heavily
on empirical, trial-and-error methods guided by expert intuition. Our work aims
to support the materials science community by providing a practical,
data-driven resource. We have curated a comprehensive dataset of 17K
expert-verified synthesis recipes from open-access literature, which forms the
basis of our newly developed benchmark, AlchemyBench. AlchemyBench offers an
end-to-end framework that supports research in large language models applied to
synthesis prediction. It encompasses key tasks, including raw materials and
equipment prediction, synthesis procedure generation, and characterization
outcome forecasting. We propose an LLM-as-a-Judge framework that leverages
large language models for automated evaluation, demonstrating strong
statistical agreement with expert assessments. Overall, our contributions offer
a supportive foundation for exploring the capabilities of LLMs in predicting
and guiding materials synthesis, ultimately paving the way for more efficient
experimental design and accelerated innovation in materials science.
67bd23159e826530ef606d4d
Heegyu Kim
claimed_verified
2025-02-25T09:40:23.108Z
5f3a52317e583543386218db
https://cdn-avatars.huggingface.co/v1/production/uploads/5f3a52317e583543386218db/WLnW1_fCic9NWfMjE3yB-.jpeg
Heegyu Kim
user
heegyu
67bd23159e826530ef606d4e
Taeyang Jeon
67bd23159e826530ef606d4f
Seungtaek Choi
claimed_verified
2025-02-26T08:38:03.427Z
658a7aed74d1a1cbd0598bfb
/avatars/055dc3c82820644b316b60d118c6ff94.svg
Seungtaek Choi
user
seungtaek-choi
67bd23159e826530ef606d50
Jihoon Hong
67bd23159e826530ef606d51
Dongwon Jeon
67bd23159e826530ef606d52
Sungbum Cho
67bd23159e826530ef606d53
Ga-Yeon Baek
67bd23159e826530ef606d54
Kyung-Won Kwak
67bd23159e826530ef606d55
Dong-Hee Lee
67bd23159e826530ef606d56
Sun-Jin Choi
67bd23159e826530ef606d57
Jisu Bae
67bd23159e826530ef606d58
Chihoon Lee
67bd23159e826530ef606d59
Yunseo Kim
67bd23159e826530ef606d5a
Jinsung Park
67bd23159e826530ef606d5b
Hyunsouk Cho
Towards Fully-Automated Materials Discovery via Large-Scale Synthesis
  Dataset and Expert-Level LLM-as-a-Judge
Materials synthesis is vital for innovations such as energy storage,
catalysis, electronics, and biomedical devices. Yet, the process relies heavily
on empirical, trial-and-error methods guided by expert intuition. Our work aims
to support the materials science community by providing a practical,
data-driven resource. We have curated a comprehensive dataset of 17K
expert-verified synthesis recipes from open-access literature, which forms the
basis of our newly developed benchmark, AlchemyBench. AlchemyBench offers an
end-to-end framework that supports research in large language models applied to
synthesis prediction. It encompasses key tasks, including raw materials and
equipment prediction, synthesis procedure generation, and characterization
outcome forecasting. We propose an LLM-as-a-Judge framework that leverages
large language models for automated evaluation, demonstrating strong
statistical agreement with expert assessments. Overall, our contributions offer
a supportive foundation for exploring the capabilities of LLMs in predicting
and guiding materials synthesis, ultimately paving the way for more efficient
experimental design and accelerated innovation in materials science.
An AlchemyBench benchmark framework uses large language models to predict and guide materials synthesis, improving efficiency and innovation through automated evaluation and expert agreement.
large language models LLM-as-a-Judge synthesis prediction materials synthesis automated evaluation