title: Towards Fully-Automated Materials Discovery via Large-Scale Synthesis
    Dataset and Expert-Level LLM-as-a-Judge
summary: Materials synthesis is vital for innovations such as energy storage,
  catalysis, electronics, and biomedical devices. Yet, the process relies heavily
  on empirical, trial-and-error methods guided by expert intuition. Our work aims
  to support the materials science community by providing a practical,
  data-driven resource. We have curated a comprehensive dataset of 17K
  expert-verified synthesis recipes from open-access literature, which forms the
  basis of our newly developed benchmark, AlchemyBench. AlchemyBench offers an
  end-to-end framework that supports research in large language models applied to
  synthesis prediction. It encompasses key tasks, including raw materials and
  equipment prediction, synthesis procedure generation, and characterization
  outcome forecasting. We propose an LLM-as-a-Judge framework that leverages
  large language models for automated evaluation, demonstrating strong
  statistical agreement with expert assessments. Overall, our contributions offer
  a supportive foundation for exploring the capabilities of LLMs in predicting
  and guiding materials synthesis, ultimately paving the way for more efficient
  experimental design and accelerated innovation in materials science.
paper_authors._id: 67bd23159e826530ef606d4d
paper_authors.name: Heegyu Kim
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-25T09:40:23.108Z
paper_authors._id: 5f3a52317e583543386218db
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/5f3a52317e583543386218db/WLnW1_fCic9NWfMjE3yB-.jpeg
paper_authors.fullname: Heegyu Kim
paper_authors.type: user
paper_authors.user: heegyu
paper_authors._id: 67bd23159e826530ef606d4e
paper_authors.name: Taeyang Jeon
paper_authors._id: 67bd23159e826530ef606d4f
paper_authors.name: Seungtaek Choi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-26T08:38:03.427Z
paper_authors._id: 658a7aed74d1a1cbd0598bfb
paper_authors.avatarUrl: /avatars/055dc3c82820644b316b60d118c6ff94.svg
paper_authors.fullname: Seungtaek Choi
paper_authors.type: user
paper_authors.user: seungtaek-choi
paper_authors._id: 67bd23159e826530ef606d50
paper_authors.name: Jihoon Hong
paper_authors._id: 67bd23159e826530ef606d51
paper_authors.name: Dongwon Jeon
paper_authors._id: 67bd23159e826530ef606d52
paper_authors.name: Sungbum Cho
paper_authors._id: 67bd23159e826530ef606d53
paper_authors.name: Ga-Yeon Baek
paper_authors._id: 67bd23159e826530ef606d54
paper_authors.name: Kyung-Won Kwak
paper_authors._id: 67bd23159e826530ef606d55
paper_authors.name: Dong-Hee Lee
paper_authors._id: 67bd23159e826530ef606d56
paper_authors.name: Sun-Jin Choi
paper_authors._id: 67bd23159e826530ef606d57
paper_authors.name: Jisu Bae
paper_authors._id: 67bd23159e826530ef606d58
paper_authors.name: Chihoon Lee
paper_authors._id: 67bd23159e826530ef606d59
paper_authors.name: Yunseo Kim
paper_authors._id: 67bd23159e826530ef606d5a
paper_authors.name: Jinsung Park
paper_authors._id: 67bd23159e826530ef606d5b
paper_authors.name: Hyunsouk Cho
paper_title: Towards Fully-Automated Materials Discovery via Large-Scale Synthesis
    Dataset and Expert-Level LLM-as-a-Judge
paper_summary: Materials synthesis is vital for innovations such as energy storage,
  catalysis, electronics, and biomedical devices. Yet, the process relies heavily
  on empirical, trial-and-error methods guided by expert intuition. Our work aims
  to support the materials science community by providing a practical,
  data-driven resource. We have curated a comprehensive dataset of 17K
  expert-verified synthesis recipes from open-access literature, which forms the
  basis of our newly developed benchmark, AlchemyBench. AlchemyBench offers an
  end-to-end framework that supports research in large language models applied to
  synthesis prediction. It encompasses key tasks, including raw materials and
  equipment prediction, synthesis procedure generation, and characterization
  outcome forecasting. We propose an LLM-as-a-Judge framework that leverages
  large language models for automated evaluation, demonstrating strong
  statistical agreement with expert assessments. Overall, our contributions offer
  a supportive foundation for exploring the capabilities of LLMs in predicting
  and guiding materials synthesis, ultimately paving the way for more efficient
  experimental design and accelerated innovation in materials science.
paper_ai_keywords: LLM-as-a-Judge large language models synthesis prediction raw materials prediction equipment prediction synthesis procedure generation characterization outcome forecasting