title: AutoMat: Enabling Automated Crystal Structure Reconstruction from
    Microscopy via Agentic Tool Use
summary: Machine learning-based interatomic potentials and force fields depend
  critically on accurate atomic structures, yet such data are scarce due to the
  limited availability of experimentally resolved crystals. Although
  atomic-resolution electron microscopy offers a potential source of structural
  data, converting these images into simulation-ready formats remains
  labor-intensive and error-prone, creating a bottleneck for model training and
  validation. We introduce AutoMat, an end-to-end, agent-assisted pipeline that
  automatically transforms scanning transmission electron microscopy (STEM)
  images into atomic crystal structures and predicts their physical properties.
  AutoMat combines pattern-adaptive denoising, physics-guided template retrieval,
  symmetry-aware atomic reconstruction, fast relaxation and property prediction
  via MatterSim, and coordinated orchestration across all stages. We propose the
  first dedicated STEM2Mat-Bench for this task and evaluate performance using
  lattice RMSD, formation energy MAE, and structure-matching success rate. By
  orchestrating external tool calls, AutoMat enables a text-only LLM to
  outperform vision-language models in this domain, achieving closed-loop
  reasoning throughout the pipeline. In large-scale experiments over 450
  structure samples, AutoMat substantially outperforms existing multimodal large
  language models and tools. These results validate both AutoMat and
  STEM2Mat-Bench, marking a key step toward bridging microscopy and atomistic
  simulation in materials science.The code and dataset are publicly available at
  https://github.com/yyt-2378/AutoMat and
  https://huggingface.co/datasets/yaotianvector/STEM2Mat.
paper_authors._id: 682c057b36dd2dbca3931150
paper_authors.name: Yaotian Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-22T07:18:15.239Z
paper_authors._id: 64ca1b79ec9a33183aa3bd29
paper_authors.avatarUrl: /avatars/cafeafeccced927aac986575338a804c.svg
paper_authors.fullname: yang
paper_authors.type: user
paper_authors.user: yaotianvector
paper_authors._id: 682c057b36dd2dbca3931151
paper_authors.name: Yiwen Tang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-22T07:18:12.714Z
paper_authors._id: 6552f1ad5d55ccb20e9142a0
paper_authors.avatarUrl: /avatars/0e3e80cba64b5ae0bc5638694ac33dbf.svg
paper_authors.fullname: Ivan Tang
paper_authors.type: user
paper_authors.user: IvanTang
paper_authors._id: 682c057b36dd2dbca3931152
paper_authors.name: Yizhe Chen
paper_authors._id: 682c057b36dd2dbca3931153
paper_authors.name: Xiao Chen
paper_authors._id: 682c057b36dd2dbca3931154
paper_authors.name: Jiangjie Qiu
paper_authors._id: 682c057b36dd2dbca3931155
paper_authors.name: Hao Xiong
paper_authors._id: 682c057b36dd2dbca3931156
paper_authors.name: Haoyu Yin
paper_authors._id: 682c057b36dd2dbca3931157
paper_authors.name: Zhiyao Luo
paper_authors._id: 682c057b36dd2dbca3931158
paper_authors.name: Yifei Zhang
paper_authors._id: 682c057b36dd2dbca3931159
paper_authors.name: Sijia Tao
paper_authors._id: 682c057b36dd2dbca393115a
paper_authors.name: Wentao Li
paper_authors._id: 682c057b36dd2dbca393115b
paper_authors.name: Qinghua Zhang
paper_authors._id: 682c057b36dd2dbca393115c
paper_authors.name: Yuqiang Li
paper_authors._id: 682c057b36dd2dbca393115d
paper_authors.name: Wanli Ouyang
paper_authors._id: 682c057b36dd2dbca393115e
paper_authors.name: Bin Zhao
paper_authors._id: 682c057b36dd2dbca393115f
paper_authors.name: Xiaonan Wang
paper_authors._id: 682c057b36dd2dbca3931160
paper_authors.name: Fei Wei
paper_title: AutoMat: Enabling Automated Crystal Structure Reconstruction from
    Microscopy via Agentic Tool Use
paper_summary: Machine learning-based interatomic potentials and force fields depend
  critically on accurate atomic structures, yet such data are scarce due to the
  limited availability of experimentally resolved crystals. Although
  atomic-resolution electron microscopy offers a potential source of structural
  data, converting these images into simulation-ready formats remains
  labor-intensive and error-prone, creating a bottleneck for model training and
  validation. We introduce AutoMat, an end-to-end, agent-assisted pipeline that
  automatically transforms scanning transmission electron microscopy (STEM)
  images into atomic crystal structures and predicts their physical properties.
  AutoMat combines pattern-adaptive denoising, physics-guided template retrieval,
  symmetry-aware atomic reconstruction, fast relaxation and property prediction
  via MatterSim, and coordinated orchestration across all stages. We propose the
  first dedicated STEM2Mat-Bench for this task and evaluate performance using
  lattice RMSD, formation energy MAE, and structure-matching success rate. By
  orchestrating external tool calls, AutoMat enables a text-only LLM to
  outperform vision-language models in this domain, achieving closed-loop
  reasoning throughout the pipeline. In large-scale experiments over 450
  structure samples, AutoMat substantially outperforms existing multimodal large
  language models and tools. These results validate both AutoMat and
  STEM2Mat-Bench, marking a key step toward bridging microscopy and atomistic
  simulation in materials science.The code and dataset are publicly available at
  https://github.com/yyt-2378/AutoMat and
  https://huggingface.co/datasets/yaotianvector/STEM2Mat.
paper_ai_summary: AutoMat, an agent-assisted pipeline, transforms atomic-resolution STEM images into simulation-ready atomic crystal structures and predicts their properties, overcoming the bottleneck in data availability and processing.
paper_ai_keywords: end-to-end pipeline agent-assisted scanning transmission electron microscopy (STEM) pattern-adaptive denoising physics-guided template retrieval symmetry-aware atomic reconstruction fast relaxation property prediction MatterSim STEM2Mat-Bench lattice RMSD formation energy MAE structure-matching success rate multimodal large language models