MOOSE-Chem3: Toward Experiment-Guided Hypothesis Ranking via Simulated
  Experimental Feedback
Hypothesis ranking is a crucial component of automated scientific discovery,
particularly in natural sciences where wet-lab experiments are costly and
throughput-limited. Existing approaches focus on pre-experiment ranking,
relying solely on large language model's internal reasoning without
incorporating empirical outcomes from experiments. We introduce the task of
experiment-guided ranking, which aims to prioritize candidate hypotheses based
on the results of previously tested ones. However, developing such strategies
is challenging due to the impracticality of repeatedly conducting real
experiments in natural science domains. To address this, we propose a simulator
grounded in three domain-informed assumptions, modeling hypothesis performance
as a function of similarity to a known ground truth hypothesis, perturbed by
noise. We curate a dataset of 124 chemistry hypotheses with experimentally
reported outcomes to validate the simulator. Building on this simulator, we
develop a pseudo experiment-guided ranking method that clusters hypotheses by
shared functional characteristics and prioritizes candidates based on insights
derived from simulated experimental feedback. Experiments show that our method
outperforms pre-experiment baselines and strong ablations.
68341f661d53989a8ecb685d
Wanhao Liu
claimed_verified
2025-05-26T08:10:54.880Z
6684b284dc7b0ae2cc67660c
/avatars/54b3c0c4d808f293d78085d4d504570a.svg
liuwanhao
user
wanhaoliu
68341f661d53989a8ecb685e
Zonglin Yang
claimed_verified
2025-05-26T08:08:35.046Z
646a11791556443f24b582e9
/avatars/b54d416ddca2f124492ba51bc4b95fd2.svg
Zonglin Yang
user
ZonglinY
68341f661d53989a8ecb685f
Jue Wang
68341f661d53989a8ecb6860
Lidong Bing
claimed_verified
2025-07-22T07:54:38.527Z
6454685a548f22be598414c4
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/eMjMWKJ-AouF7eY1-RzGF.jpeg
Lidong Bing
user
LidongBing
68341f661d53989a8ecb6861
Di Zhang
claimed_verified
2025-05-26T08:40:05.397Z
64bce15bafd1e46c5504ad38
https://cdn-avatars.huggingface.co/v1/production/uploads/64bce15bafd1e46c5504ad38/vkEjiu-mIagKlrXzDH75o.png
Di Zhang
user
di-zhang-fdu
68341f661d53989a8ecb6862
Dongzhan Zhou
68341f661d53989a8ecb6863
Yuqiang Li
68341f661d53989a8ecb6864
Houqiang Li
68341f661d53989a8ecb6865
Erik Cambria
68341f661d53989a8ecb6866
Wanli Ouyang
MOOSE-Chem3: Toward Experiment-Guided Hypothesis Ranking via Simulated
  Experimental Feedback
Hypothesis ranking is a crucial component of automated scientific discovery,
particularly in natural sciences where wet-lab experiments are costly and
throughput-limited. Existing approaches focus on pre-experiment ranking,
relying solely on large language model's internal reasoning without
incorporating empirical outcomes from experiments. We introduce the task of
experiment-guided ranking, which aims to prioritize candidate hypotheses based
on the results of previously tested ones. However, developing such strategies
is challenging due to the impracticality of repeatedly conducting real
experiments in natural science domains. To address this, we propose a simulator
grounded in three domain-informed assumptions, modeling hypothesis performance
as a function of similarity to a known ground truth hypothesis, perturbed by
noise. We curate a dataset of 124 chemistry hypotheses with experimentally
reported outcomes to validate the simulator. Building on this simulator, we
develop a pseudo experiment-guided ranking method that clusters hypotheses by
shared functional characteristics and prioritizes candidates based on insights
derived from simulated experimental feedback. Experiments show that our method
outperforms pre-experiment baselines and strong ablations.
A novel simulator and experiment-guided ranking method improve hypothesis prioritization in scientific discovery by incorporating simulated experimental outcomes.
hypothesis ranking automated scientific discovery natural sciences wet-lab experiments large language model pre-experiment ranking experiment-guided ranking hypothesis performance similarity noise dataset pseudo experiment-guided ranking clustering functional characteristics simulated experimental feedback