InferAct: Inferring Safe Actions for LLM-Based Agents Through Preemptive Evaluation and Human Feedback
<|reference_start|>InferAct: Inferring Safe Actions for LLM-Based Agents Through Preemptive Evaluation and Human Feedback: A crucial requirement for deploying LLM-based agents in real-life applications is robustness against risky or irreversible mistakes. However, existing research lacks a focus on the preemptive evaluation of reasoning trajectories performed by LLM agents, leading to a gap in ensuring safe and reliable operations. To explore better solutions, this paper introduces InferAct, a novel approach that leverages the Theory-of-Mind capability of LLMs to proactively detect potential errors before critical actions are executed (e.g., "buy-now" in automatic online trading or web shopping). InferAct is also capable of integrating human feedback to prevent irreversible risks and enhance the actor agent's decision-making process. Experiments on three widely used tasks demonstrate the effectiveness of InferAct. The proposed solution presents a novel approach and concrete contributions toward developing LLM agents that can be safely deployed in different environments involving critical decision-making.<|reference_end|>
@article{fang2024inferact:,
    title={InferAct: Inferring Safe Actions for LLM-Based Agents Through Preemptive
  Evaluation and Human Feedback},
    author={Haishuo Fang, Xiaodan Zhu, Iryna Gurevych},
    journal={arXiv preprint arXiv:2407.11843},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.11843},
    primaryClass={cs.CL cs.AI}
}