DeLLMa: A Framework for Decision Making Under Uncertainty with Large Language Models
<|reference_start|>DeLLMa: A Framework for Decision Making Under Uncertainty with Large Language Models: The potential of large language models (LLMs) as decision support tools is increasingly being explored in fields such as business, engineering, and medicine, which often face challenging tasks of decision-making under uncertainty. In this paper, we show that directly prompting LLMs on these types of decision-making problems can yield poor results, especially as the problem complexity increases. To aid in these tasks, we propose DeLLMa (Decision-making Large Language Model assistant), a framework designed to enhance decision-making accuracy in uncertain environments. DeLLMa involves a multi-step scaffolding procedure, drawing upon principles from decision theory and utility theory, to provide a rational and human-auditable decision-making process. We validate our framework on multiple realistic decision-making environments, demonstrating that DeLLMa can consistently enhance the decision-making performance of leading language models, and achieve up to a 40% increase in accuracy over competing methods.<|reference_end|>
@article{liu2024dellma:,
    title={DeLLMa: Decision Making Under Uncertainty with Large Language Models},
    author={Ollie Liu, Deqing Fu, Dani Yogatama, Willie Neiswanger},
    journal={arXiv preprint arXiv:2402.02392},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.02392},
    primaryClass={cs.AI cs.CL cs.LG}
}