Using Large Language Models to Automate and Expedite Reinforcement Learning with Reward Machine
<|reference_start|>Using Large Language Models to Automate and Expedite Reinforcement Learning with Reward Machine: We present LARL-RM (Large language model-generated Automaton for Reinforcement Learning with Reward Machine) algorithm in order to encode high-level knowledge into reinforcement learning using automaton to expedite the reinforcement learning. Our method uses Large Language Models (LLM) to obtain high-level domain-specific knowledge using prompt engineering instead of providing the reinforcement learning algorithm directly with the high-level knowledge which requires an expert to encode the automaton. We use chain-of-thought and few-shot methods for prompt engineering and demonstrate that our method works using these approaches. Additionally, LARL-RM allows for fully closed-loop reinforcement learning without the need for an expert to guide and supervise the learning since LARL-RM can use the LLM directly to generate the required high-level knowledge for the task at hand. We also show the theoretical guarantee of our algorithm to converge to an optimal policy. We demonstrate that LARL-RM speeds up the convergence by 30% by implementing our method in two case studies.<|reference_end|>
@article{alsadat2024using,
    title={Using Large Language Models to Automate and Expedite Reinforcement
  Learning with Reward Machine},
    author={Shayan Meshkat Alsadat, Jean-Raphael Gaglione, Daniel Neider, Ufuk
  Topcu, and Zhe Xu},
    journal={arXiv preprint arXiv:2402.07069},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.07069},
    primaryClass={cs.LG cs.AI cs.CL}
}