title: Natural Language Reinforcement Learning
summary: Reinforcement Learning (RL) mathematically formulates decision-making with
  Markov Decision Process (MDP). With MDPs, researchers have achieved remarkable
  breakthroughs across various domains, including games, robotics, and language
  models. This paper seeks a new possibility, Natural Language Reinforcement
  Learning (NLRL), by extending traditional MDP to natural language-based
  representation space. Specifically, NLRL innovatively redefines RL principles,
  including task objectives, policy, value function, Bellman equation, and policy
  iteration, into their language counterparts. With recent advancements in large
  language models (LLMs), NLRL can be practically implemented to achieve RL-like
  policy and value improvement by either pure prompting or gradient-based
  training. Experiments over Maze, Breakthrough, and Tic-Tac-Toe games
  demonstrate the effectiveness, efficiency, and interpretability of the NLRL
  framework among diverse use cases. Our code will be released at
  https://github.com/waterhorse1/Natural-language-RL.
paper_authors._id: 673fe3f3d2dba4a5fb919036
paper_authors.name: Xidong Feng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-22T13:17:03.910Z
paper_authors._id: 640c99124b5450a208ae6443
paper_authors.avatarUrl: /avatars/3309f1c8f1340a93e8b7a22fb38069fa.svg
paper_authors.fullname: Xidong Feng
paper_authors.type: user
paper_authors.user: Waterhorse
paper_authors._id: 673fe3f3d2dba4a5fb919037
paper_authors.name: Ziyu Wan
paper_authors._id: 673fe3f3d2dba4a5fb919038
paper_authors.name: Haotian Fu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-22T13:17:15.019Z
paper_authors._id: 64c960f3d45e142a0ecdd2f4
paper_authors.avatarUrl: /avatars/ccf68aea3c0fa5bcd5ca4d747405b347.svg
paper_authors.fullname: Haotian Fu
paper_authors.type: user
paper_authors.user: Minusadd
paper_authors._id: 673fe3f3d2dba4a5fb919039
paper_authors.name: Bo Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-22T10:02:02.835Z
paper_authors._id: 635e3a76106f984574c36409
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1667120725800-635e3a76106f984574c36409.png
paper_authors.fullname: Bo Liu
paper_authors.type: user
paper_authors.user: Benjamin-eecs
paper_authors._id: 673fe3f3d2dba4a5fb91903a
paper_authors.name: Mengyue Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-22T13:17:20.998Z
paper_authors._id: 64bd68b0e38420aabad26a80
paper_authors.avatarUrl: /avatars/fdecc96ddf7ee1809c1dfc8d0807161d.svg
paper_authors.fullname: Mengyue Yang
paper_authors.type: user
paper_authors.user: ymy4323460
paper_authors._id: 673fe3f3d2dba4a5fb91903b
paper_authors.name: Girish A. Koushik
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-22T13:17:30.013Z
paper_authors._id: 6065c5161e776d68dd40b18d
paper_authors.avatarUrl: /avatars/13f65b99e2b6dbd0be9f88cb5e967175.svg
paper_authors.fullname: Girish Koushik
paper_authors.type: user
paper_authors.user: mindhunter23
paper_authors._id: 673fe3f3d2dba4a5fb91903c
paper_authors.name: Zhiyuan Hu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-22T13:17:55.835Z
paper_authors._id: 64351475901c5734bcb64248
paper_authors.avatarUrl: /avatars/12346d4301c1bfb00ce0ea128a93cc15.svg
paper_authors.fullname: Zhiyuan Hu
paper_authors.type: user
paper_authors.user: zhiyuanhucs
paper_authors._id: 673fe3f3d2dba4a5fb91903d
paper_authors.name: Ying Wen
paper_authors._id: 673fe3f3d2dba4a5fb91903e
paper_authors.name: Jun Wang
paper_title: Natural Language Reinforcement Learning
paper_summary: Reinforcement Learning (RL) mathematically formulates decision-making with
  Markov Decision Process (MDP). With MDPs, researchers have achieved remarkable
  breakthroughs across various domains, including games, robotics, and language
  models. This paper seeks a new possibility, Natural Language Reinforcement
  Learning (NLRL), by extending traditional MDP to natural language-based
  representation space. Specifically, NLRL innovatively redefines RL principles,
  including task objectives, policy, value function, Bellman equation, and policy
  iteration, into their language counterparts. With recent advancements in large
  language models (LLMs), NLRL can be practically implemented to achieve RL-like
  policy and value improvement by either pure prompting or gradient-based
  training. Experiments over Maze, Breakthrough, and Tic-Tac-Toe games
  demonstrate the effectiveness, efficiency, and interpretability of the NLRL
  framework among diverse use cases. Our code will be released at
  https://github.com/waterhorse1/Natural-language-RL.
paper_ai_summary: Natural Language Reinforcement Learning (NLRL) redefines traditional RL concepts in a language-based framework, leveraging large language models to achieve efficient and interpretable policy improvement.
paper_ai_keywords: Reinforcement Learning NLRL Markov Decision Process MDP task objectives policy value function Bellman equation policy iteration large language models LLMs gradient-based training