Language Models can be Logical Solvers
Logical reasoning is a fundamental aspect of human intelligence and a key
component of tasks like problem-solving and decision-making. Recent
advancements have enabled Large Language Models (LLMs) to potentially exhibit
reasoning capabilities, but complex logical reasoning remains a challenge. The
state-of-the-art, solver-augmented language models, use LLMs to parse natural
language logical questions into symbolic representations first and then adopt
external logical solvers to take in the symbolic representations and output the
answers. Despite their impressive performance, any parsing errors will
inevitably result in the failure of the execution of the external logical
solver and no answer to the logical questions. In this paper, we introduce
LoGiPT, a novel language model that directly emulates the reasoning processes
of logical solvers and bypasses the parsing errors by learning to strict
adherence to solver syntax and grammar. LoGiPT is fine-tuned on a newly
constructed instruction-tuning dataset derived from revealing and refining the
invisible reasoning process of deductive solvers. Experimental results on two
public deductive reasoning datasets demonstrate that LoGiPT outperforms
state-of-the-art solver-augmented LMs and few-shot prompting methods on
competitive LLMs like ChatGPT or GPT-4.
6551911a0c11ee1eb9ff5807
Jiazhan Feng
admin_assigned
2023-11-13T11:01:06.873Z
649a95f951e1ea30f1475530
/avatars/d4d167c3985e702044e8290408dde661.svg
Jamie Jiazhan Feng
user
jzfeng
6551911a0c11ee1eb9ff5808
Ruochen Xu
admin_assigned
2023-11-13T11:01:14.926Z
64a234a8d94c3f9ccec8476d
/avatars/633adf359b1a65f458c3b5a7e59f11c0.svg
Ruochen Xu
user
ruochenx
6551911a0c11ee1eb9ff5809
Junheng Hao
admin_assigned
2023-11-13T11:01:21.904Z
5f04c4394ec31d33a72116d6
/avatars/75d4b9020070e73604b12e5adc1c8201.svg
Junheng Hao
user
jeffhao
6551911a0c11ee1eb9ff580a
Hiteshi Sharma
6551911a0c11ee1eb9ff580b
Yelong Shen
admin_assigned
2023-11-13T11:02:06.617Z
6454c337a13edf669cd5d8ea
/avatars/a383a0dda7c2ef6a0d6c3c64651f42ff.svg
Yelong Shen
user
uuu6
6551911a0c11ee1eb9ff580c
Dongyan Zhao
admin_assigned
2023-11-13T11:02:47.258Z
629ee8171fec321a2d3c458d
/avatars/a8ae78b193ba2a43c6e36e185e0ef2b9.svg
Zhaodong Yan
user
Anhforth
6551911a0c11ee1eb9ff580d
Weizhu Chen
admin_assigned
2023-11-13T11:02:54.267Z
64da876370446182be5b608d
/avatars/e412fdc71404ecdf638e416846e3ebfb.svg
Weizhu Chen
user
chenweizhu
Language Models can be Logical Solvers
Logical reasoning is a fundamental aspect of human intelligence and a key
component of tasks like problem-solving and decision-making. Recent
advancements have enabled Large Language Models (LLMs) to potentially exhibit
reasoning capabilities, but complex logical reasoning remains a challenge. The
state-of-the-art, solver-augmented language models, use LLMs to parse natural
language logical questions into symbolic representations first and then adopt
external logical solvers to take in the symbolic representations and output the
answers. Despite their impressive performance, any parsing errors will
inevitably result in the failure of the execution of the external logical
solver and no answer to the logical questions. In this paper, we introduce
LoGiPT, a novel language model that directly emulates the reasoning processes
of logical solvers and bypasses the parsing errors by learning to strict
adherence to solver syntax and grammar. LoGiPT is fine-tuned on a newly
constructed instruction-tuning dataset derived from revealing and refining the
invisible reasoning process of deductive solvers. Experimental results on two
public deductive reasoning datasets demonstrate that LoGiPT outperforms
state-of-the-art solver-augmented LMs and few-shot prompting methods on
competitive LLMs like ChatGPT or GPT-4.
LoGiPT, a new language model, directly emulates logical solver reasoning processes and outperforms state-of-the-art solver-augmented LMs and few-shot prompting methods on deductive reasoning tasks.
Large Language Models LLMs logical reasoning symbolic representations logical solvers LoGiPT deductive solvers instruction-tuning few-shot prompting ChatGPT GPT-4