Large Language Model Cascades with Mixture of Thoughts Representations
  for Cost-efficient Reasoning
Large language models (LLMs) such as GPT-4 have exhibited remarkable
performance in a variety of tasks, but this strong performance often comes with
the high expense of using paid API services. In this paper, we are motivated to
study building an LLM cascade to save the cost of using LLMs, particularly for
performing reasoning (e.g., mathematical, causal) tasks. Our cascade pipeline
follows the intuition that simpler questions can be addressed by a weaker but
more affordable LLM, whereas only the challenging questions necessitate the
stronger and more expensive LLM. To realize this decision-making, we consider
the "answer consistency" of the weaker LLM as a signal of the question
difficulty and propose several methods for the answer sampling and consistency
checking, including one leveraging a mixture of two thought representations
(i.e., Chain-of-Thought and Program-of-Thought). Through experiments on six
reasoning benchmark datasets, with GPT-3.5-turbo and GPT-4 being the weaker and
stronger LLMs, respectively, we demonstrate that our proposed LLM cascades can
achieve performance comparable to using solely the stronger LLM but require
only 40% of its cost.
651f611382dbe82d3f63b631
Murong Yue
admin_assigned
2023-10-09T08:57:41.787Z
63e1ac9f7fbb6ae4d4f312a3
/avatars/7d999543d3ca3df3fc8bb96cd3a3f8cb.svg
Murong Yue
user
MurongYue
651f611382dbe82d3f63b632
Jie Zhao
claimed_verified
2023-10-10T08:17:34.298Z
652486d606e839c5540af4d3
/avatars/035acfaa1b6224853da0aba60d1bf0f7.svg
Jie Zhao
user
jie-zhao
651f611382dbe82d3f63b633
Min Zhang
651f611382dbe82d3f63b634
Liang Du
651f611382dbe82d3f63b635
Ziyu Yao
admin_assigned
2023-10-09T08:58:52.259Z
62d87e143cd901df9446a37a
/avatars/b9a7d24bbe6e49ac90902fafee3a74e7.svg
Ziyu Yao
user
ziyuyao
Large Language Model Cascades with Mixture of Thoughts Representations
  for Cost-efficient Reasoning
Large language models (LLMs) such as GPT-4 have exhibited remarkable
performance in a variety of tasks, but this strong performance often comes with
the high expense of using paid API services. In this paper, we are motivated to
study building an LLM cascade to save the cost of using LLMs, particularly for
performing reasoning (e.g., mathematical, causal) tasks. Our cascade pipeline
follows the intuition that simpler questions can be addressed by a weaker but
more affordable LLM, whereas only the challenging questions necessitate the
stronger and more expensive LLM. To realize this decision-making, we consider
the "answer consistency" of the weaker LLM as a signal of the question
difficulty and propose several methods for the answer sampling and consistency
checking, including one leveraging a mixture of two thought representations
(i.e., Chain-of-Thought and Program-of-Thought). Through experiments on six
reasoning benchmark datasets, with GPT-3.5-turbo and GPT-4 being the weaker and
stronger LLMs, respectively, we demonstrate that our proposed LLM cascades can
achieve performance comparable to using solely the stronger LLM but require
only 40% of its cost.
An LLM cascade system reduces costs by using a less expensive LLM for simpler tasks and a more expensive LLM for challenging tasks, maintaining performance through answer consistency checks.
LLM cascade answer consistency Chain-of-Thought Program-of-Thought reasoning benchmark datasets