Large Language Model Programs
In recent years, large pre-trained language models (LLMs) have demonstrated
the ability to follow instructions and perform novel tasks from a few examples.
The possibility to parameterise an LLM through such in-context examples widens
their capability at a much lower cost than finetuning. We extend this line of
reasoning and present a method which further expands the capabilities of an LLM
by embedding it within an algorithm or program. To demonstrate the benefits of
this approach, we present an illustrative example of evidence-supported
question-answering. We obtain a 6.4\% improvement over the chain of thought
baseline through a more algorithmic approach without any finetuning.
Furthermore, we highlight recent work from this perspective and discuss the
advantages and disadvantages in comparison to the standard approaches.
645af7fd7d8ec84b7b5e68b7
Imanol Schlag
645af7fd7d8ec84b7b5e68b8
Sainbayar Sukhbaatar
claimed_verified
2024-07-30T16:01:39.688Z
66a8611eb51510d82ed54231
/avatars/ad559e774fee4914091b82c9831ae2a2.svg
Sainbayar Sukhbaatar
user
sainbar
645af7fd7d8ec84b7b5e68b9
Asli Celikyilmaz
645af7fd7d8ec84b7b5e68ba
Wen-tau Yih
645af7fd7d8ec84b7b5e68bb
Jason Weston
645af7fd7d8ec84b7b5e68bc
Jürgen Schmidhuber
645af7fd7d8ec84b7b5e68bd
Xian Li
Large Language Model Programs
In recent years, large pre-trained language models (LLMs) have demonstrated
the ability to follow instructions and perform novel tasks from a few examples.
The possibility to parameterise an LLM through such in-context examples widens
their capability at a much lower cost than finetuning. We extend this line of
reasoning and present a method which further expands the capabilities of an LLM
by embedding it within an algorithm or program. To demonstrate the benefits of
this approach, we present an illustrative example of evidence-supported
question-answering. We obtain a 6.4\% improvement over the chain of thought
baseline through a more algorithmic approach without any finetuning.
Furthermore, we highlight recent work from this perspective and discuss the
advantages and disadvantages in comparison to the standard approaches.
Embedding large pre-trained language models within algorithms improves question-answering performance by 6.4% compared to chain-of-thought methods without fine-tuning.
pre-trained language models in-context examples parameterisation algorithmic approach evidence-supported question-answering chain of thought