PlaSma: Making Small Language Models Better Procedural Knowledge Models
  for (Counterfactual) Planning
Procedural planning, which entails decomposing a high-level goal into a
sequence of temporally ordered steps, is an important yet intricate task for
machines. It involves integrating common-sense knowledge to reason about
complex contextualized situations that are often counterfactual, e.g.
"scheduling a doctor's appointment without a phone". While current approaches
show encouraging results using large language models (LLMs), they are hindered
by drawbacks such as costly API calls and reproducibility issues. In this
paper, we advocate planning using smaller language models. We present PlaSma, a
novel two-pronged approach to endow small language models with procedural
knowledge and (counterfactual) planning capabilities. More concretely, we
develop symbolic procedural knowledge distillation to enhance the implicit
knowledge in small language models and an inference-time algorithm to
facilitate more structured and accurate reasoning. In addition, we introduce a
novel task, Counterfactual Planning, that requires a revision of a plan to cope
with a counterfactual situation. In both the original and counterfactual
setting, we show that orders-of-magnitude smaller models (770M-11B parameters)
can compete and often surpass their larger teacher models' capabilities.
6477fb018ab7e732b6da569d
Faeze Brahman
claimed_verified
2023-06-02T07:16:35.946Z
5f99d555777efc07d7f1e36f
/avatars/6a43c0abe4574e8515d380816ab33305.svg
fa brah
user
fabrh
6477fb018ab7e732b6da569e
Chandra Bhagavatula
6477fb018ab7e732b6da569f
Valentina Pyatkin
claimed_verified
2024-01-04T09:33:33.736Z
6556cff80e7a7067a934445f
https://cdn-avatars.huggingface.co/v1/production/uploads/6556cff80e7a7067a934445f/PoT7qQ6tVqLGGrYvGBbr8.jpeg
Valentina Pyatkin
user
valpy
6477fb018ab7e732b6da56a0
Jena D. Hwang
claimed_verified
2025-06-05T07:31:45.995Z
65c157eaf9957360824b0d87
https://cdn-avatars.huggingface.co/v1/production/uploads/65c157eaf9957360824b0d87/c1NWTKsFLATJLR7W6Kmqz.jpeg
jena hwang
user
jenah
6477fb018ab7e732b6da56a1
Xiang Lorraine Li
claimed_verified
2023-11-16T15:51:04.226Z
651dd123a4747c77c11bb960
/avatars/c7d921d80db1d2f2c1179c576fdb50be.svg
Xiang Lorraine Li
user
lorraine921
6477fb018ab7e732b6da56a2
Hirona J. Arai
6477fb018ab7e732b6da56a3
Soumya Sanyal
claimed_verified
2024-02-07T11:17:44.784Z
64a72adf3987f4dd3c40d0a6
https://cdn-avatars.huggingface.co/v1/production/uploads/64a72adf3987f4dd3c40d0a6/VPNPN_TRjhJikt4NJPTxq.jpeg
Soumya Sanyal
user
soumyasanyal
6477fb018ab7e732b6da56a4
Keisuke Sakaguchi
6477fb018ab7e732b6da56a5
Xiang Ren
6477fb018ab7e732b6da56a6
Yejin Choi
PlaSma: Making Small Language Models Better Procedural Knowledge Models
  for (Counterfactual) Planning
Procedural planning, which entails decomposing a high-level goal into a
sequence of temporally ordered steps, is an important yet intricate task for
machines. It involves integrating common-sense knowledge to reason about
complex contextualized situations that are often counterfactual, e.g.
"scheduling a doctor's appointment without a phone". While current approaches
show encouraging results using large language models (LLMs), they are hindered
by drawbacks such as costly API calls and reproducibility issues. In this
paper, we advocate planning using smaller language models. We present PlaSma, a
novel two-pronged approach to endow small language models with procedural
knowledge and (counterfactual) planning capabilities. More concretely, we
develop symbolic procedural knowledge distillation to enhance the implicit
knowledge in small language models and an inference-time algorithm to
facilitate more structured and accurate reasoning. In addition, we introduce a
novel task, Counterfactual Planning, that requires a revision of a plan to cope
with a counterfactual situation. In both the original and counterfactual
setting, we show that orders-of-magnitude smaller models (770M-11B parameters)
can compete and often surpass their larger teacher models' capabilities.
PlaSma enhances small language models with procedural knowledge and planning capabilities through symbolic distillation and an inference algorithm, demonstrating competitive performance with larger models.
procedural planning large language models procedural knowledge knowledge distillation procedural knowledge distillation inference-time algorithm symbolic procedural knowledge counterfactual planning small language models