title: DSPy: Compiling Declarative Language Model Calls into Self-Improving
    Pipelines
summary: The ML community is rapidly exploring techniques for prompting language
  models (LMs) and for stacking them into pipelines that solve complex tasks.
  Unfortunately, existing LM pipelines are typically implemented using hard-coded
  "prompt templates", i.e. lengthy strings discovered via trial and error. Toward
  a more systematic approach for developing and optimizing LM pipelines, we
  introduce DSPy, a programming model that abstracts LM pipelines as text
  transformation graphs, i.e. imperative computational graphs where LMs are
  invoked through declarative modules. DSPy modules are parameterized, meaning
  they can learn (by creating and collecting demonstrations) how to apply
  compositions of prompting, finetuning, augmentation, and reasoning techniques.
  We design a compiler that will optimize any DSPy pipeline to maximize a given
  metric. We conduct two case studies, showing that succinct DSPy programs can
  express and optimize sophisticated LM pipelines that reason about math word
  problems, tackle multi-hop retrieval, answer complex questions, and control
  agent loops. Within minutes of compiling, a few lines of DSPy allow GPT-3.5 and
  llama2-13b-chat to self-bootstrap pipelines that outperform standard few-shot
  prompting (generally by over 25% and 65%, respectively) and pipelines with
  expert-created demonstrations (by up to 5-46% and 16-40%, respectively). On top
  of that, DSPy programs compiled to open and relatively small LMs like
  770M-parameter T5 and llama2-13b-chat are competitive with approaches that rely
  on expert-written prompt chains for proprietary GPT-3.5. DSPy is available at
  https://github.com/stanfordnlp/dspy
paper_authors._id: 651f6316e3c86280559d2eb2
paper_authors.name: Omar Khattab
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-09T08:47:57.705Z
paper_authors._id: 5f764762f1e7ef6e919a1e0f
paper_authors.avatarUrl: /avatars/9445f6efa3f78a870e338988d9ce5573.svg
paper_authors.fullname: Omar Khattab
paper_authors.type: user
paper_authors.user: okhattab
paper_authors._id: 651f6316e3c86280559d2eb3
paper_authors.name: Arnav Singhvi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-09T08:48:05.597Z
paper_authors._id: 64b835e3d2176850e0abadcd
paper_authors.avatarUrl: /avatars/e1e5fe6f814d549721927daab4d7e8b6.svg
paper_authors.fullname: Arnav Singhvi
paper_authors.type: user
paper_authors.user: arnavs11
paper_authors._id: 651f6316e3c86280559d2eb4
paper_authors.name: Paridhi Maheshwari
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-09T08:48:13.348Z
paper_authors._id: 640e34432f9c7b364d157408
paper_authors.avatarUrl: /avatars/59552e26a13183607ebb668669476b39.svg
paper_authors.fullname: Paridhi Maheshwari
paper_authors.type: user
paper_authors.user: paridhi
paper_authors._id: 651f6316e3c86280559d2eb5
paper_authors.name: Zhiyuan Zhang
paper_authors._id: 651f6316e3c86280559d2eb6
paper_authors.name: Keshav Santhanam
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-09T08:48:44.241Z
paper_authors._id: 633328b6f695c3df38d97434
paper_authors.avatarUrl: /avatars/017e15cad6406653a9340cdbbee7dc51.svg
paper_authors.fullname: Keshav Santhanam
paper_authors.type: user
paper_authors.user: santhnm2
paper_authors._id: 651f6316e3c86280559d2eb7
paper_authors.name: Sri Vardhamanan
paper_authors._id: 651f6316e3c86280559d2eb8
paper_authors.name: Saiful Haq
paper_authors._id: 651f6316e3c86280559d2eb9
paper_authors.name: Ashutosh Sharma
paper_authors._id: 651f6316e3c86280559d2eba
paper_authors.name: Thomas T. Joshi
paper_authors._id: 651f6316e3c86280559d2ebb
paper_authors.name: Hanna Moazam
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-09T08:49:44.690Z
paper_authors._id: 6505eb335aa2ad0324c205b5
paper_authors.avatarUrl: /avatars/982a90c601c9d5eb4f2c3828ebc106ee.svg
paper_authors.fullname: Hanna Moazam
paper_authors.type: user
paper_authors.user: hmoazam
paper_authors._id: 651f6316e3c86280559d2ebc
paper_authors.name: Heather Miller
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-10-09T07:51:21.809Z
paper_authors._id: 6520235b94f3a1a209c1bcc1
paper_authors.avatarUrl: /avatars/49a4815846825cd1334fa080c6e71c5d.svg
paper_authors.fullname: Heather Miller
paper_authors.type: user
paper_authors.user: heathercmiller
paper_authors._id: 651f6316e3c86280559d2ebd
paper_authors.name: Matei Zaharia
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-09T08:49:52.303Z
paper_authors._id: 642509c6d476e4ad5568dd31
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/642509c6d476e4ad5568dd31/dPiEV70hMeuY-cveLNPQg.jpeg
paper_authors.fullname: Matei Zaharia
paper_authors.type: user
paper_authors.user: mateiz
paper_authors._id: 651f6316e3c86280559d2ebe
paper_authors.name: Christopher Potts
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-09T08:50:04.621Z
paper_authors._id: 62606790e3dd8b49f360f050
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1650485126923-noauth.png
paper_authors.fullname: Chris Potts
paper_authors.type: user
paper_authors.user: cgpotts
paper_title: DSPy: Compiling Declarative Language Model Calls into Self-Improving
    Pipelines
paper_summary: The ML community is rapidly exploring techniques for prompting language
  models (LMs) and for stacking them into pipelines that solve complex tasks.
  Unfortunately, existing LM pipelines are typically implemented using hard-coded
  "prompt templates", i.e. lengthy strings discovered via trial and error. Toward
  a more systematic approach for developing and optimizing LM pipelines, we
  introduce DSPy, a programming model that abstracts LM pipelines as text
  transformation graphs, i.e. imperative computational graphs where LMs are
  invoked through declarative modules. DSPy modules are parameterized, meaning
  they can learn (by creating and collecting demonstrations) how to apply
  compositions of prompting, finetuning, augmentation, and reasoning techniques.
  We design a compiler that will optimize any DSPy pipeline to maximize a given
  metric. We conduct two case studies, showing that succinct DSPy programs can
  express and optimize sophisticated LM pipelines that reason about math word
  problems, tackle multi-hop retrieval, answer complex questions, and control
  agent loops. Within minutes of compiling, a few lines of DSPy allow GPT-3.5 and
  llama2-13b-chat to self-bootstrap pipelines that outperform standard few-shot
  prompting (generally by over 25% and 65%, respectively) and pipelines with
  expert-created demonstrations (by up to 5-46% and 16-40%, respectively). On top
  of that, DSPy programs compiled to open and relatively small LMs like
  770M-parameter T5 and llama2-13b-chat are competitive with approaches that rely
  on expert-written prompt chains for proprietary GPT-3.5. DSPy is available at
  https://github.com/stanfordnlp/dspy
paper_ai_summary: DSPy is a programming model for developing and optimizing language model pipelines by abstracting them as text transformation graphs with parameterized modules that learn through demonstrations.
paper_ai_keywords: language models LM pipelines prompt templates textual transformation graphs parameterized modules compiler reasoning techniques few-shot prompting agent loops math word problems multi-hop retrieval complex questions T5 llama2-13b-chat standard few-shot prompting