title: Learning to Decode Collaboratively with Multiple Language Models
summary: We propose a method to teach multiple large language models (LLM) to
  collaborate by interleaving their generations at the token level. We model the
  decision of which LLM generates the next token as a latent variable. By
  optimizing the marginal likelihood of a training set under our latent variable
  model, the base LLM automatically learns when to generate itself and when to
  call on one of the ``assistant'' language models to generate, all without
  direct supervision. Token-level collaboration during decoding allows for a
  fusion of each model's expertise in a manner tailored to the specific task at
  hand. Our collaborative decoding is especially useful in cross-domain settings
  where a generalist base LLM learns to invoke domain expert models. On
  instruction-following, domain-specific QA, and reasoning tasks, we show that
  the performance of the joint system exceeds that of the individual models.
  Through qualitative analysis of the learned latent decisions, we show models
  trained with our method exhibit several interesting collaboration patterns,
  e.g., template-filling. Our code is available at
  https://github.com/clinicalml/co-llm.
paper_authors._id: 65e95204014f4be34f6ce7ed
paper_authors.name: Shannon Zejiang Shen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-07T09:59:44.246Z
paper_authors._id: 613f897ffbfd59f147a88c81
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1659455519119-613f897ffbfd59f147a88c81.jpeg
paper_authors.fullname: Shannon Shen
paper_authors.type: user
paper_authors.user: shannons
paper_authors._id: 65e95204014f4be34f6ce7ee
paper_authors.name: Hunter Lang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-07T09:59:51.040Z
paper_authors._id: 62cdc3a5342b1d5dab9012d4
paper_authors.avatarUrl: /avatars/2ec0afa7ef65a014f8926407da26ab17.svg
paper_authors.fullname: Hunter Lang
paper_authors.type: user
paper_authors.user: hjl
paper_authors._id: 65e95204014f4be34f6ce7ef
paper_authors.name: Bailin Wang
paper_authors._id: 65e95204014f4be34f6ce7f0
paper_authors.name: Yoon Kim
paper_authors._id: 65e95204014f4be34f6ce7f1
paper_authors.name: David Sontag
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-07T10:00:48.683Z
paper_authors._id: 634ffb851d81beb8e244a4bb
paper_authors.avatarUrl: /avatars/16fff0e1a8cc8466240aa084eee467f3.svg
paper_authors.fullname: David Sontag
paper_authors.type: user
paper_authors.user: dsontag
paper_title: Learning to Decode Collaboratively with Multiple Language Models
paper_summary: We propose a method to teach multiple large language models (LLM) to
  collaborate by interleaving their generations at the token level. We model the
  decision of which LLM generates the next token as a latent variable. By
  optimizing the marginal likelihood of a training set under our latent variable
  model, the base LLM automatically learns when to generate itself and when to
  call on one of the ``assistant'' language models to generate, all without
  direct supervision. Token-level collaboration during decoding allows for a
  fusion of each model's expertise in a manner tailored to the specific task at
  hand. Our collaborative decoding is especially useful in cross-domain settings
  where a generalist base LLM learns to invoke domain expert models. On
  instruction-following, domain-specific QA, and reasoning tasks, we show that
  the performance of the joint system exceeds that of the individual models.
  Through qualitative analysis of the learned latent decisions, we show models
  trained with our method exhibit several interesting collaboration patterns,
  e.g., template-filling. Our code is available at
  https://github.com/clinicalml/co-llm.
paper_ai_summary: Multiple large language models collaborate by interleaving token-level generation decisions, optimizing performance across tasks without direct supervision.
paper_ai_keywords: large language models LLM token-level collaboration latent variable marginal likelihood joint system instruction-following domain-specific QA reasoning tasks latent decisions template-filling