title: PLANNER: Generating Diversified Paragraph via Latent Language Diffusion
    Model
summary: Autoregressive models for text sometimes generate repetitive and low-quality
  output because errors accumulate during the steps of generation. This issue is
  often attributed to exposure bias - the difference between how a model is
  trained, and how it is used during inference. Denoising diffusion models
  provide an alternative approach in which a model can revisit and revise its
  output. However, they can be computationally expensive and prior efforts on
  text have led to models that produce less fluent output compared to
  autoregressive models, especially for longer text and paragraphs. In this
  paper, we propose PLANNER, a model that combines latent semantic diffusion with
  autoregressive generation, to generate fluent text while exercising global
  control over paragraphs. The model achieves this by combining an autoregressive
  "decoding" module with a "planning" module that uses latent diffusion to
  generate semantic paragraph embeddings in a coarse-to-fine manner. The proposed
  method is evaluated on various conditional generation tasks, and results on
  semantic generation, text completion and summarization show its effectiveness
  in generating high-quality long-form text in an efficient manner.
paper_authors._id: 647eb35d1a1fcad2fdc6006c
paper_authors.name: Yizhe Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-06T17:23:37.181Z
paper_authors._id: 640d1d188aee167ccda1a52a
paper_authors.avatarUrl: /avatars/c320b53a31ffba41bd9ad71da99ebc83.svg
paper_authors.fullname: Yizhe Zhang
paper_authors.type: user
paper_authors.user: YizheZ
paper_authors._id: 647eb35d1a1fcad2fdc6006d
paper_authors.name: Jiatao Gu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-06T21:03:22.734Z
paper_authors._id: 6164e72d73996c363c52e66d
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1634002684894-noauth.png
paper_authors.fullname: Jiatao Gu
paper_authors.type: user
paper_authors.user: thomagram
paper_authors._id: 647eb35d1a1fcad2fdc6006e
paper_authors.name: Zhuofeng Wu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-26T21:10:06.449Z
paper_authors._id: 63c75671656e7822e23df874
paper_authors.avatarUrl: /avatars/6fbe669287405b2b47cafcf4961cfa75.svg
paper_authors.fullname: Zhuofeng Wu
paper_authors.type: user
paper_authors.user: cserxy
paper_authors._id: 647eb35d1a1fcad2fdc6006f
paper_authors.name: Shuangfei Zhai
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-13T15:25:19.804Z
paper_authors._id: 675bf04240726aac79ae8e7b
paper_authors.avatarUrl: /avatars/8ae95255dfce91a6f5e76f14eae9e9e5.svg
paper_authors.fullname: Shuangfei Zhai
paper_authors.type: user
paper_authors.user: shuangfei
paper_authors._id: 647eb35d1a1fcad2fdc60070
paper_authors.name: Josh Susskind
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-06T17:23:41.773Z
paper_authors._id: 6470d2247fd7ecdbd0ec3cc9
paper_authors.avatarUrl: /avatars/52c5eca12499a1aa9bd49c43d4f20685.svg
paper_authors.fullname: Joshua M. Susskind
paper_authors.type: user
paper_authors.user: jsusskind
paper_authors._id: 647eb35d1a1fcad2fdc60071
paper_authors.name: Navdeep Jaitly
paper_title: PLANNER: Generating Diversified Paragraph via Latent Language Diffusion
    Model
paper_summary: Autoregressive models for text sometimes generate repetitive and low-quality
  output because errors accumulate during the steps of generation. This issue is
  often attributed to exposure bias - the difference between how a model is
  trained, and how it is used during inference. Denoising diffusion models
  provide an alternative approach in which a model can revisit and revise its
  output. However, they can be computationally expensive and prior efforts on
  text have led to models that produce less fluent output compared to
  autoregressive models, especially for longer text and paragraphs. In this
  paper, we propose PLANNER, a model that combines latent semantic diffusion with
  autoregressive generation, to generate fluent text while exercising global
  control over paragraphs. The model achieves this by combining an autoregressive
  "decoding" module with a "planning" module that uses latent diffusion to
  generate semantic paragraph embeddings in a coarse-to-fine manner. The proposed
  method is evaluated on various conditional generation tasks, and results on
  semantic generation, text completion and summarization show its effectiveness
  in generating high-quality long-form text in an efficient manner.
paper_ai_keywords: autoregressive models exposure bias denoising diffusion models latent semantic diffusion semantic paragraph embeddings coarse-to-fine manner semantic generation text completion summarization high-quality long-form text