title: Normalizing Flows are Capable Generative Models
summary: Normalizing Flows (NFs) are likelihood-based models for continuous inputs.
  They have demonstrated promising results on both density estimation and
  generative modeling tasks, but have received relatively little attention in
  recent years. In this work, we demonstrate that NFs are more powerful than
  previously believed. We present TarFlow: a simple and scalable architecture
  that enables highly performant NF models. TarFlow can be thought of as a
  Transformer-based variant of Masked Autoregressive Flows (MAFs): it consists of
  a stack of autoregressive Transformer blocks on image patches, alternating the
  autoregression direction between layers. TarFlow is straightforward to train
  end-to-end, and capable of directly modeling and generating pixels. We also
  propose three key techniques to improve sample quality: Gaussian noise
  augmentation during training, a post training denoising procedure, and an
  effective guidance method for both class-conditional and unconditional
  settings. Putting these together, TarFlow sets new state-of-the-art results on
  likelihood estimation for images, beating the previous best methods by a large
  margin, and generates samples with quality and diversity comparable to
  diffusion models, for the first time with a stand-alone NF model. We make our
  code available at https://github.com/apple/ml-tarflow.
paper_authors._id: 675c831d9e17c3619871d3e6
paper_authors.name: Shuangfei Zhai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-16T11:03:10.096Z
paper_authors._id: 675bf04240726aac79ae8e7b
paper_authors.avatarUrl: /avatars/8ae95255dfce91a6f5e76f14eae9e9e5.svg
paper_authors.fullname: Shuangfei Zhai
paper_authors.type: user
paper_authors.user: shuangfei
paper_authors._id: 675c831d9e17c3619871d3e7
paper_authors.name: Ruixiang Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-16T11:02:43.750Z
paper_authors._id: 61b79ec37b131aad98e42968
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1639423675612-noauth.jpeg
paper_authors.fullname: Ruixiang Zhang
paper_authors.type: user
paper_authors.user: onlogn
paper_authors._id: 675c831d9e17c3619871d3e8
paper_authors.name: Preetum Nakkiran
paper_authors._id: 675c831d9e17c3619871d3e9
paper_authors.name: David Berthelot
paper_authors._id: 675c831d9e17c3619871d3ea
paper_authors.name: Jiatao Gu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-16T11:02:21.807Z
paper_authors._id: 6164e72d73996c363c52e66d
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1634002684894-noauth.png
paper_authors.fullname: Jiatao Gu
paper_authors.type: user
paper_authors.user: thomagram
paper_authors._id: 675c831d9e17c3619871d3eb
paper_authors.name: Huangjie Zheng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-16T11:02:15.470Z
paper_authors._id: 62f6dc90d278a8f3e7877dc0
paper_authors.avatarUrl: /avatars/bce37487397278d5daf6d95f4dc25221.svg
paper_authors.fullname: Huangjie Zheng
paper_authors.type: user
paper_authors.user: hjzheng
paper_authors._id: 675c831d9e17c3619871d3ec
paper_authors.name: Tianrong Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-16T11:02:08.979Z
paper_authors._id: 65d2f3e270354febcb101773
paper_authors.avatarUrl: /avatars/4368d52d19feef195ee5931f038d974b.svg
paper_authors.fullname: chentianrong
paper_authors.type: user
paper_authors.user: kyrieppx
paper_authors._id: 675c831d9e17c3619871d3ed
paper_authors.name: Miguel Angel Bautista
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-16T11:01:56.375Z
paper_authors._id: 65dd58aa29b761fff4bd3a4c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/SD7Vh0i5HwJJsQxXpSTFa.jpeg
paper_authors.fullname: Miguel Angel Bautista
paper_authors.type: user
paper_authors.user: FlippyCode
paper_authors._id: 675c831d9e17c3619871d3ee
paper_authors.name: Navdeep Jaitly
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-16T11:01:49.180Z
paper_authors._id: 631119df88942700629f4aab
paper_authors.avatarUrl: /avatars/251c20a8fd48831a8c0cc0d198a42455.svg
paper_authors.fullname: Navdeep Jaitly
paper_authors.type: user
paper_authors.user: ndjaitly
paper_authors._id: 675c831d9e17c3619871d3ef
paper_authors.name: Josh Susskind
paper_title: Normalizing Flows are Capable Generative Models
paper_summary: Normalizing Flows (NFs) are likelihood-based models for continuous inputs.
  They have demonstrated promising results on both density estimation and
  generative modeling tasks, but have received relatively little attention in
  recent years. In this work, we demonstrate that NFs are more powerful than
  previously believed. We present TarFlow: a simple and scalable architecture
  that enables highly performant NF models. TarFlow can be thought of as a
  Transformer-based variant of Masked Autoregressive Flows (MAFs): it consists of
  a stack of autoregressive Transformer blocks on image patches, alternating the
  autoregression direction between layers. TarFlow is straightforward to train
  end-to-end, and capable of directly modeling and generating pixels. We also
  propose three key techniques to improve sample quality: Gaussian noise
  augmentation during training, a post training denoising procedure, and an
  effective guidance method for both class-conditional and unconditional
  settings. Putting these together, TarFlow sets new state-of-the-art results on
  likelihood estimation for images, beating the previous best methods by a large
  margin, and generates samples with quality and diversity comparable to
  diffusion models, for the first time with a stand-alone NF model. We make our
  code available at https://github.com/apple/ml-tarflow.
paper_ai_keywords: Normalizing Flows (NFs) likelihood-based models density estimation generative modeling TarFlow Transformer-based variant Masked Autoregressive Flows (MAFs) autoregressive Transformer blocks autoregression direction Gaussian noise augmentation post training denoising procedure guidance method class-conditional settings unconditional settings likelihood estimation diffusion models stand-alone NF model