title: Steering Rectified Flow Models in the Vector Field for Controlled Image
    Generation
summary: Diffusion models (DMs) excel in photorealism, image editing, and solving
  inverse problems, aided by classifier-free guidance and image inversion
  techniques. However, rectified flow models (RFMs) remain underexplored for
  these tasks. Existing DM-based methods often require additional training, lack
  generalization to pretrained latent models, underperform, and demand
  significant computational resources due to extensive backpropagation through
  ODE solvers and inversion processes. In this work, we first develop a
  theoretical and empirical understanding of the vector field dynamics of RFMs in
  efficiently guiding the denoising trajectory. Our findings reveal that we can
  navigate the vector field in a deterministic and gradient-free manner.
  Utilizing this property, we propose FlowChef, which leverages the vector field
  to steer the denoising trajectory for controlled image generation tasks,
  facilitated by gradient skipping. FlowChef is a unified framework for
  controlled image generation that, for the first time, simultaneously addresses
  classifier guidance, linear inverse problems, and image editing without the
  need for extra training, inversion, or intensive backpropagation. Finally, we
  perform extensive evaluations and show that FlowChef significantly outperforms
  baselines in terms of performance, memory, and time requirements, achieving new
  state-of-the-art results. Project Page: https://flowchef.github.io.
paper_authors._id: 674e89d1ec384b629bba5e61
paper_authors.name: Maitreya Patel
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-03T11:14:47.767Z
paper_authors._id: 622d2ff38d04fd29a9ccf1a7
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/622d2ff38d04fd29a9ccf1a7/ORpGrlU8Lm_oSEVftcZEK.jpeg
paper_authors.fullname: Maitreya Patel
paper_authors.type: user
paper_authors.user: mpatel57
paper_authors._id: 674e89d1ec384b629bba5e62
paper_authors.name: Song Wen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-04T09:46:25.505Z
paper_authors._id: 638d79bf5869d78eb7e8ccec
paper_authors.avatarUrl: /avatars/c047dc15cd650f91a276115d8d1ad10a.svg
paper_authors.fullname: Song Wen
paper_authors.type: user
paper_authors.user: swen9
paper_authors._id: 674e89d1ec384b629bba5e63
paper_authors.name: Dimitris N. Metaxas
paper_authors._id: 674e89d1ec384b629bba5e64
paper_authors.name: Yezhou Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-03T18:43:20.762Z
paper_authors._id: 6486a850903d5878eb98110f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6486a850903d5878eb98110f/ETSE52Fdq5kVPzy4R5tGt.jpeg
paper_authors.fullname: 'YZ' Yezhou Yang
paper_authors.type: user
paper_authors.user: AiWithoutBorders
paper_title: Steering Rectified Flow Models in the Vector Field for Controlled Image
    Generation
paper_summary: Diffusion models (DMs) excel in photorealism, image editing, and solving
  inverse problems, aided by classifier-free guidance and image inversion
  techniques. However, rectified flow models (RFMs) remain underexplored for
  these tasks. Existing DM-based methods often require additional training, lack
  generalization to pretrained latent models, underperform, and demand
  significant computational resources due to extensive backpropagation through
  ODE solvers and inversion processes. In this work, we first develop a
  theoretical and empirical understanding of the vector field dynamics of RFMs in
  efficiently guiding the denoising trajectory. Our findings reveal that we can
  navigate the vector field in a deterministic and gradient-free manner.
  Utilizing this property, we propose FlowChef, which leverages the vector field
  to steer the denoising trajectory for controlled image generation tasks,
  facilitated by gradient skipping. FlowChef is a unified framework for
  controlled image generation that, for the first time, simultaneously addresses
  classifier guidance, linear inverse problems, and image editing without the
  need for extra training, inversion, or intensive backpropagation. Finally, we
  perform extensive evaluations and show that FlowChef significantly outperforms
  baselines in terms of performance, memory, and time requirements, achieving new
  state-of-the-art results. Project Page: https://flowchef.github.io.
paper_ai_summary: FlowChef is a unified framework for controlled image generation using rectified flow models, achieving improvements over diffusion models in photorealism, image editing, and inverse problems without additional training or intensive backpropagation.
paper_ai_keywords: diffusion models rectified flow models vector field dynamics denoising trajectory gradient skipping classifier guidance linear inverse problems image editing