title: Posterior-Mean Rectified Flow: Towards Minimum MSE Photo-Realistic Image
    Restoration
summary: Photo-realistic image restoration algorithms are typically evaluated by
  distortion measures (e.g., PSNR, SSIM) and by perceptual quality measures
  (e.g., FID, NIQE), where the desire is to attain the lowest possible distortion
  without compromising on perceptual quality. To achieve this goal, current
  methods typically attempt to sample from the posterior distribution, or to
  optimize a weighted sum of a distortion loss (e.g., MSE) and a perceptual
  quality loss (e.g., GAN). Unlike previous works, this paper is concerned
  specifically with the optimal estimator that minimizes the MSE under a
  constraint of perfect perceptual index, namely where the distribution of the
  reconstructed images is equal to that of the ground-truth ones. A recent
  theoretical result shows that such an estimator can be constructed by optimally
  transporting the posterior mean prediction (MMSE estimate) to the distribution
  of the ground-truth images. Inspired by this result, we introduce
  Posterior-Mean Rectified Flow (PMRF), a simple yet highly effective algorithm
  that approximates this optimal estimator. In particular, PMRF first predicts
  the posterior mean, and then transports the result to a high-quality image
  using a rectified flow model that approximates the desired optimal transport
  map. We investigate the theoretical utility of PMRF and demonstrate that it
  consistently outperforms previous methods on a variety of image restoration
  tasks.
paper_authors._id: 66fce92874eea66d7bfb5bce
paper_authors.name: Guy Ohayon
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-02T07:41:15.156Z
paper_authors._id: 653651369ded17e619f658b7
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/653651369ded17e619f658b7/qvEaG8hR3XfL4f3jxkFu3.jpeg
paper_authors.fullname: Guy Ohayon
paper_authors.type: user
paper_authors.user: ohayonguy
paper_authors._id: 66fce92874eea66d7bfb5bcf
paper_authors.name: Tomer Michaeli
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-02T09:07:39.200Z
paper_authors._id: 6457c3f50664b05a5b1d941d
paper_authors.avatarUrl: /avatars/0132be4d068d4b314f6b498b65411461.svg
paper_authors.fullname: Tomer Michaeli
paper_authors.type: user
paper_authors.user: Michato
paper_authors._id: 66fce92874eea66d7bfb5bd0
paper_authors.name: Michael Elad
paper_title: Posterior-Mean Rectified Flow: Towards Minimum MSE Photo-Realistic Image
    Restoration
paper_summary: Photo-realistic image restoration algorithms are typically evaluated by
  distortion measures (e.g., PSNR, SSIM) and by perceptual quality measures
  (e.g., FID, NIQE), where the desire is to attain the lowest possible distortion
  without compromising on perceptual quality. To achieve this goal, current
  methods typically attempt to sample from the posterior distribution, or to
  optimize a weighted sum of a distortion loss (e.g., MSE) and a perceptual
  quality loss (e.g., GAN). Unlike previous works, this paper is concerned
  specifically with the optimal estimator that minimizes the MSE under a
  constraint of perfect perceptual index, namely where the distribution of the
  reconstructed images is equal to that of the ground-truth ones. A recent
  theoretical result shows that such an estimator can be constructed by optimally
  transporting the posterior mean prediction (MMSE estimate) to the distribution
  of the ground-truth images. Inspired by this result, we introduce
  Posterior-Mean Rectified Flow (PMRF), a simple yet highly effective algorithm
  that approximates this optimal estimator. In particular, PMRF first predicts
  the posterior mean, and then transports the result to a high-quality image
  using a rectified flow model that approximates the desired optimal transport
  map. We investigate the theoretical utility of PMRF and demonstrate that it
  consistently outperforms previous methods on a variety of image restoration
  tasks.
paper_ai_summary: The paper presents Posterior-Mean Rectified Flow (PMRF), an algorithm that combines posterior mean prediction with rectified flow to achieve optimal image restoration by balancing distortion and perceptual quality.
paper_ai_keywords: posterior distribution distortion loss perceptual quality loss MSE GAN posterior mean prediction MMSE estimate optimal transport PMRF rectified flow model optimal transport map