Posterior-Mean Rectified Flow: Towards Minimum MSE Photo-Realistic Image
  Restoration
Photo-realistic image restoration algorithms are typically evaluated by
distortion measures (e.g., PSNR, SSIM) and by perceptual quality measures
(e.g., FID, NIQE), where the desire is to attain the lowest possible distortion
without compromising on perceptual quality. To achieve this goal, current
methods typically attempt to sample from the posterior distribution, or to
optimize a weighted sum of a distortion loss (e.g., MSE) and a perceptual
quality loss (e.g., GAN). Unlike previous works, this paper is concerned
specifically with the optimal estimator that minimizes the MSE under a
constraint of perfect perceptual index, namely where the distribution of the
reconstructed images is equal to that of the ground-truth ones. A recent
theoretical result shows that such an estimator can be constructed by optimally
transporting the posterior mean prediction (MMSE estimate) to the distribution
of the ground-truth images. Inspired by this result, we introduce
Posterior-Mean Rectified Flow (PMRF), a simple yet highly effective algorithm
that approximates this optimal estimator. In particular, PMRF first predicts
the posterior mean, and then transports the result to a high-quality image
using a rectified flow model that approximates the desired optimal transport
map. We investigate the theoretical utility of PMRF and demonstrate that it
consistently outperforms previous methods on a variety of image restoration
tasks.
66fce92874eea66d7bfb5bce
Guy Ohayon
claimed_verified
2024-10-02T07:41:15.156Z
653651369ded17e619f658b7
https://cdn-avatars.huggingface.co/v1/production/uploads/653651369ded17e619f658b7/qvEaG8hR3XfL4f3jxkFu3.jpeg
Guy Ohayon
user
ohayonguy
66fce92874eea66d7bfb5bcf
Tomer Michaeli
admin_assigned
2024-10-02T09:07:39.200Z
6457c3f50664b05a5b1d941d
/avatars/0132be4d068d4b314f6b498b65411461.svg
Tomer Michaeli
user
Michato
66fce92874eea66d7bfb5bd0
Michael Elad
Posterior-Mean Rectified Flow: Towards Minimum MSE Photo-Realistic Image
  Restoration
Photo-realistic image restoration algorithms are typically evaluated by
distortion measures (e.g., PSNR, SSIM) and by perceptual quality measures
(e.g., FID, NIQE), where the desire is to attain the lowest possible distortion
without compromising on perceptual quality. To achieve this goal, current
methods typically attempt to sample from the posterior distribution, or to
optimize a weighted sum of a distortion loss (e.g., MSE) and a perceptual
quality loss (e.g., GAN). Unlike previous works, this paper is concerned
specifically with the optimal estimator that minimizes the MSE under a
constraint of perfect perceptual index, namely where the distribution of the
reconstructed images is equal to that of the ground-truth ones. A recent
theoretical result shows that such an estimator can be constructed by optimally
transporting the posterior mean prediction (MMSE estimate) to the distribution
of the ground-truth images. Inspired by this result, we introduce
Posterior-Mean Rectified Flow (PMRF), a simple yet highly effective algorithm
that approximates this optimal estimator. In particular, PMRF first predicts
the posterior mean, and then transports the result to a high-quality image
using a rectified flow model that approximates the desired optimal transport
map. We investigate the theoretical utility of PMRF and demonstrate that it
consistently outperforms previous methods on a variety of image restoration
tasks.
The paper presents Posterior-Mean Rectified Flow (PMRF), an algorithm that combines posterior mean prediction with rectified flow to achieve optimal image restoration by balancing distortion and perceptual quality.
posterior distribution distortion loss perceptual quality loss MSE GAN posterior mean prediction MMSE estimate optimal transport PMRF rectified flow model optimal transport map