PhoCoLens: Photorealistic and Consistent Reconstruction in Lensless Imaging
<|reference_start|>PhoCoLens: Photorealistic and Consistent Reconstruction in Lensless Imaging: Lensless cameras offer significant advantages in size, weight, and cost compared to traditional lens-based systems. Without a focusing lens, lensless cameras rely on computational algorithms to recover the scenes from multiplexed measurements. However, current algorithms struggle with inaccurate forward imaging models and insufficient priors to reconstruct high-quality images. To overcome these limitations, we introduce a novel two-stage approach for consistent and photorealistic lensless image reconstruction. The first stage of our approach ensures data consistency by focusing on accurately reconstructing the low-frequency content with a spatially varying deconvolution method that adjusts to changes in the Point Spread Function (PSF) across the camera's field of view. The second stage enhances photorealism by incorporating a generative prior from pre-trained diffusion models. By conditioning on the low-frequency content retrieved in the first stage, the diffusion model effectively reconstructs the high-frequency details that are typically lost in the lensless imaging process, while also maintaining image fidelity. Our method achieves a superior balance between data fidelity and visual quality compared to existing methods, as demonstrated with two popular lensless systems, PhlatCam and DiffuserCam. Project website: https://phocolens.github.io/.<|reference_end|>
@article{cai2024phocolens:,
    title={PhoCoLens: Photorealistic and Consistent Reconstruction in Lensless
  Imaging},
    author={Xin Cai, Zhiyuan You, Hailong Zhang, Wentao Liu, Jinwei Gu, Tianfan
  Xue},
    journal={arXiv preprint arXiv:2409.17996},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.17996},
    primaryClass={eess.IV cs.CV cs.LG}
}