Real-World Computational Aberration Correction via Quantized Domain-Mixing Representation
<|reference_start|>Real-World Computational Aberration Correction via Quantized Domain-Mixing Representation: Relying on paired synthetic data, existing learning-based Computational Aberration Correction (CAC) methods are confronted with the intricate and multifaceted synthetic-to-real domain gap, which leads to suboptimal performance in real-world applications. In this paper, in contrast to improving the simulation pipeline, we deliver a novel insight into real-world CAC from the perspective of Unsupervised Domain Adaptation (UDA). By incorporating readily accessible unpaired real-world data into training, we formalize the Domain Adaptive CAC (DACAC) task, and then introduce a comprehensive Real-world aberrated images (Realab) dataset to benchmark it. The setup task presents a formidable challenge due to the intricacy of understanding the target aberration domain. To this intent, we propose a novel Quntized Domain-Mixing Representation (QDMR) framework as a potent solution to the issue. QDMR adapts the CAC model to the target domain from three key aspects: (1) reconstructing aberrated images of both domains by a VQGAN to learn a Domain-Mixing Codebook (DMC) which characterizes the degradation-aware priors; (2) modulating the deep features in CAC model with DMC to transfer the target domain knowledge; and (3) leveraging the trained VQGAN to generate pseudo target aberrated images from the source ones for convincing target domain supervision. Extensive experiments on both synthetic and real-world benchmarks reveal that the models with QDMR consistently surpass the competitive methods in mitigating the synthetic-to-real gap, which produces visually pleasant real-world CAC results with fewer artifacts. Codes and datasets will be made publicly available.<|reference_end|>
@article{jiang2024representing,
    title={Representing Domain-Mixing Optical Degradation for Real-World
  Computational Aberration Correction via Vector Quantization},
    author={Qi Jiang, Zhonghua Yi, Shaohua Gao, Yao Gao, Xiaolong Qian, Hao Shi,
  Lei Sun, JinXing Niu, Kaiwei Wang, Kailun Yang, Jian Bai},
    journal={arXiv preprint arXiv:2403.10012},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.10012},
    primaryClass={cs.CV cs.RO eess.IV physics.optics}
}