  Restoring and inpainting the visual memories that are present, but often
impaired, in old photos remains an intriguing but unsolved research topic.
Decades-old photos often suffer from severe and commingled degradation such as
cracks, defocus, and color-fading, which are difficult to treat individually
and harder to repair when they interact. Deep learning presents a plausible
avenue, but the lack of large-scale datasets of old photos makes addressing
this restoration task very challenging. Here we present a novel reference-based
end-to-end learning framework that is able to both repair and colorize old,
degraded pictures. Our proposed framework consists of three modules: a
restoration sub-network that conducts restoration from degradations, a
similarity network that performs color histogram matching and color transfer,
and a colorization subnet that learns to predict the chroma elements of images
conditioned on chromatic reference signals. The overall system makes uses of
color histogram priors from reference images, which greatly reduces the need
for large-scale training data. We have also created a first-of-a-kind public
dataset of real old photos that are paired with ground truth ''pristine''
photos that have been manually restored by PhotoShop experts. We conducted
extensive experiments on this dataset and synthetic datasets, and found that
our method significantly outperforms previous state-of-the-art models using
both qualitative comparisons and quantitative measurements. The code is
available at https://github.com/DerrickXuNu/Pik-Fix.

Computer Vision and Pattern Recognition;Artificial Intelligence