RestoreFormer++: Towards Real-World Blind Face Restoration from
  Undegraded Key-Value Pairs
Blind face restoration aims at recovering high-quality face images from those
with unknown degradations. Current algorithms mainly introduce priors to
complement high-quality details and achieve impressive progress. However, most
of these algorithms ignore abundant contextual information in the face and its
interplay with the priors, leading to sub-optimal performance. Moreover, they
pay less attention to the gap between the synthetic and real-world scenarios,
limiting the robustness and generalization to real-world applications. In this
work, we propose RestoreFormer++, which on the one hand introduces
fully-spatial attention mechanisms to model the contextual information and the
interplay with the priors, and on the other hand, explores an extending
degrading model to help generate more realistic degraded face images to
alleviate the synthetic-to-real-world gap. Compared with current algorithms,
RestoreFormer++ has several crucial benefits. First, instead of using a
multi-head self-attention mechanism like the traditional visual transformer, we
introduce multi-head cross-attention over multi-scale features to fully explore
spatial interactions between corrupted information and high-quality priors. In
this way, it can facilitate RestoreFormer++ to restore face images with higher
realness and fidelity. Second, in contrast to the recognition-oriented
dictionary, we learn a reconstruction-oriented dictionary as priors, which
contains more diverse high-quality facial details and better accords with the
restoration target. Third, we introduce an extending degrading model that
contains more realistic degraded scenarios for training data synthesizing, and
thus helps to enhance the robustness and generalization of our RestoreFormer++
model. Extensive experiments show that RestoreFormer++ outperforms
state-of-the-art algorithms on both synthetic and real-world datasets.
64db2af13764a5137e713a77
Zhouxia Wang
claimed_verified
2023-08-16T13:22:29.544Z
64db92a5858f8a41c11669b7
/avatars/e834d8f1d4781e3bb0b5d6d25b3b3505.svg
Zhouxia Wang
user
wzhouxiff
64db2af13764a5137e713a78
Jiawei Zhang
64db2af13764a5137e713a79
Tianshui Chen
extracted_pending
2024-09-19T10:07:09.171Z
66ebf632a867b95ddb5b9414
/avatars/07a00952128d7fc031260d23e1923500.svg
Tianshui
user
TianshuiChen
64db2af13764a5137e713a7a
Wenping Wang
64db2af13764a5137e713a7b
Ping Luo
RestoreFormer++: Towards Real-World Blind Face Restoration from
  Undegraded Key-Value Pairs
Blind face restoration aims at recovering high-quality face images from those
with unknown degradations. Current algorithms mainly introduce priors to
complement high-quality details and achieve impressive progress. However, most
of these algorithms ignore abundant contextual information in the face and its
interplay with the priors, leading to sub-optimal performance. Moreover, they
pay less attention to the gap between the synthetic and real-world scenarios,
limiting the robustness and generalization to real-world applications. In this
work, we propose RestoreFormer++, which on the one hand introduces
fully-spatial attention mechanisms to model the contextual information and the
interplay with the priors, and on the other hand, explores an extending
degrading model to help generate more realistic degraded face images to
alleviate the synthetic-to-real-world gap. Compared with current algorithms,
RestoreFormer++ has several crucial benefits. First, instead of using a
multi-head self-attention mechanism like the traditional visual transformer, we
introduce multi-head cross-attention over multi-scale features to fully explore
spatial interactions between corrupted information and high-quality priors. In
this way, it can facilitate RestoreFormer++ to restore face images with higher
realness and fidelity. Second, in contrast to the recognition-oriented
dictionary, we learn a reconstruction-oriented dictionary as priors, which
contains more diverse high-quality facial details and better accords with the
restoration target. Third, we introduce an extending degrading model that
contains more realistic degraded scenarios for training data synthesizing, and
thus helps to enhance the robustness and generalization of our RestoreFormer++
model. Extensive experiments show that RestoreFormer++ outperforms
state-of-the-art algorithms on both synthetic and real-world datasets.
RestoreFormer++ improves blind face restoration by incorporating fully-spatial attention over multi-scale features and using a realistic degrading model to enhance realness, robustness, and generalization.
fully-spatial attention mechanisms multi-head cross-attention multi-scale features reconstruction-oriented dictionary extending degrading model high-quality face images blind face restoration synthetic-to-real-world gap