title: RestoreFormer++: Towards Real-World Blind Face Restoration from
    Undegraded Key-Value Pairs
summary: Blind face restoration aims at recovering high-quality face images from those
  with unknown degradations. Current algorithms mainly introduce priors to
  complement high-quality details and achieve impressive progress. However, most
  of these algorithms ignore abundant contextual information in the face and its
  interplay with the priors, leading to sub-optimal performance. Moreover, they
  pay less attention to the gap between the synthetic and real-world scenarios,
  limiting the robustness and generalization to real-world applications. In this
  work, we propose RestoreFormer++, which on the one hand introduces
  fully-spatial attention mechanisms to model the contextual information and the
  interplay with the priors, and on the other hand, explores an extending
  degrading model to help generate more realistic degraded face images to
  alleviate the synthetic-to-real-world gap. Compared with current algorithms,
  RestoreFormer++ has several crucial benefits. First, instead of using a
  multi-head self-attention mechanism like the traditional visual transformer, we
  introduce multi-head cross-attention over multi-scale features to fully explore
  spatial interactions between corrupted information and high-quality priors. In
  this way, it can facilitate RestoreFormer++ to restore face images with higher
  realness and fidelity. Second, in contrast to the recognition-oriented
  dictionary, we learn a reconstruction-oriented dictionary as priors, which
  contains more diverse high-quality facial details and better accords with the
  restoration target. Third, we introduce an extending degrading model that
  contains more realistic degraded scenarios for training data synthesizing, and
  thus helps to enhance the robustness and generalization of our RestoreFormer++
  model. Extensive experiments show that RestoreFormer++ outperforms
  state-of-the-art algorithms on both synthetic and real-world datasets.
paper_authors._id: 64db2af13764a5137e713a77
paper_authors.name: Zhouxia Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-08-16T13:22:29.544Z
paper_authors._id: 64db92a5858f8a41c11669b7
paper_authors.avatarUrl: /avatars/e834d8f1d4781e3bb0b5d6d25b3b3505.svg
paper_authors.fullname: Zhouxia Wang
paper_authors.type: user
paper_authors.user: wzhouxiff
paper_authors._id: 64db2af13764a5137e713a78
paper_authors.name: Jiawei Zhang
paper_authors._id: 64db2af13764a5137e713a79
paper_authors.name: Tianshui Chen
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-09-19T10:07:09.171Z
paper_authors._id: 66ebf632a867b95ddb5b9414
paper_authors.avatarUrl: /avatars/07a00952128d7fc031260d23e1923500.svg
paper_authors.fullname: Tianshui
paper_authors.type: user
paper_authors.user: TianshuiChen
paper_authors._id: 64db2af13764a5137e713a7a
paper_authors.name: Wenping Wang
paper_authors._id: 64db2af13764a5137e713a7b
paper_authors.name: Ping Luo
paper_title: RestoreFormer++: Towards Real-World Blind Face Restoration from
    Undegraded Key-Value Pairs
paper_summary: Blind face restoration aims at recovering high-quality face images from those
  with unknown degradations. Current algorithms mainly introduce priors to
  complement high-quality details and achieve impressive progress. However, most
  of these algorithms ignore abundant contextual information in the face and its
  interplay with the priors, leading to sub-optimal performance. Moreover, they
  pay less attention to the gap between the synthetic and real-world scenarios,
  limiting the robustness and generalization to real-world applications. In this
  work, we propose RestoreFormer++, which on the one hand introduces
  fully-spatial attention mechanisms to model the contextual information and the
  interplay with the priors, and on the other hand, explores an extending
  degrading model to help generate more realistic degraded face images to
  alleviate the synthetic-to-real-world gap. Compared with current algorithms,
  RestoreFormer++ has several crucial benefits. First, instead of using a
  multi-head self-attention mechanism like the traditional visual transformer, we
  introduce multi-head cross-attention over multi-scale features to fully explore
  spatial interactions between corrupted information and high-quality priors. In
  this way, it can facilitate RestoreFormer++ to restore face images with higher
  realness and fidelity. Second, in contrast to the recognition-oriented
  dictionary, we learn a reconstruction-oriented dictionary as priors, which
  contains more diverse high-quality facial details and better accords with the
  restoration target. Third, we introduce an extending degrading model that
  contains more realistic degraded scenarios for training data synthesizing, and
  thus helps to enhance the robustness and generalization of our RestoreFormer++
  model. Extensive experiments show that RestoreFormer++ outperforms
  state-of-the-art algorithms on both synthetic and real-world datasets.
paper_ai_summary: RestoreFormer++ improves blind face restoration by incorporating fully-spatial attention over multi-scale features and using a realistic degrading model to enhance realness, robustness, and generalization.
paper_ai_keywords: fully-spatial attention mechanisms multi-head cross-attention multi-scale features reconstruction-oriented dictionary extending degrading model high-quality face images blind face restoration synthetic-to-real-world gap