title: ColorFlow: Retrieval-Augmented Image Sequence Colorization
summary: Automatic black-and-white image sequence colorization while preserving
  character and object identity (ID) is a complex task with significant market
  demand, such as in cartoon or comic series colorization. Despite advancements
  in visual colorization using large-scale generative models like diffusion
  models, challenges with controllability and identity consistency persist,
  making current solutions unsuitable for industrial application.To address this,
  we propose ColorFlow, a three-stage diffusion-based framework tailored for
  image sequence colorization in industrial applications. Unlike existing methods
  that require per-ID finetuning or explicit ID embedding extraction, we propose
  a novel robust and generalizable Retrieval Augmented Colorization pipeline for
  colorizing images with relevant color references. Our pipeline also features a
  dual-branch design: one branch for color identity extraction and the other for
  colorization, leveraging the strengths of diffusion models. We utilize the
  self-attention mechanism in diffusion models for strong in-context learning and
  color identity matching. To evaluate our model, we introduce ColorFlow-Bench, a
  comprehensive benchmark for reference-based colorization. Results show that
  ColorFlow outperforms existing models across multiple metrics, setting a new
  standard in sequential image colorization and potentially benefiting the art
  industry. We release our codes and models on our project page:
  https://zhuang2002.github.io/ColorFlow/.
paper_authors._id: 6760ea776fc1fd2b8ed9c802
paper_authors.name: Junhao Zhuang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-17T08:03:56.429Z
paper_authors._id: 64970d3d9c3b29dca8633f87
paper_authors.avatarUrl: /avatars/11e3c9c66d28490d6d09925f9aa47cd1.svg
paper_authors.fullname: JunhaoZhuang
paper_authors.type: user
paper_authors.user: JunhaoZhuang
paper_authors._id: 6760ea776fc1fd2b8ed9c803
paper_authors.name: Xuan Ju
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-17T09:25:47.793Z
paper_authors._id: 62d4577bc85b0fcf7fde39bb
paper_authors.avatarUrl: /avatars/a3a5729e33ae89ce9ba408830db3c835.svg
paper_authors.fullname: Xuan Ju
paper_authors.type: user
paper_authors.user: juxuan27
paper_authors._id: 6760ea776fc1fd2b8ed9c804
paper_authors.name: Zhaoyang Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-17T08:03:54.477Z
paper_authors._id: 658409ceca19ccf6d9989add
paper_authors.avatarUrl: /avatars/3ac1dbd25e52435185babdeb3da28875.svg
paper_authors.fullname: Zhaoyang Zhang
paper_authors.type: user
paper_authors.user: ZyZcuhk
paper_authors._id: 6760ea776fc1fd2b8ed9c805
paper_authors.name: Yong Liu
paper_authors._id: 6760ea776fc1fd2b8ed9c806
paper_authors.name: Shiyi Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-09T08:55:21.766Z
paper_authors._id: 6315d306a9456afe2b9bf34a
paper_authors.avatarUrl: /avatars/7285b4e7d84b528d1a50f8ee4eb10727.svg
paper_authors.fullname: ElevenZ
paper_authors.type: user
paper_authors.user: shiyi0408
paper_authors._id: 6760ea776fc1fd2b8ed9c807
paper_authors.name: Chun Yuan
paper_authors._id: 6760ea776fc1fd2b8ed9c808
paper_authors.name: Ying Shan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-17T09:25:16.323Z
paper_authors._id: 63ca3ddc04c979828310bfcb
paper_authors.avatarUrl: /avatars/615e0d8622950b4408b40d550f02a894.svg
paper_authors.fullname: Ying Shan
paper_authors.type: user
paper_authors.user: yshan2u
paper_title: ColorFlow: Retrieval-Augmented Image Sequence Colorization
paper_summary: Automatic black-and-white image sequence colorization while preserving
  character and object identity (ID) is a complex task with significant market
  demand, such as in cartoon or comic series colorization. Despite advancements
  in visual colorization using large-scale generative models like diffusion
  models, challenges with controllability and identity consistency persist,
  making current solutions unsuitable for industrial application.To address this,
  we propose ColorFlow, a three-stage diffusion-based framework tailored for
  image sequence colorization in industrial applications. Unlike existing methods
  that require per-ID finetuning or explicit ID embedding extraction, we propose
  a novel robust and generalizable Retrieval Augmented Colorization pipeline for
  colorizing images with relevant color references. Our pipeline also features a
  dual-branch design: one branch for color identity extraction and the other for
  colorization, leveraging the strengths of diffusion models. We utilize the
  self-attention mechanism in diffusion models for strong in-context learning and
  color identity matching. To evaluate our model, we introduce ColorFlow-Bench, a
  comprehensive benchmark for reference-based colorization. Results show that
  ColorFlow outperforms existing models across multiple metrics, setting a new
  standard in sequential image colorization and potentially benefiting the art
  industry. We release our codes and models on our project page:
  https://zhuang2002.github.io/ColorFlow/.
paper_ai_summary: ColorFlow, a three-stage diffusion-based framework with a dual-branch design, achieves superior image sequence colorization by incorporating retrieval augmented colorization and self-attention mechanisms, outperforming existing models.
paper_ai_keywords: diffusion-based framework retrieval augmented colorization dual-branch design self-attention mechanism reference-based colorization