In-Context Edit: Enabling Instructional Image Editing with In-Context
  Generation in Large Scale Diffusion Transformer
Instruction-based image editing enables robust image modification via natural
language prompts, yet current methods face a precision-efficiency tradeoff.
Fine-tuning methods demand significant computational resources and large
datasets, while training-free techniques struggle with instruction
comprehension and edit quality. We resolve this dilemma by leveraging
large-scale Diffusion Transformer (DiT)' enhanced generation capacity and
native contextual awareness. Our solution introduces three contributions: (1)
an in-context editing framework for zero-shot instruction compliance using
in-context prompting, avoiding structural changes; (2) a LoRA-MoE hybrid tuning
strategy that enhances flexibility with efficient adaptation and dynamic expert
routing, without extensive retraining; and (3) an early filter inference-time
scaling method using vision-language models (VLMs) to select better initial
noise early, improving edit quality. Extensive evaluations demonstrate our
method's superiority: it outperforms state-of-the-art approaches while
requiring only 0.5% training data and 1% trainable parameters compared to
conventional baselines. This work establishes a new paradigm that enables
high-precision yet efficient instruction-guided editing. Codes and demos can be
found in https://river-zhang.github.io/ICEdit-gh-pages/.
6811c39584adfa26b82a4370
Zechuan Zhang
6811c39584adfa26b82a4371
Ji Xie
claimed_verified
2025-05-02T06:36:45.182Z
64e99fc07e2ec711a7138262
https://cdn-avatars.huggingface.co/v1/production/uploads/64e99fc07e2ec711a7138262/FmP3F8_UXgh9K-0gwS99A.jpeg
Ji Xie
user
sanaka87
6811c39584adfa26b82a4372
Yu Lu
6811c39584adfa26b82a4373
Zongxin Yang
6811c39584adfa26b82a4374
Yi Yang
In-Context Edit: Enabling Instructional Image Editing with In-Context
  Generation in Large Scale Diffusion Transformer
Instruction-based image editing enables robust image modification via natural
language prompts, yet current methods face a precision-efficiency tradeoff.
Fine-tuning methods demand significant computational resources and large
datasets, while training-free techniques struggle with instruction
comprehension and edit quality. We resolve this dilemma by leveraging
large-scale Diffusion Transformer (DiT)' enhanced generation capacity and
native contextual awareness. Our solution introduces three contributions: (1)
an in-context editing framework for zero-shot instruction compliance using
in-context prompting, avoiding structural changes; (2) a LoRA-MoE hybrid tuning
strategy that enhances flexibility with efficient adaptation and dynamic expert
routing, without extensive retraining; and (3) an early filter inference-time
scaling method using vision-language models (VLMs) to select better initial
noise early, improving edit quality. Extensive evaluations demonstrate our
method's superiority: it outperforms state-of-the-art approaches while
requiring only 0.5% training data and 1% trainable parameters compared to
conventional baselines. This work establishes a new paradigm that enables
high-precision yet efficient instruction-guided editing. Codes and demos can be
found in https://river-zhang.github.io/ICEdit-gh-pages/.
A novel in-context editing framework using a Diffusion Transformer enhances instruction-following precision and efficiency without extensive retraining or large datasets.
instruction-based image editing natural language prompts precision-efficiency tradeoff fine-tuning training-free techniques Diffusion Transformer (DiT) in-context editing framework LoRA-MoE hybrid tuning vision-language models (VLMs) early filter inference-time scaling