InstantDrag: Improving Interactivity in Drag-based Image Editing
Drag-based image editing has recently gained popularity for its interactivity
and precision. However, despite the ability of text-to-image models to generate
samples within a second, drag editing still lags behind due to the challenge of
accurately reflecting user interaction while maintaining image content. Some
existing approaches rely on computationally intensive per-image optimization or
intricate guidance-based methods, requiring additional inputs such as masks for
movable regions and text prompts, thereby compromising the interactivity of the
editing process. We introduce InstantDrag, an optimization-free pipeline that
enhances interactivity and speed, requiring only an image and a drag
instruction as input. InstantDrag consists of two carefully designed networks:
a drag-conditioned optical flow generator (FlowGen) and an optical
flow-conditioned diffusion model (FlowDiffusion). InstantDrag learns motion
dynamics for drag-based image editing in real-world video datasets by
decomposing the task into motion generation and motion-conditioned image
generation. We demonstrate InstantDrag's capability to perform fast,
photo-realistic edits without masks or text prompts through experiments on
facial video datasets and general scenes. These results highlight the
efficiency of our approach in handling drag-based image editing, making it a
promising solution for interactive, real-time applications.
66e836da9ebf20e0639f3541
Joonghyuk Shin
claimed_verified
2024-09-17T07:11:47.682Z
631074d895c34b95407945f0
/avatars/699baf06ec818650dec5752aca87c5b4.svg
Joonghyuk Shin
user
alex4727
66e836da9ebf20e0639f3542
Daehyeon Choi
claimed_verified
2025-01-15T08:50:46.969Z
6616702547ea6347974667e5
/avatars/6bf95e50ba19df163ef89867ed63fecc.svg
Daehyeon Choi
user
daehyeonchoi
66e836da9ebf20e0639f3543
Jaesik Park
InstantDrag: Improving Interactivity in Drag-based Image Editing
Drag-based image editing has recently gained popularity for its interactivity
and precision. However, despite the ability of text-to-image models to generate
samples within a second, drag editing still lags behind due to the challenge of
accurately reflecting user interaction while maintaining image content. Some
existing approaches rely on computationally intensive per-image optimization or
intricate guidance-based methods, requiring additional inputs such as masks for
movable regions and text prompts, thereby compromising the interactivity of the
editing process. We introduce InstantDrag, an optimization-free pipeline that
enhances interactivity and speed, requiring only an image and a drag
instruction as input. InstantDrag consists of two carefully designed networks:
a drag-conditioned optical flow generator (FlowGen) and an optical
flow-conditioned diffusion model (FlowDiffusion). InstantDrag learns motion
dynamics for drag-based image editing in real-world video datasets by
decomposing the task into motion generation and motion-conditioned image
generation. We demonstrate InstantDrag's capability to perform fast,
photo-realistic edits without masks or text prompts through experiments on
facial video datasets and general scenes. These results highlight the
efficiency of our approach in handling drag-based image editing, making it a
promising solution for interactive, real-time applications.
InstantDrag, an optimization-free pipeline with two networks, performs fast, photo-realistic drag-based image editing without requiring masks or text prompts.
drag-conditioned optical flow optical flow-conditioned diffusion model motion generation motion-conditioned image generation