Click2Mask: Local Editing with Dynamic Mask Generation
Recent advancements in generative models have revolutionized image generation
and editing, making these tasks accessible to non-experts. This paper focuses
on local image editing, particularly the task of adding new content to a
loosely specified area. Existing methods often require a precise mask or a
detailed description of the location, which can be cumbersome and prone to
errors. We propose Click2Mask, a novel approach that simplifies the local
editing process by requiring only a single point of reference (in addition to
the content description). A mask is dynamically grown around this point during
a Blended Latent Diffusion (BLD) process, guided by a masked CLIP-based
semantic loss. Click2Mask surpasses the limitations of segmentation-based and
fine-tuning dependent methods, offering a more user-friendly and contextually
accurate solution. Our experiments demonstrate that Click2Mask not only
minimizes user effort but also delivers competitive or superior local image
manipulation results compared to SoTA methods, according to both human
judgement and automatic metrics. Key contributions include the simplification
of user input, the ability to freely add objects unconstrained by existing
segments, and the integration potential of our dynamic mask approach within
other editing methods.
66e83376a3f5a4cad5ebde9b
Omer Regev
admin_assigned
2024-09-17T07:57:06.688Z
632add454f9006565ae542be
/avatars/0937fd1690052f854a5fb6f80cfca696.svg
Omer Regev
user
omeregev
66e83376a3f5a4cad5ebde9c
Omri Avrahami
claimed_verified
2024-09-17T07:12:03.303Z
62f6ab4fc3372328414c8689
/avatars/4f728a5b70c9fe4a64e80e2b643ca620.svg
Omri Avrahami
user
omriav
66e83376a3f5a4cad5ebde9d
Dani Lischinski
admin_assigned
2024-09-17T07:57:11.953Z
644cc8a81e0cef1987125d22
/avatars/a728cefda3b95c39a3c9aa6485a11c60.svg
Dani Lischinski
user
danix3d
Click2Mask: Local Editing with Dynamic Mask Generation
Recent advancements in generative models have revolutionized image generation
and editing, making these tasks accessible to non-experts. This paper focuses
on local image editing, particularly the task of adding new content to a
loosely specified area. Existing methods often require a precise mask or a
detailed description of the location, which can be cumbersome and prone to
errors. We propose Click2Mask, a novel approach that simplifies the local
editing process by requiring only a single point of reference (in addition to
the content description). A mask is dynamically grown around this point during
a Blended Latent Diffusion (BLD) process, guided by a masked CLIP-based
semantic loss. Click2Mask surpasses the limitations of segmentation-based and
fine-tuning dependent methods, offering a more user-friendly and contextually
accurate solution. Our experiments demonstrate that Click2Mask not only
minimizes user effort but also delivers competitive or superior local image
manipulation results compared to SoTA methods, according to both human
judgement and automatic metrics. Key contributions include the simplification
of user input, the ability to freely add objects unconstrained by existing
segments, and the integration potential of our dynamic mask approach within
other editing methods.
Click2Mask simplifies local image editing by using a single point of reference and dynamic mask generation within a Blended Latent Diffusion (BLD) process, guided by CLIP-based semantic loss.
Blended Latent Diffusion BLD CLIP-based semantic loss dynamic mask generation