Nearly Zero-Cost Protection Against Mimicry by Personalized Diffusion
  Models
Recent advancements in diffusion models revolutionize image generation but
pose risks of misuse, such as replicating artworks or generating deepfakes.
Existing image protection methods, though effective, struggle to balance
protection efficacy, invisibility, and latency, thus limiting practical use. We
introduce perturbation pre-training to reduce latency and propose a
mixture-of-perturbations approach that dynamically adapts to input images to
minimize performance degradation. Our novel training strategy computes
protection loss across multiple VAE feature spaces, while adaptive targeted
protection at inference enhances robustness and invisibility. Experiments show
comparable protection performance with improved invisibility and drastically
reduced inference time. The code and demo are available at
https://webtoon.github.io/impasto
67621bbacf56ce09ea4756d5
Namhyuk Ahn
admin_assigned
2024-12-18T13:36:57.224Z
63686c355bb06007ea110ec9
/avatars/902f4625b976bc5f734d6821402e9026.svg
Namhyuk Ahn
user
nmhkahn
67621bbacf56ce09ea4756d6
KiYoon Yoo
admin_assigned
2024-12-18T13:36:51.276Z
62e88331dd52172c3896181c
/avatars/c892d351437f680596f0ccfa63af2ebe.svg
KiYoon Yoo
user
kiyoonyoo
67621bbacf56ce09ea4756d7
Wonhyuk Ahn
admin_assigned
2024-12-18T13:36:45.473Z
64c35318aeb4eae54dc7973e
/avatars/51352a300f432e72252d3551839c6b7f.svg
Wonhyuk Ahn
user
whahn
67621bbacf56ce09ea4756d8
Daesik Kim
admin_assigned
2024-12-18T13:36:39.657Z
638fe9c639f7e2a7f9d2b868
/avatars/7e67f59202ae66d054e3cd39526eb331.svg
Daesik Kim
user
daesik82
67621bbacf56ce09ea4756d9
Seung-Hun Nam
admin_assigned
2024-12-18T13:36:26.120Z
666a4f884649c647cf84aecd
/avatars/cb6b8622fc38b370a33c6b68410c87e7.svg
Seung-Hun Nam
user
shnam
Nearly Zero-Cost Protection Against Mimicry by Personalized Diffusion
  Models
Recent advancements in diffusion models revolutionize image generation but
pose risks of misuse, such as replicating artworks or generating deepfakes.
Existing image protection methods, though effective, struggle to balance
protection efficacy, invisibility, and latency, thus limiting practical use. We
introduce perturbation pre-training to reduce latency and propose a
mixture-of-perturbations approach that dynamically adapts to input images to
minimize performance degradation. Our novel training strategy computes
protection loss across multiple VAE feature spaces, while adaptive targeted
protection at inference enhances robustness and invisibility. Experiments show
comparable protection performance with improved invisibility and drastically
reduced inference time. The code and demo are available at
https://webtoon.github.io/impasto
Perturbation pre-training and a mixture-of-perturbations approach improve image protection by reducing latency, enhancing invisibility, and maintaining performance.
diffusion models image protection perturbation pre-training mixture-of-perturbations VAE feature spaces adaptive targeted protection