ConceptPrune: Concept Editing in Diffusion Models via Skilled Neuron Pruning
<|reference_start|>ConceptPrune: Concept Editing in Diffusion Models via Skilled Neuron Pruning: While large-scale text-to-image diffusion models have demonstrated impressive image-generation capabilities, there are significant concerns about their potential misuse for generating unsafe content, violating copyright, and perpetuating societal biases. Recently, the text-to-image generation community has begun addressing these concerns by editing or unlearning undesired concepts from pre-trained models. However, these methods often involve data-intensive and inefficient fine-tuning or utilize various forms of token remapping, rendering them susceptible to adversarial jailbreaks. In this paper, we present a simple and effective training-free approach, ConceptPrune, wherein we first identify critical regions within pre-trained models responsible for generating undesirable concepts, thereby facilitating straightforward concept unlearning via weight pruning. Experiments across a range of concepts including artistic styles, nudity, object erasure, and gender debiasing demonstrate that target concepts can be efficiently erased by pruning a tiny fraction, approximately 0.12% of total weights, enabling multi-concept erasure and robustness against various white-box and black-box adversarial attacks.<|reference_end|>
@article{chavhan2024conceptprune:,
    title={ConceptPrune: Concept Editing in Diffusion Models via Skilled Neuron
  Pruning},
    author={Ruchika Chavhan and Da Li and Timothy Hospedales},
    journal={arXiv preprint arXiv:2405.19237},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.19237},
    primaryClass={cs.CV cs.AI cs.LG}
}