Composite Concept Extraction through Backdooring
<|reference_start|>Composite Concept Extraction through Backdooring: Learning composite concepts, such as \textquotedbl red car\textquotedbl , from individual examples -- like a white car representing the concept of \textquotedbl car\textquotedbl{} and a red strawberry representing the concept of \textquotedbl red\textquotedbl -- is inherently challenging. This paper introduces a novel method called Composite Concept Extractor (CoCE), which leverages techniques from traditional backdoor attacks to learn these composite concepts in a zero-shot setting, requiring only examples of individual concepts. By repurposing the trigger-based model backdooring mechanism, we create a strategic distortion in the manifold of the target object (e.g., \textquotedbl car\textquotedbl ) induced by example objects with the target property (e.g., \textquotedbl red\textquotedbl ) from objects \textquotedbl red strawberry\textquotedbl , ensuring the distortion selectively affects the target objects with the target property. Contrastive learning is then employed to further refine this distortion, and a method is formulated for detecting objects that are influenced by the distortion. Extensive experiments with in-depth analysis across different datasets demonstrate the utility and applicability of our proposed approach.<|reference_end|>
@article{ghosh2024composite,
    title={Composite Concept Extraction through Backdooring},
    author={Banibrata Ghosh, Haripriya Harikumar, Khoa D Doan, Svetha Venkatesh,
  Santu Rana},
    journal={arXiv preprint arXiv:2406.13411},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.13411},
    primaryClass={cs.CV cs.LG}
}