Conceptual Learning via Embedding Approximations for Reinforcing Interpretability and Transparency
<|reference_start|>Conceptual Learning via Embedding Approximations for Reinforcing Interpretability and Transparency: Concept bottleneck models (CBMs) have emerged as critical tools in domains where interpretability is paramount. These models rely on predefined textual descriptions, referred to as concepts, to inform their decision-making process and offer more accurate reasoning. As a result, the selection of concepts used in the model is of utmost significance. This study proposes \underline{\textbf{C}}onceptual \underline{\textbf{L}}earning via \underline{\textbf{E}}mbedding \underline{\textbf{A}}pproximations for \underline{\textbf{R}}einforcing Interpretability and Transparency, abbreviated as CLEAR, a framework for constructing a CBM for image classification. Using score matching and Langevin sampling, we approximate the embedding of concepts within the latent space of a vision-language model (VLM) by learning the scores associated with the joint distribution of images and concepts. A concept selection process is then employed to optimize the similarity between the learned embeddings and the predefined ones. The derived bottleneck offers insights into the CBM's decision-making process, enabling more comprehensive interpretations. Our approach was evaluated through extensive experiments and achieved state-of-the-art performance on various benchmarks. The code for our experiments is available at https://github.com/clearProject/CLEAR/tree/main<|reference_end|>
@article{dikter2024conceptual,
    title={Conceptual Learning via Embedding Approximations for Reinforcing
  Interpretability and Transparency},
    author={Maor Dikter, Tsachi Blau, Chaim Baskin},
    journal={arXiv preprint arXiv:2406.08840},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.08840},
    primaryClass={cs.CV cs.LG}
}