Towards an Improved Metric for Evaluating Disentangled Representations
<|reference_start|>Towards an Improved Metric for Evaluating Disentangled Representations: Disentangled representation learning plays a pivotal role in making representations controllable, interpretable and transferable. Despite its significance in the domain, the quest for reliable and consistent quantitative disentanglement metric remains a major challenge. This stems from the utilisation of diverse metrics measuring different properties and the potential bias introduced by their design. Our work undertakes a comprehensive examination of existing popular disentanglement evaluation metrics, comparing them in terms of measuring aspects of disentanglement (viz. Modularity, Compactness, and Explicitness), detecting the factor-code relationship, and describing the degree of disentanglement. We propose a new framework for quantifying disentanglement, introducing a metric entitled \emph{EDI}, that leverages the intuitive concept of \emph{exclusivity} and improved factor-code relationship to minimize ad-hoc decisions. An in-depth analysis reveals that EDI measures essential properties while offering more stability than existing metrics, advocating for its adoption as a standardised approach.<|reference_end|>
@article{julka2024towards,
    title={Towards an Improved Metric for Evaluating Disentangled Representations},
    author={Sahib Julka, Yashu Wang, Michael Granitzer},
    journal={arXiv preprint arXiv:2410.03056},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.03056},
    primaryClass={cs.LG cs.AI}
}