What is the $\textitintrinsic$ dimension of your binary data? -- and how to compute it quickly
<|reference_start|>What is the $\textitintrinsic$ dimension of your binary data? -- and how to compute it quickly: Dimensionality is an important aspect for analyzing and understanding (high-dimensional) data. In their 2006 ICDM paper Tatti et al. answered the question for a (interpretable) dimension of binary data tables by introducing a normalized correlation dimension. In the present work we revisit their results and contrast them with a concept based notion of intrinsic dimension (ID) recently introduced for geometric data sets. To do this, we present a novel approximation for this ID that is based on computing concepts only up to a certain support value. We demonstrate and evaluate our approximation using all available datasets from Tatti et al., which have between 469 and 41271 extrinsic dimensions.<|reference_end|>
@article{hanika2024what,
    title={What is the $\textit{intrinsic}$ dimension of your binary data? -- and
  how to compute it quickly},
    author={Tom Hanika and Tobias Hille},
    journal={arXiv preprint arXiv:2404.06326},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.06326},
    primaryClass={cs.LG cs.AI}
}