Correlation Dimension of Natural Language in a Statistical Manifold
<|reference_start|>Correlation Dimension of Natural Language in a Statistical Manifold: The correlation dimension of natural language is measured by applying the Grassberger-Procaccia algorithm to high-dimensional sequences produced by a large-scale language model. This method, previously studied only in a Euclidean space, is reformulated in a statistical manifold via the Fisher-Rao distance. Language exhibits a multifractal, with global self-similarity and a universal dimension around 6.5, which is smaller than those of simple discrete random sequences and larger than that of a Barab\'asi-Albert process. Long memory is the key to producing self-similarity. Our method is applicable to any probabilistic model of real-world discrete sequences, and we show an application to music data.<|reference_end|>
@article{du2024correlation,
    title={Correlation Dimension of Natural Language in a Statistical Manifold},
    author={Xin Du, Kumiko Tanaka-Ishii},
    journal={Physical Review Research, 6(2), L022028 (2024)},
    year={2024},
    doi={10.1103/PhysRevResearch.6.L022028},
    archivePrefix={arXiv},
    eprint={2405.06321},
    primaryClass={cs.CL cond-mat.stat-mech cs.AI}
}