Maximum Manifold Capacity Representations in State Representation Learning
<|reference_start|>Maximum Manifold Capacity Representations in State Representation Learning: The expanding research on manifold-based self-supervised learning (SSL) builds on the manifold hypothesis, which suggests that the inherent complexity of high-dimensional data can be unraveled through lower-dimensional manifold embeddings. Capitalizing on this, DeepInfomax with an unbalanced atlas (DIM-UA) has emerged as a powerful tool and yielded impressive results for state representations in reinforcement learning. Meanwhile, Maximum Manifold Capacity Representation (MMCR) presents a new frontier for SSL by optimizing class separability via manifold compression. However, MMCR demands extensive input views, resulting in significant computational costs and protracted pre-training durations. Bridging this gap, we present an innovative integration of MMCR into existing SSL methods, incorporating a discerning regularization strategy that enhances the lower bound of mutual information. We also propose a novel state representation learning method extending DIM-UA, embedding a nuclear norm loss to enforce manifold consistency robustly. On experimentation with the Atari Annotated RAM Interface, our method improves DIM-UA significantly with the same number of target encoding dimensions. The mean F1 score averaged over categories is 78% compared to 75% of DIM-UA. There are also compelling gains when implementing SimCLR and Barlow Twins. This supports our SSL innovation as a paradigm shift, enabling more nuanced high-dimensional data representations.<|reference_end|>
@article{meng2024maximum,
    title={Maximum Manifold Capacity Representations in State Representation
  Learning},
    author={Li Meng, Morten Goodwin, Anis Yazidi, Paal Engelstad},
    journal={arXiv preprint arXiv:2405.13848},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.13848},
    primaryClass={cs.LG}
}