Understanding Matrix Function Normalizations in Covariance Pooling through the Lens of Riemannian Geometry
<|reference_start|>Understanding Matrix Function Normalizations in Covariance Pooling through the Lens of Riemannian Geometry: Global Covariance Pooling (GCP) has been demonstrated to improve the performance of Deep Neural Networks (DNNs) by exploiting second-order statistics of high-level representations. GCP typically performs classification of the covariance matrices by applying matrix function normalization, such as matrix logarithm or power, followed by a Euclidean classifier. However, covariance matrices inherently lie in a Riemannian manifold, known as the Symmetric Positive Definite (SPD) manifold. The current literature does not provide a satisfactory explanation of why Euclidean classifiers can be applied directly to Riemannian features after the normalization of the matrix power. To mitigate this gap, this paper provides a comprehensive and unified understanding of the matrix logarithm and power from a Riemannian geometry perspective. The underlying mechanism of matrix functions in GCP is interpreted from two perspectives: one based on tangent classifiers (Euclidean classifiers on the tangent space) and the other based on Riemannian classifiers. Via theoretical analysis and empirical validation through extensive experiments on fine-grained and large-scale visual classification datasets, we conclude that the working mechanism of the matrix functions should be attributed to the Riemannian classifiers they implicitly respect.<|reference_end|>
@article{chen2024understanding,
    title={Understanding Matrix Function Normalizations in Covariance Pooling
  through the Lens of Riemannian Geometry},
    author={Ziheng Chen, Yue Song, Xiao-Jun Wu, Gaowen Liu, Nicu Sebe},
    journal={arXiv preprint arXiv:2407.10484},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.10484},
    primaryClass={cs.CV cs.LG}
}