Chentsov's theorem
In information geometry, Chentsov's theorem states that the Fisher information metric is, up to rescaling, the unique Riemannian metric on a statistical manifold that is invariant under sufficient statistics. See also 

Fisher information
Sufficient statistic
Information geometry

References 
 N. N. Čencov (1981), Statistical Decision Rules and Optimal Inference, Translations of mathematical monographs; v. 53, American Mathematical Society, http://www.ams.org/books/mmono/053/
 Shun'ichi Amari, Hiroshi Nagaoka (2000) Methods of information geometry, Translations of mathematical monographs; v. 191, American Mathematical Society, http://www.ams.org/books/mmono/191/ (Theorem 2.6)
 
 

Differential geometry
Information geometry
Statistical distance