Entropy Rate Estimation for Markov Chains with Large State Space
Entropy estimation is one of the prototypical problems in distribution property testing. To consistently estimate the Shannon entropy of a distribution on S elements with independent samples, the optimal sample complexity scales sublinearly with S as Θ( S log S ) as shown by Valiant and Valiant [4]. Extending the theory and algorithms for entropy estimation to dependent data, this paper considers the problem of estimating the entropy rate of a stationary reversible Markov chain with S states from a sample path of n observations. We show that
• Provided the Markov chain mixes not too slowly, i.e., the relaxation time is at most O( S ln 3 S
• Provided the Markov chain mixes not too slowly, i.e., the relaxation time is at most O( S ln 3 S ), consistent estimation is achievable when n S 2 log S .
• Provided the Markov chain has some slight dependency, i.e., the relaxation time is at least 1 + Ω(
), consistent estimation is impossible when n S 2 log S .
Under both assumptions, the optimal estimation accuracy is shown to be Θ(
n log S ). In comparison, the empirical entropy rate requires at least Ω(S 2 ) samples to be consistent, even when the Markov chain is memoryless. In addition to synthetic experiments, we also apply the estimators that achieve the optimal sample complex-ity to estimate the entropy rate of the English language in the Penn Treebank and the Google One Billion Words corpora, which provides a natural benchmark for language modeling and relates it directly to the widely used perplexity measure.
The paper proposes an entropy estimate for Markov chains by reduction to optimal entropy estimation for i.i.d samples. Sample complexity analysis is provided for different mixing scenarios with a minimax rate established for a particular rate. The estimator is used to assess the capacity of language models.    This is a very clear and well-written paper. I appreciate the efforts done by the authors to summarize the results.  My main question, however, is regarding the experiments. Can the authors comment on the confidence of the entropy estimate given the large size of the alphabet? This is crucial to justify its use for the suggested application. Also, could the authors include the empirical entropy estimates?  ============================================================= UPDATE: I thank the authors for the their response. I believe the empirical entropy results setrengthen the paper and I recommend including them in the final version.