A Probabilistic Upper Bound on Differential Entropy
<|reference_start|>A Probabilistic Upper Bound on Differential Entropy: A novel, non-trivial, probabilistic upper bound on the entropy of an unknown one-dimensional distribution, given the support of the distribution and a sample from that distribution, is presented. No knowledge beyond the support of the unknown distribution is required, nor is the distribution required to have a density. Previous distribution-free bounds on the cumulative distribution function of a random variable given a sample of that variable are used to construct the bound. A simple, fast, and intuitive algorithm for computing the entropy bound from a sample is provided.<|reference_end|>
@article{destefano2005a,
    title={A Probabilistic Upper Bound on Differential Entropy},
    author={Joseph DeStefano and Erik Learned-Miller},
    journal={arXiv preprint arXiv:cs/0504091},
    year={2005},
    archivePrefix={arXiv},
    eprint={cs/0504091},
    primaryClass={cs.IT math.IT}
}