The standard upper bound on discrete entropy was derived based on the differential entropy bound for continuous random variables. A tighter discrete entropy bound is derived using the transformation formula of Jacobi theta function. The new bound is applicable only when the probability mass function of the discrete random variable satisfies certain conditions. Its application to the class of binomial random variables is presented as an example.
['Wai Ho Mow']
A tight upper bound on discrete entropy