Rényi Entropy

Rényi entropy is defined as:

where , .

As ,  converges to , which is Shannon's measure of entropy.

Rényi's measure satisfies

for .

REFERENCES:

Karmeshu, J. (Ed.). Entropy Measures, Maximum Entropy Principle and Emerging Applications. New York: Springer-Verlag, 2003.

Rényi, A. "On Measures of Entropy and Information." Proc. Fourth Berkeley Symp. Math. Stat. and Probability, Vol. 1. Berkeley, CA: University of California Press, pp. 547-561, 1961.