entropy (version 1.0.0)

entropy.empirical: Empirical entropy estimator

Description

entropy.empirical estimates the Shannon entropy H of the random variable Y from the corresponding observed counts y by plug-in of the empirical frequencies.

freqs.empirical computes the empirical frequencies from counts y.

Usage

entropy.empirical(y, unit=c("log", "log2", "log10"))
freqs.empirical(y)

Arguments

y

vector of counts.

unit

the unit in which entropy is measured.

Value

• entropy.empirical returns an estimate of the Shannon entropy.

  freqs.empirical returns the underlying frequencies.

Details

The empirical entropy estimator is a plug-in estimator: in the definition of the Shannon entropy the bin probabilities are replaced by the respective empirical frequencies.

The empirical entropy estimator is the maximum likelihood estimator. If there are many zero counts and the sample size is small it is very inefficient and also strongly biased.

See Also

entropy.MillerMadow, entropy.plugin.

Examples

# load entropy library 
library("entropy")

# observed counts for each bin
y = c(4, 2, 3, 0, 2, 4, 0, 0, 2, 1, 1)  

# empirical estimate of entropy
entropy.empirical(y)