entropy: Estimating Entropy From Observed Counts

Description

entropy estimates the Shannon entropy H of the random variable Y from the corresponding observed counts y.

freqs estimates bin frequencies from the counts y.

Usage

entropy(y, lambda.freqs, method=c("ML", "MM", "Jeffreys", "Laplace", "SG",
    "minimax", "CS", "NSB", "shrink"), unit=c("log", "log2", "log10"), verbose=TRUE, ...)
freqs(y, lambda.freqs, method=c("ML", "MM", "Jeffreys", "Laplace", "SG",
    "minimax", "CS", "NSB", "shrink"), verbose=TRUE)

Value

entropy returns an estimate of the Shannon entropy.

freqs returns a vector with estimated bin frequencies (if available).

Arguments

y: vector of counts.
method: the method employed to estimate entropy (see Details).
unit: the unit in which entropy is measured. The default is "nats" (natural units). For computing entropy in "bits" set unit="log2".
lambda.freqs: shrinkage intensity (for "shrink" option).
verbose: verbose option (for "shrink" option).
...: option passed on to entropy.NSB.

Author

Korbinian Strimmer (https://strimmerlab.github.io).

Details

The entropy function allows to estimate entropy from observed counts by a variety of methods:

method="ML":: maximum likelihood, see entropy.empirical
method="MM":: bias-corrected maximum likelihood, see entropy.MillerMadow
method="Jeffreys":: entropy.Dirichlet with a=1/2
method="Laplace":: entropy.Dirichlet with a=1
method="SG":: entropy.Dirichlet with a=a=1/length(y)
method="minimax":: entropy.Dirichlet with a=sqrt(sum(y))/length(y
method="CS":: see entropy.ChaoShen
method="NSB":: see entropy.NSB
method="shrink":: see entropy.shrink

The freqs function estimates the underlying bin frequencies. Note that estimated frequencies are not available for method="MM", method="CS" and method="NSB". In these instances a vector containing NAs is returned.

Examples

Run this code

# load entropy library 
library("entropy")

# observed counts for each bin
y = c(4, 2, 3, 0, 2, 4, 0, 0, 2, 1, 1)  

entropy(y, method="ML")
entropy(y, method="MM")
entropy(y, method="Jeffreys")
entropy(y, method="Laplace")
entropy(y, method="SG")
entropy(y, method="minimax")
entropy(y, method="CS")
#entropy(y, method="NSB")
entropy(y, method="shrink")

Run the code above in your browser using DataLab

Get 50% off unlimited learning