rinform (version 1.0.2)

shannon_relative_entropy: Shannon Relative Entropy

Description

Compute the base-b Shannon relative entropy between posterior p and prior q distributions.

Usage

shannon_relative_entropy(p, q, b = 2)

Arguments

p

Dist specifying the posterior distribution.

q

Dist specifying the prior distribution.

b

Numeric giving the base of the logarithm.

Value

Numeric giving the Shannon relative entropy.

Examples

# NOT RUN {
p <- Dist(c(4, 1))
q <- Dist(c(1, 1))
shannon_relative_entropy(p, q)          # 0.2780719
shannon_relative_entropy(q, p)          # 0.3219281
shannon_relative_entropy(p, q, b = 3)   # 0.1754438
shannon_relative_entropy(q, p, b = 3)   # 0.2031140

p <- Dist(c(1, 0))
q <- Dist(c(1, 1))
shannon_relative_entropy(p, q)          # 1.0
shannon_relative_entropy(q, p)          # NaN
shannon_relative_entropy(p, q, b = 3)   # 0.6309298
shannon_relative_entropy(q, p, b = 3)   # NaN

# }