VGAM (version 1.1-6)

KLD: Kullback-Leibler Divergence


Calculates the Kullback-Leibler divergence for certain fitted model objects


KLD(object, …)
 KLDvglm(object, …)



Some VGAM object, for example, having class vglm-class. Currently object must be intercept-only.

Other possible arguments fed into KLDvglm in order to compute the KLD.


Returns a numeric nonnegative value with the corresponding KLD. A 0 value means no difference between an ordinary parent or base distribution.


Numerical problems might occur if any of the evaluated probabilities of the unscaled parent distribution are very close to 0.


The Kullback-Leibler divergence (KLD), or relative entropy, is a measure of how one probability distribution differs from a second reference probability distribution. Currently the VGAM package computes the KLD for GAITD regression models (e.g., see gaitdpoisson and gaitdnbinomial) where the reference distribution is the (unscaled) parent or base distribution. For such, the formula for the KLD simplifies somewhat. Hence one can obtain a quantitative measure for the overall effect of altering, inflating, truncating and deflating certain (special) values.


Kullback, S. and Leibler, R. A. (1951). On information and sufficiency. Annals of Mathematical Statistics, 22, 79--86.

M'Kendrick, A. G. (1925). Applications of mathematics to medical problems. Proc. Edinb. Math. Soc., 44, 98--130.

See Also

gaitdpoisson, gaitdnbinomial.


Run this code
#  McKendrick (1925): Data from 223 Indian village households
cholera <- data.frame(ncases = 0:4,  # Number of cholera cases,
                      wfreq  = c(168, 32, 16, 6, 1))  # Frequencies
fit7 <- vglm(ncases ~ 1, gaitdpoisson(i.mlm = 0, ilambda.p = 1),
             weight = wfreq, data = cholera, trace = TRUE)
coef(fit7, matrix = TRUE)
# }

Run the code above in your browser using DataLab