biogram (version 1.4)

calc_ig: Calculate IG for single feature

Description

Computes information gain of single feature and target vector.

Usage

calc_ig(feature, target_b, len_target, pos_target)

Arguments

feature

feature vector.

target_b

target in bits (as per as.bit).

len_target

length of the target vector.

pos_target

number of positive cases in the target vector.

Value

A numeric vector of length 1 representing information gain in nats.

Details

The information gain term is used here (improperly) as a synonym of mutual information. It is defined as: $$IG(X; Y) = \sum_{y \in Y} \sum_{x \in X} p(x, y) \log \left(\frac{p(x, y)}{p(x) p(y)} \right)$$

In biogram package information gain is computed using following relationship: \(IG = E(S) - E(S|F)\)

References

Cover TM, Thomas JA Elements of Information Theory, 2nd Edition Wiley, 2006.

Examples

# NOT RUN {
tar <- sample(0L:1, 100, replace = TRUE)
feat <- sample(0L:1, 100, replace = TRUE)
library(bit) # used to code vector as bit
calc_ig(feat, as.bit(tar), 100, sum(tar))
# }