riskyr (version 0.1.0)

comp_accu: Compute acccuracy of current classification results.

Description

comp_accu computes current accuracy metrics from the 4 essential frequencies (hi, mi, fa, cr) that constitute the current confusion matrix and are contained in freq.

Usage

comp_accu(hi = freq$hi, mi = freq$mi, fa = freq$fa, cr = freq$cr,
  w = 0.5)

Arguments

hi

The number of hits hi (or true positives).

mi

The number of misses mi (or false negatives).

fa

The number of false alarms fa (or false positives).

cr

The number of correct rejections cr (or true negatives).

w

The weighting parameter w (from 0 to 1) for computing weighted accuracy wacc. Default: w = .50 (i.e., yielding balanced accuracy bacc).

Value

A list accu containing current accuracy metrics.

Details

Currently computed metrics include:

1. acc: Overall accuracy as the proportion (or probability) of correctly classifying cases or of dec.cor cases:

   acc = dec.cor/N = (hi + cr)/(hi + mi + fa + cr)

   Values range from 0 (no correct prediction) to 1 (perfect prediction).

2. wacc: Weighted accuracy, as a weighted average of the sensitivity sens (aka. hit rate HR, TPR, power or recall) and the the specificity spec (aka. TNR) in which sens is multiplied by a weighting parameter w (ranging from 0 to 1) and spec is multiplied by w's complement (1 - w):

   wacc = (w * sens) + ((1 - w) * spec)

   If w = .50, wacc becomes balanced accuracy bacc.

3. mcc: The Matthews correlation coefficient (with values ranging from -1 to +1):

   mcc = ((hi * cr) - (fa * mi)) / sqrt((hi + fa) * (hi + mi) * (cr + fa) * (cr + mi))

   A value of mcc = 0 implies random performance; mcc = 1 implies perfect performance.

   See Wikipedia: Matthews correlation coefficient for additional information.

4. f1s: The harmonic mean of the positive predictive value PPV (aka. precision) and the sensitivity sens (aka. hit rate HR, TPR, power or recall):

   f1s = 2 * (PPV * sens) / (PPV + sens)

   See Wikipedia: F1 score for additional information.

Note that some accuracy metrics can be interpreted as probabilities (e.g., acc) or correlations (e.g., mcc).

References

Consult Wikipedia: Confusion matrix for additional information.

See Also

The corresponding data frame ; num for basic numeric parameters; freq for current frequency information; txt for current text settings; pal for current color settings; popu for a table of the current population.

Other metrics: accu, comp_acc

Other functions computing probabilities: comp_FDR, comp_FOR, comp_NPV, comp_PPV, comp_acc, comp_comp_pair, comp_complement, comp_complete_prob_set, comp_fart, comp_mirt, comp_ppod, comp_prob_freq, comp_prob, comp_sens, comp_spec

Examples

# NOT RUN {
comp_accu()  # => computes accuracy metrics for current default scenario
comp_accu(hi = 1, mi = 2, fa = 3, cr = 4)  # medium accuracy, but cr > hi

# Extreme cases:
comp_accu(hi = 1, mi = 1, fa = 1, cr = 1)  # random performance
comp_accu(hi = 1, mi = 0, fa = 0, cr = 1)  # perfect accuracy/optimal performance
comp_accu(hi = 0, mi = 1, fa = 1, cr = 0)  # zero accuracy/worst performance, but see f1s
comp_accu(hi = 1, mi = 0, fa = 0, cr = 0)  # perfect accuracy, but see wacc and mcc

# Effects of w:
comp_accu(hi = 3, mi = 2, fa = 1, cr = 4, w = 1/2)  # equal weights to sens and spec
comp_accu(hi = 3, mi = 2, fa = 1, cr = 4, w = 2/3)  # more weight to sens
comp_accu(hi = 3, mi = 2, fa = 1, cr = 4, w = 1/3)  # more weight to spec


# }