Conf: Confusion Matrix And Associated Statistics

Description

Calculates a cross-tabulation of observed and predicted classes with associated statistics.

Usage

Conf(x, ...)

## S3 method for class 'table':
Conf(x, pos = NULL, ...) 
## S3 method for class 'matrix':
Conf(x, pos = NULL, ...)
## S3 method for class 'default':
Conf(x, ref, pos = NULL, na.rm = TRUE, ...) 
  
## S3 method for class 'rpart':
Conf(x, ...)
## S3 method for class 'multinom':
Conf(x, ...)
## S3 method for class 'glm':
Conf(x, cutoff = 0.5, ...)
## S3 method for class 'randomForest':
Conf(x, ...)
## S3 method for class 'svm':
Conf(x, ...)
## S3 method for class 'regr':
Conf(x, ...)

## S3 method for class 'Conf':
plot(x, main="Confusion Matrix", ...)

## S3 method for class 'Conf':
print(x, digits = max(3, getOption("digits") - 3), ...) 

Sens(x, ...) 
Spec(x, ...)

Arguments

a vector, normally a factor, of predicted classes (for the default method) or an object of class table.

ref

a vector, normally a factor, of classes to be used as the reference.

pos

a character string that defines the factor level corresponding to the "positive" results.

cutoff

the cutoff point for separating the classes. This is only used if x is a glm-model and ignored else.

main

overall title for the plot.

digits

controls the number of digits to print.

na.rm

a logical value indicating whether or not missing values should be removed. Defaults to FALSE.

...

further arguments to be passed to or from methods.

Value

a list with elements
tablethe results of table on data and reference
positivethe positive result level
overalla numeric vector with overall accuracy and Kappa statistic values
byClassthe sensitivity, specificity, positive predictive value, negative predictive value, prevalence, dection rate and detection prevalence for each class. For two class systems, this is calculated once using the positive argument

Details

The functions requires that the factors have exactly the same levels. For two class problems, the sensitivity, specificity, positive predictive value and negative predictive value is calculated using the positive argument. Also, the prevalence of the "event" is computed from the data (unless passed in as an argument), the detection rate (the rate of true events also predicted to be events) and the detection prevalence (the prevalence of predicted events). Suppose a 2x2 table with notation rcc{ Reference Predicted Event No Event Event A B No Event C D } The formulas used here are: $$Sensitivity = A/(A+C)$$ $$Specificity = D/(B+D)$$ $$Prevalence = (A+C)/(A+B+C+D)$$ $$PPV = (sensitivity * Prevalence)/((sensitivity*Prevalence) + ((1-specificity)*(1-Prevalence)))$$ $$NPV = (specificity * (1-Prevalence))/(((1-sensitivity)*Prevalence) + ((specificity)*(1-Prevalence)))$$ $$Detection Rate = A/(A+B+C+D)$$ $$Detection Prevalence = (A+B)/(A+B+C+D)$$ See the references for discusions of the first five formulas. For more than two classes, these results are calculated comparing each factor level to the remaining levels (i.e. a "one versus all" approach). The overall accuracy and unweighted Kappa statistic are calculated. A p-value from McNemar's test is also computed using mcnemar.test (which can produce NA values with sparse tables). The overall accuracy rate is computed along with a 95 percent confidence interval for this rate (using BinomCI) and a one-sided test to see if the accuracy is better than the "no information rate," which is taken to be the largest class percentage in the data. The sensitivity is defined as the proportion of positive results out of the number of samples which were actually positive. When there are no positive results, sensitivity is not defined and a value of NA is returned. Similarly, when there are no negative results, specificity is not defined and a value of NA is returned. Similar statements are true for predictive values.

References

Kuhn, M. (2008) Building predictive models in R using the caret package Journal of Statistical Software, (http://www.jstatsoft.org/v28/i05/).

Examples

Run this code

# let tab be a confusion table
tab <- structure(c(233L, 29L, 56L, 74L), .Dim = c(2L, 2L), 
          .Dimnames = structure(list(pred = c("0", "1"), obs = c("0", "1")), 
             .Names = c("pred", "obs")))

Conf(tab, pos="1")


pred <- Untable(tab)[,"pred"]
obs <- Untable(tab)[,"obs"]

Conf(x = pred, ref = obs)
Conf(x = pred, ref = obs, pos="1")

Sens(tab)   # Sensitivity
Spec(tab)   # Specificity

Run the code above in your browser using DataLab