sortinghat (version 0.1)

errorest_apparent: Calculates the Apparent Error Rate for a specified classifier given a data set.

Description

For a given data matrix and its corresponding vector of labels, we calculate the apparent error rate (AER) for a given classifier.

Usage

errorest_apparent(x, y, train, classify, ...)

Arguments

x

a matrix of n observations (rows) and p features (columns)

y

a vector of n class labels

train

a function that builds the classifier. (See details.)

classify

a function that classifies observations from the classifier constructed by train. (See details.)

...

additional arguments passed to the function specified in train.

Value

• object of class errorest. The object is a named list that contains the following elements:

  the calculated apparent error-rate estimate

Details

The AER simply uses the data set as both the training and test data sets. The AER is well known to be biased downward in that it underestimates the true error rate of the classifier.

For the given classifier, two functions must be provided 1. to train the classifier and 2. to classify unlabeled observations. The training function is provided as train and the classification function as classify.

We expect that the first two arguments of the train function are x and y, corresponding to the data matrix and the vector of their labels, respectively. Additional arguments can be passed to the train function.

We stay with the usual R convention for the classify function. We expect that this function takes two arguments: 1. an object argument which contains the trained classifier returned from the function specified in train; and 2. a newdata argument which contains a matrix of observations to be classified -- the matrix should have rows corresponding to the individual observations and columns corresponding to the features (covariates). For an example, see lda.

Examples

require('MASS')
iris_x <- data.matrix(iris[, -5])
iris_y <- iris[, 5]

# Because the \\code{classify} function returns multiples objects in a list,
# we provide a wrapper function that returns only the class labels.
lda_wrapper <- function(object, newdata) { predict(object, newdata)$class }
errorest_apparent(x = iris_x, y = iris_y, train = MASS:::lda, classify = lda_wrapper)
# Output: 0.02

# The following code is equivalent for this example:
lda_out <- MASS:::lda(x = iris_x, grouping = iris_y)
lda_classifications <- predict(lda_out, newdata = iris_x)$class
mean(lda_classifications != iris_y)
# Output: 0.02