postResample
Calculates performance across resamples
Given two numeric vectors of data, the mean squared error and R-squared are calculated. For two factors, the overall agreement rate and Kappa are determined.
- Keywords
- utilities
Usage
postResample(pred, obs)
defaultSummary(data, lev = NULL, model = NULL)
Arguments
- pred
- A vector of numeric data (could be a factor)
- obs
- A vector of numeric data (could be a factor)
- data
- a data frame or matrix with columns
obs
andpred
for hte observed and predicted outcomes - lev
- a character vector of factors levels for the response. In regression cases, this would be
NULL
. - model
- a character string for the model name (as taken form the
method
argument oftrain
.
Details
postResample
is meant to be used with apply
across a matrix. For numeric data
the code checks to see if the standard deviation of either vector is zero. If so, the correlation
between those samples is assigned a value of zero. NA
values are ignored everywhere.
Note that many models have more predictors (or parameters) than data points, so the typical mean squared
error denominator (n - p) does not apply. Root mean squared error is calculated using sqrt(mean((pred - obs)^2
.
Also, R-squared is calculated as the square of the correlation between the observed and predicted outcomes.
For defaultSummary
is the default function to compute performance metrics in train
. It is a wrapper around postResample
.
Other functions can be used via the summaryFunction
argument of trainControl
. Custom functions must have the same arguments asdefaultSummary
.
Value
- A vector of performance estimates.
See Also
Examples
predicted <- matrix(rnorm(50), ncol = 5)
observed <- rnorm(10)
apply(predicted, 2, postResample, obs = observed)