Collation and Visualization of Resampling Results
These functions provide methods for collection, analyzing and visualizing a set of resampling results from a common data set.
## S3 method for class 'default': resamples(x, modelNames = names(x), ...)
## S3 method for class 'resamples': summary(object, ...)
- a list of two or more
trainobjects with a common set of resampling indices in the
- an optional set of names to give to the resampling results
- an object generated by
- not currently used
The results from
train can have more than one performance metric per resample. Each metric in the input object is saved.
resamples checks that the resampling results match; that is, the indices in the object
trainObject$control$index are the same. Also, the argument
returnResamp should have a value of
"final" for each model.
The summary function computes summary statistics across each model/metric combination.
- An object with class
call the call values a data frame of results where rows correspond to resampled data sets and columns indicate the model and metric models a character string of model labels metrics a character string of performance metrics methods a character string of the
methodargument values for each model
data(BloodBrain) set.seed(1) tmp <- createDataPartition(logBBB, p = .8, times = 100) rpartFit <- train(bbbDescr, logBBB, "rpart", tuneLength = 16, trControl = trainControl( method = "LGOCV", index = tmp)) ctreeFit <- train(bbbDescr, logBBB, "ctree", trControl = trainControl( method = "LGOCV", index = tmp)) earthFit <- train(bbbDescr, logBBB, "earth", tuneLength = 20, trControl = trainControl( method = "LGOCV", index = tmp)) resamps <- resamples(list(CART = rpartFit, CondInfTree = ctreeFit, MARS = earthFit)) ## or load pre-calculated results using: ## load(url("http://caret.r-forge.r-project.org/Classification_and_Regression_Training_files/exampleModels.RData")) resamps summary(resamps)