Lattice Functions for Visualizing Resampling Results
Lattice functions for visualizing resampling results across models
## S3 method for class 'resamples': xyplot(x, data = NULL, what = "scatter", models = NULL, metric = x$metric, units = "min", ...)
## S3 method for class 'resamples': dotplot(x, data = NULL, models = x$models, metric = x$metric, conf.level = 0.95, ...)
## S3 method for class 'resamples': densityplot(x, data = NULL, models = x$models, metric = x$metric, ...)
## S3 method for class 'resamples': bwplot(x, data = NULL, models = x$models, metric = x$metric, ...)
## S3 method for class 'resamples': splom(x, data = NULL, variables = "models", models = x$models, metric = NULL, panelRange = NULL, ...)
## S3 method for class 'resamples': parallel(x, data = NULL, models = x$models, metric = x$metric, ...)
- an object generated by
- Not used
- a character string for which models to plot. Note:
xyplotrequires one or two models whereas the other methods can plot more than two.
- a character string for which metrics to use as conditioning variables in the plot.
splomrequires exactly one metric when
variables = "models"and at least two when
variables = "metrics".
- either "models" or "metrics"; which variable should be treated as the scatter plot variables?
- a common range for the panels. If
NULL, the panel ranges are derived from the values across all the models
xyplot, the type of plot. Valid options are: "scatter" (for a plot of the resampled results between two models), "BlandAltman" (a Bland-Altman, aka MA plot between two models), "tTime" (for the total time to run
- either "sec", "min" or "hour"; which
whatis either "tTime", "mTime" or "pTime", how should the timings be scaled?
- the confidence level for intervals about the mean (obtained using
- further arguments to pass to either
The ideas and methods here are based on Hothorn et al (2005) and Eugster et al (2008).
dotplot plots the average performance value (with two-sided confidence limits) for each model and metric.
bwplot display univariate visualizations of the resampling distributions while
splom shows the pair-wise relationships.
- a lattice object
Hothorn et al. The design and analysis of benchmark experiments. Journal of Computational and Graphical Statistics (2005) vol. 14 (3) pp. 675-699
Eugster et al. Exploratory and inferential analysis of benchmark experiments. Ludwigs-Maximilians-Universitat Munchen, Department of Statistics, Tech. Rep (2008) vol. 30
#load(url("http://caret.r-forge.r-project.org/Classification_and_Regression_Training_files/exampleModels.RData")) resamps <- resamples(list(CART = rpartFit, CondInfTree = ctreeFit, MARS = earthFit)) dotplot(resamps, scales =list(x = list(relation = "free")), between = list(x = 2)) bwplot(resamps, metric = "RMSE") densityplot(resamps, auto.key = list(columns = 3), pch = "|") xyplot(resamps, models = c("CART", "MARS"), metric = "RMSE") splom(resamps, metric = "RMSE") splom(resamps, variables = "metrics") parallel(resamps, metric = "RMSE")