## S3 method for class 'resamples':
diff(x, models = x$models, metric = x$metrics,
test = t.test,
confLevel = 0.95, adjustment = "bonferroni",
...)## S3 method for class 'diff.resamples':
summary(object, digits = max(3, getOption("digits") - 3), ...)
compare_models(a, b, metric = a$metric[1])
resamples
estimate
and p.value
diff.resamples
p.adjust
.dotplot.diff.resamples
. See Details below.test
"diff.resamples"
with elements:test
An object of class "summary.diff.resamples"
with elements:
compare_models
) an object of class htest
resulting from t.test
.For each metric, all pair-wise differences are computed and tested to assess if the difference is equal to zero.
When a Bonferroni correction is used, the confidence level is changed from confLevel
to 1-((1-confLevel)/p)
here p
is the number of pair-wise comparisons are being made. For other correction methods, no such change is used.
compare_models
is a shorthand function to compare two models using a single metric. It returns the results of t.test
on the differences.
Eugster et al. Exploratory and inferential analysis of benchmark experiments. Ludwigs-Maximilians-Universitat Munchen, Department of Statistics, Tech. Rep (2008) vol. 30
resamples
, dotplot.diff.resamples
, densityplot.diff.resamples
, bwplot.diff.resamples
, levelplot.diff.resamples
#load(url("http://topepo.github.io/caret/exampleModels.RData"))
resamps <- resamples(list(CART = rpartFit,
CondInfTree = ctreeFit,
MARS = earthFit))
difs <- diff(resamps)
difs
summary(difs)
compare_models(rpartFit, ctreeFit)
Run the code above in your browser using DataLab