diff.resamples

0th

Percentile

Inferential Assessments About Model Performance

Methods for making inferences about differences between models

Keywords
models
Usage
## S3 method for class 'resamples':
diff(x, models = x$models, metric = x$metrics, test = t.test, ...)
## S3 method for class 'diff.resamples':
summary(object, digits = max(3, getOption("digits") - 3), ...)
Arguments
x
an object generated by resamples
models
a character string for which models to compare
metric
a character string for which metrics to compare
test
a function to compute differences. The output of this function should have scalar outputs called estimate and p.value
object
a object generated by diff.resamples
digits
the number of significant differences to display when printing
...
further arguments to pass to test
Details

For each metric, all pair-wise differences are computed and tested to assess if the difference is equal to zero.

Value

  • An object of class "diff.resamples" with elements:
  • callthe call
  • difsa list for each metric being compared. Each list contains a matrix with differences in columns and resamples in rows
  • statisticsa list of results generated by test
  • modelsa character string for which models were compared.
  • metricsa character string of performance metrics that were used
  • or...

    An object of class "summary.diff.resamples" with elements:

  • callthe call
  • tablea list of tables that show the differences and p-values

See Also

resamples

Aliases
  • diff.resamples
  • summary.diff.resamples
Examples
#load(url("http://caret.r-forge.r-project.org/Classification_and_Regression_Training_files/exampleModels.RData"))

resamps <- resamples(list(CART = rpartFit,
                          CondInfTree = ctreeFit,
                          MARS = earthFit))

difs <- diff(resamps)

difs

summary(difs)
Documentation reproduced from package caret, version 4.41, License: GPL-2

Community examples

Looks like there are no examples yet.