mlr (version 2.7)

friedmanPostHocTestBMR: Perform a posthoc Friedman-Nemenyi test.

Description

Performs a posthoc.friedman.nemenyi.test for a BenchmarkResult and a selected measure. This means all pairwise comparisons of learners are performed. The null hypothesis of the post hoc test is that each pair of learners is equal. If the null hypothesis of the included ad hoc friedman.test can be rejected a pairwise.htest is returned. If not, the function returns the corresponding friedman.test

Usage

friedmanPostHocTestBMR(bmr, measure = NULL, p.value = 0.05,
  aggregation = "default")

Value

A list of class pairwise.htest. See posthoc.friedman.nemenyi.test for details. Additionally two components are added to the list:f.rejnull[logical(1)]whether the according friedman.test rejects the Null hypothesis at the selected p.valuecrit.difference[list(2)]Minimal difference the mean ranks of two learners need to have in order to be significantly different

See Also

Other benchmark: BenchmarkResult, benchmark, convertBMRToRankMatrix, friedmanTestBMR, generateCritDifferencesData, getBMRAggrPerformances, getBMRFeatSelResults, getBMRFilteredFeatures, getBMRLearnerIds, getBMRLearners, getBMRMeasureIds, getBMRMeasures, getBMRPerformances, getBMRPredictions, getBMRTaskIds, getBMRTuneResults, plotBMRBoxplots, plotBMRRanksAsBarChart, plotBMRSummary, plotCritDifferences

Examples

lrns = list(makeLearner("classif.nnet"), makeLearner("classif.rpart"))
tasks = list(iris.task, sonar.task)
rdesc = makeResampleDesc("CV", iters = 2L)
res = benchmark(lrns, tasks, rdesc, acc)
friedmanPostHocTestBMR(res, acc, p.value = 0.1)