"Bonferroni-Dunn" test or the "Nemenyi" test.
"Bonferroni-Dunn" usually yields higher power as it does not
compare all algorithms to each other, but all algorithms to a
baseline instead.
Learners are drawn on the y-axis according to their average rank.
For test = "nemenyi" a bar is drawn, connecting all groups of not
significantly different learners.
For test = "bd" an interval is drawn arround the algorithm selected
as baseline. All learners within this interval are not signifcantly different
from the baseline.
Calculation:
$$ CD = q_{\alpha} \sqrt{(\frac{k(k+1)}{6N})}$$
Where \(q_\alpha\) is based on the studentized range statistic.
See references for details.generateCritDifferencesData(bmr, measure = NULL, p.value = 0.05,
baseline = NULL, test = "bd")BenchmarkResult]
Benchmark result.Measure]
Performance measure.
Default is the first measure used in the benchmark experiment.numeric(1)]
P-value for the critical difference. Default: 0.05character(1)]: [learner.id]
Select a learner.id as baseline for the test = "bd"
("Bonferroni-Dunn") critical differences
diagram.The critical difference Interval will then be positioned arround this learner.
Defaults to best performing algorithm.
For test = "nemenyi", no baseline is needed as it performs all pairwise
comparisons.character(1)]
Test for which the critical differences are computed.
“bd” for the Bonferroni-Dunn Test, which is comparing all
classifiers to a baseline, thus performing a comparison
of one classifier to all others.
Algorithms not connected by a single line are statistically different.
then the baseline.
“nemenyi” for the posthoc.friedman.nemenyi.test
which is comparing all classifiers to each other. The null hypothesis that
there is a difference between the classifiers can not be rejected for all
classifiers that have a single grey bar connecting them.critDifferencesData]. List containing:
data.frame] containing the info for the descriptive
part of the plotlist] of class pairwise.htest
contains the calculated
posthoc.friedman.nemenyi.testlist] containing info on the critical difference
and its positioningbaseline chosen for plottinggenerateCalibrationData,
generateFeatureImportanceData,
generateFilterValuesData,
generateFunctionalANOVAData,
generateLearningCurveData,
generatePartialDependenceData,
generateThreshVsPerfData,
getFilterValues,
plotFilterValues Other benchmark: BenchmarkResult,
benchmark,
convertBMRToRankMatrix,
friedmanPostHocTestBMR,
friedmanTestBMR,
getBMRAggrPerformances,
getBMRFeatSelResults,
getBMRFilteredFeatures,
getBMRLearnerIds,
getBMRLearnerShortNames,
getBMRLearners,
getBMRMeasureIds,
getBMRMeasures, getBMRModels,
getBMRPerformances,
getBMRPredictions,
getBMRTaskDescriptions,
getBMRTaskIds,
getBMRTuneResults,
plotBMRBoxplots,
plotBMRRanksAsBarChart,
plotBMRSummary,
plotCritDifferences