Generates data that can be used to plot a
critical differences plot. Computes the critical differences according
to either the
`"Bonferroni-Dunn"`

test or the `"Nemenyi"`

test.

`"Bonferroni-Dunn"`

usually yields higher power as it does not
compare all algorithms to each other, but all algorithms to a
`baseline`

instead.

Learners are drawn on the y-axis according to their average rank.

For `test = "nemenyi"`

a bar is drawn, connecting all groups of not
significantly different learners.

For `test = "bd"`

an interval is drawn arround the algorithm selected
as a baseline. All learners within this interval are not signifcantly different
from the baseline.

Calculation:
$$CD = q_{\alpha} \sqrt{\left(\frac{k(k+1)}{6N}\right)}$$

Where \(q_\alpha\) is based on the studentized range statistic.
See references for details.

```
generateCritDifferencesData(
bmr,
measure = NULL,
p.value = 0.05,
baseline = NULL,
test = "bd"
)
```

(`critDifferencesData`

). List containing:

- data
(data.frame) containing the info for the descriptive part of the plot

- friedman.nemenyi.test
(list) of class

`pairwise.htest`

contains the calculated PMCMRplus::frdAllPairsNemenyiTest- cd.info
(list) containing info on the critical difference and its positioning

- baseline
`baseline`

chosen for plotting- p.value
p.value used for the PMCMRplus::frdAllPairsNemenyiTest and for computation of the critical difference

- bmr
(BenchmarkResult)

Benchmark result.- measure
(Measure)

Performance measure. Default is the first measure used in the benchmark experiment.- p.value
(

`numeric(1)`

)

P-value for the critical difference. Default: 0.05- baseline
(

`character(1)`

): (`learner.id`

)

Select a`learner.id`

as baseline for the`test = "bd"`

("Bonferroni-Dunn") critical differences diagram. The critical difference interval will then be positioned arround this learner. Defaults to best performing algorithm.

For`test = "nemenyi"`

, no baseline is needed as it performs*all pairwise comparisons*.- test
(

`character(1)`

)

Test for which the critical differences are computed.

“bd” for the Bonferroni-Dunn Test, which is comparing all classifiers to a`baseline`

, thus performing a comparison of one classifier to all others.

Algorithms not connected by a single line are statistically different from the baseline.

“nemenyi” for the PMCMRplus::frdAllPairsNemenyiTest which is comparing all classifiers to each other. The null hypothesis that there is a difference between the classifiers can not be rejected for all classifiers that have a single grey bar connecting them.

Other generate_plot_data:
`generateCalibrationData()`

,
`generateFeatureImportanceData()`

,
`generateFilterValuesData()`

,
`generateLearningCurveData()`

,
`generatePartialDependenceData()`

,
`generateThreshVsPerfData()`

,
`plotFilterValues()`

Other benchmark:
`BenchmarkResult`

,
`batchmark()`

,
`benchmark()`

,
`convertBMRToRankMatrix()`

,
`friedmanPostHocTestBMR()`

,
`friedmanTestBMR()`

,
`getBMRAggrPerformances()`

,
`getBMRFeatSelResults()`

,
`getBMRFilteredFeatures()`

,
`getBMRLearnerIds()`

,
`getBMRLearnerShortNames()`

,
`getBMRLearners()`

,
`getBMRMeasureIds()`

,
`getBMRMeasures()`

,
`getBMRModels()`

,
`getBMRPerformances()`

,
`getBMRPredictions()`

,
`getBMRTaskDescs()`

,
`getBMRTaskIds()`

,
`getBMRTuneResults()`

,
`plotBMRBoxplots()`

,
`plotBMRRanksAsBarChart()`

,
`plotBMRSummary()`

,
`plotCritDifferences()`

,
`reduceBatchmarkResults()`