This function performs a Kolmogorov-Smirnov test on each pair of algorithms in the input x to determine which algorithm gives a significantly smaller running time. The resulting p-values are arranged in a matrix, where each cell (i, j) contains a p-value from the test with alternative hypothesis: the running time of algorithm i is smaller (thus better) than that of j.
pairwise.test(x, ...)# S3 method for list
pairwise.test(x, max_eval, bootstrap.size = 30, ...)
# S3 method for DataSetList
pairwise.test(x, ftarget, bootstrap.size = 0, which = "by_FV", ...)
A matrix containing p-values of the test
either a list that contains running time sample for each algorithm as sub-lists, or a DataSetList object
all other options
list that contains the maximal running time for each algorithm as sub-lists
integer, the size of the bootstrapped sample. Set to 0 to disable bootstrapping
float, the target value used to determine the running / hitting
wheter to do fixed-target ('by_FV') or fixed-budget ('by_RT') comparison time
pairwise.test(subset(dsl, funcId == 1), 16)
Run the code above in your browser using DataLab