Calculates either the number of instances, the standardized effect size, or the power of a comparison of two algorithms
calc_instances(ninstances = NULL, power = NULL, d, sig.level = 0.05,
alternative = "two.sided", test.type = "t.test")the number of instances to be used in the experiment.
(desired) test power
minimally relevant effect size (MRES, expressed as a standardized effect size, i.e., "deviation from H0" / "standard deviation")
significance level (alpha) for the experiment
type of alternative hypothesis ("two.sided" or "one.sided")
type of test ("t.test", "wilcoxon", "binomial")
a list object containing the following items:
ninstances - number of instances
power - the power of the comparison
d - the effect size
sig.level - significance level
alternative - type of alternative hypothesis
test.type - type of test
If the parameter test.type is set to either Wilcoxon or Binomial, this
routine approximates the number of instances using the ARE of these tests
in relation to the paired t.test, using the formulas:
$$n.wilcox = n.ttest / 0.86 = 1.163 * n.ttest$$
$$n.binom = n.ttest / 0.637 = 1.570 * n.ttest$$
This routine uses the closed formula of the t-test to calculate the number of instances required for a comparison of two algorithms, considering a desired power, standardized effect size, and significance level. for cases where the number of instances is predefined, it can return the test power instead.