CAISEr (version 0.3.3)

calc_se: Calculates the standard error for simple and percent differences

Description

Calculates the sample standard error for the estimator differences between two algorithms on a given instance.

Usage

calc_se(x1, x2, dif, method = "param", boot.R = 999)

Arguments

x1

vector of observations

x2

vector of observations

dif

name of the difference for which the SE is desired. Accepts "perc" (for percent differences) or "simple" (for simple differences)

method

method used to calculate the interval. Accepts "param" (using parametric formulas based on normality of the sampling distribution of the means) or "boot" (for bootstrap).

boot.R

(optional) number of bootstrap resamples.

Value

a list object containing the following items:

• x.est - estimated value

• se - standard error

Details

• If dif == "perc" it returns the SE for sample estimator of \((mu2 - mu1) / mu1\), where \(mu1, mu2\) are the means of the populations that generated the sample vectors \(x1, x2\).

• If dif == "simple" it returns the SE for sample estimator of \((mu2 - mu1)\)

References

• F. Campelo, F. Takahashi: Sample size estimation for power and accuracy in the experimental comparison of algorithms (submitted, 2017).

Examples

# NOT RUN {
# two vectors of normally distributed observations
set.seed(1234)
x1 <- rnorm(100, 5, 1)  # mean = 5, sd = 1
x2 <- rnorm(200, 10, 2) # mean = 10, sd = 2

# Theoretical SE for simple difference: 0.1732051
calc_se(x1, x2, dif = "simple", method = "param")

# Theoretical (Fieller, no covariance) SE for percent differences: 0.04
calc_se(x1, x2, dif = "perc", method = "boot")
# }