DMAR (version 1.0.0)

ss_aipe_cv_sensitivity: Sensitivity Analysis for Sample Size Planning From the Accuracy in Parameter Estimation Perspective for the Coefficient of Variation

Description

Quantifies how much misspecification of the population coefficient of variation can distort an AIPE-based sample size plan. Given a true (population) value of the coefficient of variation and the value that was used in planning, the function simulates draws of size N from a normal population, computes the confidence interval for the coefficient of variation on each replication, and summarizes how often the realized interval width is below the desired target and how often the interval covers the population value. This is the standard sensitivity-analysis workflow described in Kelley (2007) and Maxwell, Delaney, and Kelley (2027, Section 3.11 on sample size planning).

Usage

ss_aipe_cv_sensitivity(
  true_cv = NULL,
  estimated_cv = NULL,
  width = NULL,
  assurance = NULL,
  mean = 100,
  specified_N = NULL,
  conf_level = 0.95,
  G = 1000,
  print_iter = FALSE,
  filename = NULL
)

Value

A data.frame with columns term and value

summarizing the Monte Carlo results across the G replications. The term entries are: "mean_cv", "median_cv", "sd_cv" (mean / median / SD of the G observed sample coefficients of variation); "mean_ci_width", "median_ci_width", "sd_ci_width" (corresponding summaries of the realized interval widths); "pct_ci_less_w"

(proportion of intervals at or below the planning width width); "pct_ci_miss_low" and "pct_ci_miss_high" (tail-specific non-coverage); "total_type_I_error" (overall empirical non-coverage of true_cv); plus the input echoes "total_N" (the sample size evaluated), "true_cv", "estimated_cv" (NA when specified_N was supplied instead), "width", "conf_level", and "assurance" (present only when an assurance was supplied). The proportion rows are on the 0 to 1 scale, not percentages, so total_type_I_error is the sum of pct_ci_miss_low and pct_ci_miss_high.

Arguments

true_cv

Population coefficient of variation (the data generating value)

estimated_cv

Coefficient of variation used to plan the study (the value the researcher guessed when invoking ss_aipe_cv); must be positive. Supply this or specified_N but not both.

width

Desired (full) width of the two-sided confidence interval for the population coefficient of variation

assurance

Probability with which the realized interval should be no wider than width (must be NULL or strictly between 0 and 1; NULL means plan to the expected width without an assurance constraint)

mean

Population mean used by the simulator to generate data (the standard deviation is determined by mean and true_cv, since CV = sigma/mu). Default 100.

specified_N

Pre-specified sample size to evaluate (use this when you want the sensitivity results at a fixed N rather than at the N that ss_aipe_cv would recommend); incompatible with estimated_cv.

conf_level

Desired confidence level (i.e., 1 - Type I error rate); default 0.95

G

Number of Monte Carlo replications; defaults to 1000. Increase (e.g., 5000 or 10000) for stable Type I error estimates.

print_iter

Logical. If TRUE the simulation prints the iteration index after each replication (helpful for long runs); default FALSE.

filename

An optional path for a comma separated file recording every replication (the two confidence limits, the realized coefficient of variation, a coverage indicator, and the interval width): nothing is written when filename is NULL (the default), a new file with a header row is created otherwise, an existing file at that path is appended to, and a throwaway run should point it at tempfile(fileext = ".csv").

Author

Ken Kelley [email protected]

Details

Sample size planning for the coefficient of variation under the Accuracy in Parameter Estimation framework chooses N so that the expected (or, with assurance, the high-probability) confidence interval width is no larger than width (Kelley, 2007). Because the procedure assumes the planning value estimated_cv matches the population value true_cv, in practice the realized width will deviate from the planned width whenever the planning value is wrong. This sensitivity analysis quantifies the deviation by Monte Carlo simulation: the planned N is obtained from ss_aipe_cv with estimated_cv, then samples are drawn from the true population (with coefficient of variation true_cv) and the realized confidence interval widths are summarized.

For a discussion of AIPE-based sample size planning more generally and how sensitivity analyses guard against misspecification, see Maxwell, Delaney, & Kelley (2027, Section 3.5).

References

Chattopadhyay, B., & Kelley, K. (2016). Estimation of the coefficient of variation with minimum risk: A sequential method for minimizing sampling error and study cost. Multivariate Behavioral Research, 51(5), 627--648. tools:::Rd_expr_doi("10.1080/00273171.2016.1203279")

Kelley, K. (2007). Sample size planning for the coefficient of variation from the accuracy in parameter estimation approach. Behavior Research Methods, 39(4), 755--766. tools:::Rd_expr_doi("10.3758/BF03192966")

Maxwell, S. E., Delaney, H. D., & Kelley, K. (2027). Designing experiments and analyzing data: A model comparison perspective (4th ed.). Routledge. (See Chapter 3.)

See Also

ss_aipe_cv, ci_cv, cv

design_consequences for what a chosen design delivers: power, the Type S (sign) and Type M (exaggeration) errors of the significance filter, and the expected confidence interval width.

Examples

# Textbook scenario (Kelley, 2007). A researcher plans a study to estimate the
# coefficient of variation of reaction times with a 95% confidence interval no
# wider than .10. They guess from prior work that the population coefficient of
# variation is around .25 and apply ss_aipe_cv() to obtain a planned N.

# Question 1: how does the realized interval width behave when the planning
# value is correct (well-specified case)?
set.seed(113)
ss_aipe_cv_sensitivity(
  true_cv      = .25,
  estimated_cv = .25,
  width            = .10,
  assurance        = NULL,
  conf_level       = .95,
  G                = 200,
  print_iter       = FALSE
)

# Question 2: what happens if the planning value is materially smaller than
# the true coefficient of variation (a common direction of misspecification,
# since planning values are often optimistic)? The intervals will be wider on
# average than the target and the pct_ci_less_w will fall.
set.seed(113)
ss_aipe_cv_sensitivity(
  true_cv      = .35,
  estimated_cv = .25,
  width            = .10,
  assurance        = NULL,
  conf_level       = .95,
  G                = 200,
  print_iter       = FALSE
)