Given estimated_R2 and true_R2, one can perform a sensitivity analysis to determine the
effect of a misspecified population squared multiple correlation coefficient using the Accuracy in
Parameter Estimation (AIPE) approach to sample size planning. The function evaluates the effect of
a misspecified true_R2 on the width of obtained confidence intervals.
ss_aipe_R2_sensitivity(
true_R2 = NULL,
estimated_R2 = NULL,
w = NULL,
p = NULL,
random_predictors = TRUE,
specified_N = NULL,
assurance = NULL,
conf_level = 0.95,
generate_random_predictors = TRUE,
rho_yx = 0.3,
rho_xx = 0.3,
G = 10000,
print_iter = TRUE,
filename = NULL
)A data.frame with columns term and value
summarizing the Monte Carlo sensitivity analysis across G
replications. The term entries are: mean_lower_limit,
median_lower_limit, sd_lower_limit,
mean_upper_limit, median_upper_limit,
sd_upper_limit (summaries of the realized confidence limits);
mean_R2, median_R2, sd_R2 (summaries of the
observed \(R^2\)); mean_ci_width_lower,
median_ci_width_lower, sd_ci_width_lower,
mean_ci_width_upper, median_ci_width_upper,
sd_ci_width_upper (summaries of the one-sided widths, measured
from the observed \(R^2\) to each limit); mean_ci_width,
median_ci_width, sd_ci_width (summaries of the full
interval widths); pct_ci_less_w (proportion of intervals with
width at or below the planning target w);
pct_ci_miss_low and pct_ci_miss_high (tail-specific
empirical non-coverage of true_R2);
total_type_I_error (overall empirical non-coverage, the sum of
the two tails); num_probs_with_cis (number of replications on
which a confidence interval could not be obtained); and the input
echoes total_N (the sample size evaluated), p,
true_R2, estimated_R2 (NA when specified_N was
supplied instead), width, conf_level, and
assurance (present only when an assurance was supplied). The
proportion rows are on the 0 to 1 scale, not percentages.
Value of the population squared multiple correlation coefficient
Value of the estimated (for sample size planning) squared multiple correlation coefficient
Full confidence interval width of interest
Number of predictors
Whether or not the sample size procedure and the simulation itself should be based on random (set to TRUE) or fixed predictors (set to FALSE)
Selected sample size to use in order to determine distributional properties at a given value of sample size
Parameter to ensure confidence interval width with a specified degree of certainty
Confidence interval coverage (symmetric coverage)
Specify whether the simulation should be based on random (default) or fixed regressors.
Value of the correlation between y (dependent variable) and each of the x variables (independent variables)
Value of the correlation among the x variables (independent variables)
Number of generations (i.e., replications) of the simulation
Should the iteration number (between 1 and G) during the run of the function
Optional path of a CSV file to receive the per-replication
results (the confidence limits, the observed \(R^2\), and the
one-sided and full interval widths), overwriting any file already at
that path; the default NULL writes nothing, and a throwaway run
that wants the file should point it at tempfile(fileext = ".csv").
Ken Kelley [email protected]
When estimated_R2=true_R2, the results are that of a simulation study when all assumptions
are satisfied. Rather than specifying estimated_R2, one can specify specified_N to determine
the results of a particular sample size (when doing this estimated_R2 cannot be specified).
The sample size estimation procedure technically assumes multivariate normal variables (p+1) with
fixed predictors (x/independent variables), yet the function assumes random multivariate normal
predictors (having a p+1 multivariate distribution). As Gatsonis and Sampson (1989) note in the
context of statistical power analysis (recall this function is used in the context of precision), there
is little difference in the outcome.
In the behavioral, educational, and social sciences, predictor variables are almost always random, and
thus random_predictors should generally be used. random_predictors=TRUE specifies how both
the sample size planning procedure and the confidence intervals are calculated based on the random
predictors/regressors. The internal simulation generates random or fixed predictors/regressors based on
whether variables predictor variables are random or fixed. However, when random_predictors=FALSE,
only the sample size planning procedure and the confidence intervals are calculated based on the
parameter. The parameter generate_random_predictors (where the default is TRUE so that
random predictors/regressors are generated) allows random or fixed predictor variables to be generated.
Because the sample size planning procedure and the internal simulation are both specified, for purposes
of sensitivity analysis random/fixed can be crossed to examine the effects of specifying sample size
based on one but using it on data based on the other.
Algina, J. & Olejnik, S. (2000). Determining sample size for accurate estimation of the squared multiple correlation coefficient. Multivariate Behavioral Research, 35, 119--137. tools:::Rd_expr_doi("10.1207/s15327906mbr3501_5")
Gatsonis, C. & Sampson, A. R. (1989). Multiple Correlation: Exact power and sample size calculations. Psychological Bulletin, 106(3), 516--524.
Kelley, K. (2008). Sample size planning for the squared multiple correlation coefficient: Accuracy in parameter estimation via narrow confidence intervals, Multivariate Behavioral Research, 43(4), 524--555. tools:::Rd_expr_doi("10.1080/00273170802490632")
Kelley, K., & Maxwell, S. E. (2008). Sample size planning with applications to multiple regression: Power and accuracy for omnibus and targeted effects. In P. Alasuutari, L. Bickman, & J. Brannen (Eds.), The Sage handbook of social research methods (pp. 166--192). Sage.
Maxwell, S. E., Delaney, H. D., & Kelley, K. (2027). Designing experiments and analyzing data: A model comparison perspective (4th ed.). Routledge. (See Chapter 3 on \(R^2\) as a model comparison effect size.)
Steiger, J. H., & Fouladi, R. T. (1992). R2: A computer program for interval estimation, power calculations, sample size estimation, and hypothesis testing in multiple regression. Behavior Research Methods, Instruments, & Computers, 24(4), 581--582. tools:::Rd_expr_doi("10.3758/BF03203611")
ci_R2, ci_nc_t, ss_aipe_R2
design_consequences for what a chosen design delivers:
power, the Type S (sign) and Type M (exaggeration) errors of the
significance filter, and the expected confidence interval width.
# The planner used .4 for a population squared multiple correlation
# coefficient that is really .5. G = 25 keeps the example quick; a
# reported sensitivity analysis deserves the default G of 10000.
set.seed(113)
ss_aipe_R2_sensitivity(true_R2 = .5, estimated_R2 = .4, w = .10, p = 5,
conf_level = 0.95, G = 25, print_iter = FALSE)
Run the code above in your browser using DataLab