Monte Carlo sensitivity analysis for the power of the omnibus
F-test of the squared multiple correlation coefficient (\(R^2\)).
Given an estimated_R2 used for sample size planning and a
true_R2 that actually obtains in the population (the two need not
agree), the function draws G replications, fits the regression,
compares \(F\) to its critical value, and reports the realized empirical
power and a summary of the realized \(R^2\) and \(F\) distributions.
The simulation honors the same random_predictors /
generate_random_predictors crossing as
ss_aipe_R2_sensitivity, so the user can examine the effect
of planning under one regression model (fixed or random predictors) but
actually realizing the other.
ss_power_R2_sensitivity(
true_R2 = NULL,
estimated_R2 = NULL,
desired_power = 0.85,
p = NULL,
alpha_level = 0.05,
random_predictors = TRUE,
specified_N = NULL,
generate_random_predictors = TRUE,
rho_yx = 0.3,
rho_xx = 0.3,
G = 10000,
print_iter = TRUE,
filename = NULL
)A data.frame with columns term and value
summarizing the Monte Carlo sensitivity analysis across G
replications. The term entries are: total_N (the
sample size evaluated), empirical_power (the proportion of
replications on which \(F\) exceeded the critical value),
analytic_power (computed from ss_power_R2 under
the same model as planning), mean_R2 / median_R2 /
sd_R2 and mean_F / median_F / sd_F
(summaries of the realized \(R^2\) and \(F\)), F_crit
(the critical value), and the input echoes p,
true_R2, estimated_R2 and desired_power (both
NA when specified_N was supplied instead), and
alpha_level. The result carries the
dmar_ss_power_sensitivity class, so tidy
reports the planned sample size beside the empirical and analytic power,
and glance adds the simulated \(R^2\) and
\(F\) distribution beside the echoed inputs.
Value of the population squared multiple correlation coefficient
Value of the squared multiple correlation coefficient used for sample size planning. Either estimated_R2 or specified_N must be supplied (not both).
Desired degree of statistical power used for planning
Number of predictors
Type I error rate
Whether the sample size planning step treats predictors as random (TRUE, the default) or fixed (FALSE)
Sample size at which the realized power should be computed; alternative to specifying estimated_R2
Whether the internal simulation should generate predictors as random (TRUE, the default) or fixed (FALSE)
Correlation between the dependent variable (Y) and each of the X variables
Correlation among the X variables (off-diagonal of the predictor correlation matrix)
Number of Monte Carlo replications
Whether to print the iteration number during the simulation
Optional path of a CSV file to receive the per-replication
results (the observed \(R^2\) and \(F\) statistic), overwriting any
file already at that path; the default NULL writes nothing, and a
throwaway run that wants the file should point it at
tempfile(fileext = ".csv").
Ken Kelley [email protected]
When estimated_R2 equals true_R2, the function performs a
straight Monte Carlo evaluation of the planning procedure (no
misspecification). Pass specified_N to evaluate realized power at
a specified sample size; in that case estimated_R2 must not be
supplied. The crossing of random_predictors (used in planning)
with generate_random_predictors (used in the simulation) lets
the user inspect the consequences of planning under one regression
model but realizing the other. See Gatsonis and Sampson (1989) for
the comparison of fixed and random predictor power for the omnibus
test.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
Gatsonis, C., & Sampson, A. R. (1989). Multiple correlation: Exact power and sample size calculations. Psychological Bulletin, 106(3), 516--524.
Kelley, K., & Maxwell, S. E. (2008). Sample size planning with applications to multiple regression: Power and accuracy for omnibus and targeted effects. In P. Alasuutari, L. Bickman, & J. Brannen (Eds.), The Sage handbook of social research methods (pp. 166--192). Sage.
Lee, Y. S. (1971). Some results on the sampling distribution of the multiple correlation coefficient. Journal of the Royal Statistical Society, Series B, 33(1), 117--130.
Maxwell, S. E., Delaney, H. D., & Kelley, K. (2027). Designing experiments and analyzing data: A model comparison perspective (4th ed.). Routledge. (See Chapter 3 on \(R^2\) as a model comparison effect size.)
ss_power_R2, ss_aipe_R2_sensitivity,
ci_R2
design_consequences for what a chosen design delivers:
power, the Type S (sign) and Type M (exaggeration) errors of the
significance filter, and the expected confidence interval width.
Other sample size for power:
power_fisher_exact(),
ss_aipe_mixed_effects(),
ss_power_R2(),
ss_power_c(),
ss_power_c_ancova(),
ss_power_composite_ancova(),
ss_power_composite_ancova_2group(),
ss_power_composite_anova(),
ss_power_composite_factorial_ancova(),
ss_power_composite_factorial_ancova_het(),
ss_power_composite_factorial_anova(),
ss_power_composite_sem(),
ss_power_contrast(),
ss_power_equivalence_c(),
ss_power_factorial_ancova(),
ss_power_factorial_anova(),
ss_power_indirect_effect(),
ss_power_mixed_effects(),
ss_power_one_way_anova(),
ss_power_pcm(),
ss_power_r(),
ss_power_rc(),
ss_power_reg_coef(),
ss_power_reg_coef_sensitivity(),
ss_power_rm_anova(),
ss_power_sc(),
ss_power_sem(),
ss_power_smd(),
ss_power_split_plot_anova()
set.seed(113)
# Realized power when planning under the fixed-predictor model but the
# data are actually generated with random predictors. G = 200 keeps the
# example quick; a reported analysis deserves the default G of 10000.
ss_power_R2_sensitivity(true_R2 = 0.30, estimated_R2 = 0.30,
desired_power = 0.80, p = 5,
random_predictors = FALSE,
generate_random_predictors = TRUE,
G = 200, print_iter = FALSE)
Run the code above in your browser using DataLab