ss_power_reg_coef_sensitivity: Sensitivity Analysis for the Power of a Targeted Regression Coefficient

Description

Monte Carlo sensitivity analysis for the statistical power of the t-test of a targeted regression coefficient. Given a planned (estimated_*) covariance structure and a true (true_*) covariance structure, the function draws G replications, fits the multiple regression, and reports the empirical proportion of replications on which the t-test of the targeted coefficient rejects, together with the realized distribution of \(\hat b_j\), its standard error, and the test statistic. ss_power_reg_coef_sensitivity() is the power-oriented sibling of ss_aipe_reg_coef_sensitivity (which is CI-width oriented).

Usage

ss_power_reg_coef_sensitivity(
  true_var_Y = NULL,
  true_cov_YX = NULL,
  true_cov_XX = NULL,
  estimated_var_Y = NULL,
  estimated_cov_YX = NULL,
  estimated_cov_XX = NULL,
  specified_N = NULL,
  which_predictor = 1,
  desired_power = 0.85,
  alpha_level = 0.05,
  directional = FALSE,
  standardize = FALSE,
  G = 1000,
  print_iter = TRUE,
  filename = NULL
)

Value

A data.frame with columns term and value

summarizing the Monte Carlo sensitivity analysis. The term

entries are: total_N (the sample size evaluated), empirical_power, analytic_power (computed from ss_power_reg_coef), the mean / median / SD of the realized \(\hat b_j\) (mean_b_j, median_b_j, sd_b_j), of its standard error (mean_se_b_j, median_se_b_j, sd_se_b_j), of the test statistic (mean_t, median_t, sd_t), and of the squared multiple correlation coefficient (mean_R2, median_R2, sd_R2), t_crit (the critical value), and the input echoes p, which_predictor, true_b_j and estimated_b_j (the population and planning values of the targeted coefficient implied by the supplied covariance structures), desired_power (NA when specified_N was supplied instead), and alpha_level. The result carries the dmar_ss_power_sensitivity class, so tidy

reports the planned sample size beside the empirical and analytic power, and glance adds the simulated estimator distribution beside the echoed inputs.

Arguments

true_var_Y: Population variance of the dependent variable (Y)
true_cov_YX: Population covariance vector between the p predictor variables and the dependent variable (Y)
true_cov_XX: Population covariance matrix of the p predictor variables
estimated_var_Y: Estimated variance of the dependent variable (Y) used in sample size planning. Defaults to true_var_Y.
estimated_cov_YX: Estimated covariance vector between the predictor variables and the dependent variable used in sample size planning. Defaults to true_cov_YX.
estimated_cov_XX: Estimated covariance matrix of the predictor variables used in sample size planning. Defaults to true_cov_XX.
specified_N: Directly specified sample size; if supplied, sample size planning is skipped.
which_predictor: Index identifying which of the p predictors is the targeted predictor for the power test.
desired_power: Desired degree of statistical power used for planning
alpha_level: Type I error rate
directional: Whether a one-sided or two-sided test is used
standardize: Whether each replication's data should be standardized prior to fitting (giving a standardized regression coefficient)
G: Number of Monte Carlo replications
print_iter: Whether to print the iteration number during the simulation
filename: Optional path of a CSV file to receive the per-replication results (the coefficient estimate, its standard error, the t statistic, and the observed \(R^2\)), overwriting any file already at that path; the default NULL writes nothing, and a throwaway run that wants the file should point it at tempfile(fileext = ".csv").

Author

Ken Kelley [email protected]

Details

When the estimated and true covariance structures are identical, the function performs a Monte Carlo evaluation of the planning procedure (no misspecification); when they differ, it performs a sensitivity analysis on the consequences of misspecifying the population covariance structure for the targeted coefficient's power. The planning step calls ss_power_reg_coef with the estimated covariance structure; the simulation step generates data from the true covariance structure.

References

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

Kelley, K., & Maxwell, S. E. (2003). Sample size for multiple regression: Obtaining regression coefficients that are accurate, not simply significant. Psychological Methods, 8(3), 305--321. tools:::Rd_expr_doi("10.1037/1082-989X.8.3.305")

Kelley, K., & Maxwell, S. E. (2008). Sample size planning with applications to multiple regression: Power and accuracy for omnibus and targeted effects. In P. Alasuutari, L. Bickman, & J. Brannen (Eds.), The Sage handbook of social research methods (pp. 166--192). Sage.

Maxwell, S. E. (2000). Sample size and multiple regression analysis. Psychological Methods, 5(4), 434--458. tools:::Rd_expr_doi("10.1037/1082-989X.5.4.434")

Maxwell, S. E., Delaney, H. D., & Kelley, K. (2027). Designing experiments and analyzing data: A model comparison perspective (4th ed.). Routledge. (See Chapter 4 on individual comparisons of means and Chapter 6 on trend analysis.)

Examples

Run this code

# Targeted coefficient power sensitivity with two predictors. The
# default G = 1000 replications is used in practice; G is reduced here
# so the example runs quickly.
set.seed(113)
Sigma_X <- matrix(c(1, 0.3, 0.3, 1), nrow = 2)
cov_YX  <- c(0.4, 0.3)
ss_power_reg_coef_sensitivity(
  true_var_Y = 1, true_cov_YX = cov_YX, true_cov_XX = Sigma_X,
  which_predictor = 1, desired_power = 0.80,
  G = 100, print_iter = FALSE
)

Run the code above in your browser using DataLab