Quantifies how much misspecification of the population reliability
coefficient distorts an AIPE-based sample size plan for the
composite-score reliability. On each replication the function
simulates an n \(\times\) i item-by-subject data
matrix from a single-factor parallel-tests model whose population
reliability of the sum score equals true_reliability, fits
the requested estimator (alpha or omega) via the corresponding
reliability_* function with the supplied ci_method,
and records the realized reliability estimate and its confidence
interval.
Population model. Each item has a single common-factor
loading and uncorrelated unique error. With per-item variance
normalized to 1, the loading and unique variance are chosen so the
Cronbach-style sum-score reliability equals true_reliability:
$$\lambda^2 \;=\; \frac{\rho}{i(1 - \rho) + \rho}, \qquad
\psi^2 \;=\; 1 - \lambda^2,$$
where \(\rho = \)true_reliability and \(i\) is the item
count. Item scores are
\(y_{ij} = \lambda T_i + e_{ij}\), with \(T_i \sim N(0, 1)\) and
\(e_{ij} \sim N(0, \psi^2)\).
ss_aipe_reliability_sensitivity(
true_reliability = NULL,
estimated_reliability = NULL,
i,
width,
specified_N = NULL,
estimator = c("alpha", "omega"),
ci_method = NULL,
conf_level = 0.95,
assurance = NULL,
G = 1000,
print_iter = FALSE,
filename = NULL
)A data.frame with rows for mean / median / SD of
the realized reliability and CI width, the proportion of intervals
at or below width, tail-specific and overall non-coverage
of true_reliability, and the input echoes, including assurance (present only when an
assurance was supplied).
Population reliability coefficient (in \([0, 1)\)).
Reliability used to plan the study;
the function passes the implied lambda / psi^2 to
ss_aipe_reliability.
Number of items in the composite.
Desired full width of the CI on reliability.
Sample size to evaluate (incompatible with
estimated_reliability).
One of "alpha" (default; coefficient alpha
via reliability_alpha) or "omega" (composite
reliability via reliability_omega). For a
parallel-tests population the two coincide; differences in
sample estimates reflect estimator-specific finite-sample bias
and CI behavior.
CI method passed to the estimator. Default
"bonett" for alpha and "mlr" for omega.
Confidence level (default 0.95).
Optional assurance probability passed to the planner.
Number of Monte Carlo replications.
Logical.
Optional path for a comma separated file recording
every replication (the sample reliability estimate, the two confidence
limits, the interval width, and two indicators of whether the
interval missed true_reliability below or above): nothing is
written when filename is NULL (the default), a new
file with a header row is created otherwise, an existing file at
that path is appended to, and a throwaway run should point it at
tempfile(fileext = ".csv").
Ken Kelley [email protected]
Kelley, K., & Pornprasertmanit, S. (2016). Confidence intervals for population reliability coefficients: Evaluation of methods, recommendations, and software for composite measures. Psychological Methods, 21, 69--92. tools:::Rd_expr_doi("10.1037/a0040086")
Terry, L. J., & Kelley, K. (2012). Sample size planning for composite reliability coefficients: Accuracy in parameter estimation via narrow confidence intervals. British Journal of Mathematical and Statistical Psychology, 65, 371--401. tools:::Rd_expr_doi("10.1111/j.2044-8317.2011.02030.x")
Maxwell, S. E., Delaney, H. D., & Kelley, K. (2027). Designing experiments and analyzing data: A model comparison perspective (4th ed.). Routledge.
ss_aipe_reliability, reliability_alpha, reliability_omega
design_consequences for what a chosen design delivers:
power, the Type S (sign) and Type M (exaggeration) errors of the
significance filter, and the expected confidence interval width.
Other AIPE sample size planning:
ss_aipe_c_sensitivity(),
ss_aipe_cliff_delta(),
ss_aipe_cliff_delta_sensitivity(),
ss_aipe_composite_sem(),
ss_aipe_equivalence_r(),
ss_aipe_equivalence_r_sensitivity(),
ss_aipe_equivalence_smd(),
ss_aipe_equivalence_smd_sensitivity(),
ss_aipe_icc(),
ss_aipe_icc_sensitivity(),
ss_aipe_indirect_effect(),
ss_aipe_indirect_effect_sensitivity(),
ss_aipe_mixed_effects_sensitivity(),
ss_aipe_omega_squared(),
ss_aipe_omega_squared_sensitivity(),
ss_aipe_partial_r(),
ss_aipe_partial_r_sensitivity(),
ss_aipe_pcm_sensitivity(),
ss_aipe_r(),
ss_aipe_r_sensitivity(),
ss_aipe_semipartial_r(),
ss_aipe_semipartial_r_sensitivity()
# Reduced Monte Carlo sweep (small G) so the example runs quickly;
# raise G for a production sensitivity analysis.
set.seed(113)
ss_aipe_reliability_sensitivity(
true_reliability = 0.80,
estimated_reliability = 0.80,
i = 4, width = 0.15,
estimator = "alpha",
G = 20, print_iter = FALSE
)
Run the code above in your browser using DataLab