Estimates categorical omega (\(\omega_C\); Green & Yang, 2009; Kelley & Pornprasertmanit, 2016) for a homogeneous composite of ordered-categorical items and returns a bootstrap confidence interval.
reliability_omega_categorical(
data,
ci_method = c("bca", "percentile", "bootstrap_se", "bootstrap_se_logistic", "none"),
conf_level = 0.95,
B = 10000,
seed = NULL
)A data.frame with columns term and value
and rows
"estimate" (sample \(\omega_C\)),
"se" (the bootstrap standard deviation across replications,
already on the coefficient scale; NA for
ci_method = "none"),
"lower_limit" and "upper_limit" (clamped to [0, 1]),
"conf_level", "N", "N_complete" (the complete
cases; equal to "N" here, and carried so the whole
reliability family returns one shape), and "J". Attributes
coefficient ("omega_categorical"), ci_method, and B
record the computation.
A numeric matrix or data frame of ordered-categorical item scores (integer codes for the categories). Rows with any missing values are listwise-deleted.
Method for constructing the confidence interval. See
Details. When not supplied, no interval is computed: every
interval for categorical omega is bootstrap based, and a bootstrap
is never run unless requested. Ask for "bca" (the
recommended method) or "percentile" to obtain one.
Confidence level for the interval. Defaults to
0.95.
Number of bootstrap replications. Defaults to 10000.
Random number seed used for bootstrap reproducibility.
Defaults to NULL, which leaves the user's current RNG state intact; supply an integer for reproducibility.
Ken Kelley [email protected]
Categorical omega is designed for items measured on an ordered categorical scale (e.g., Likert items). It uses a probit-link single-factor model in which each observed item \(X_j\) is modeled as a categorization of an underlying continuous response variable \(X_j^{*}\) via thresholds \(t_{j,c}\) (Muthén, 1984; Millsap & Yun-Tein, 2004), fit by diagonally weighted least squares with mean- and variance-adjusted test statistic (WLSMV), the standard estimator for ordered categorical items. With the delta parameterization (\(Var(X_j^{*}) = 1\)), the population categorical omega is $$\omega_C = \frac{\sum_{j=1}^{J} \sum_{j'=1}^{J} \sigma_{jj'}\!\left(\lambda_{j}\lambda_{j'}\right)}{\sum_{j=1}^{J} \sum_{j'=1}^{J} \sigma_{jj'}\!\left(\rho_{X_{j}^{*} X_{j'}^{*}}\right)},$$ where \(\sigma_{jj'}(r)\) is the model implied covariance of \((X_{j}, X_{j'})\) computed from a bivariate normal CDF over pairs of category thresholds and a correlation \(r\) (Green & Yang, 2009, Eq. 13--14, with the full coefficient their Eq. 21; Kelley & Pornprasertmanit, 2016, Eq. 17--18). The numerator uses model implied polychoric correlations (\(\lambda_j \lambda_{j'}\)), while the denominator uses observed polychoric correlations estimated from the data via a saturated bivariate model.
Kelley and Pornprasertmanit (2016) found in extensive Monte Carlo
simulation that the bias-corrected and accelerated (BCa) bootstrap
confidence interval for categorical omega achieved acceptable coverage
across a wide variety of threshold patterns, sample sizes, item
counts, and population reliability values, with one documented
exception: coverage dipped somewhat below the acceptable range when
the number of items and the population reliability were both high.
They specifically recommend BCa for categorical omega. Because no bootstrap runs in
DMAR unless the user requests one, the default output is the
point estimate with a message naming the call that produces the
recommended interval; request ci_method = "bca" to obtain it.
The interval is not a quick one. Every replication refits the
ordered-categorical factor model and the saturated polychoric model
behind the denominator, a fraction of a second for a scale of
moderate length, and the jackknife behind the BCa acceleration adds
one such refit per case, so the recommended interval at the default
B = 10000 runs for tens of minutes. The example on this page
therefore stops at the point estimate; the reported analysis adds
ci_method = "bca" to the same call, with seed for
reproducibility.
When to use. Use reliability_omega_categorical when items are
ordered-categorical, especially when (a) the number of categories is
small (e.g., two to five), (b) item distributions are skewed, or (c)
threshold patterns differ markedly across items. In Kelley and
Pornprasertmanit's (2016) Study 3, treating ordered items as
continuous and using reliability_omega with the
observed total variance in the denominator achieved
acceptable coverage only when threshold patterns were similar across
items; in their experience that condition is rare in practice.
Available confidence interval methods (set via ci_method).
Every interval here is bootstrap based: the rows of data are
resampled with replacement B times (10000 by default) and
categorical omega is recomputed, with the full WLSMV model refit, on
each replication (Efron & Tibshirani, 1993). Replications whose
refit fails or does not converge, most common with small samples and
sparse response categories, are dropped, and the interval is
computed from the replications that return a value. Bootstrap
results vary from run to run; supply seed for
reproducibility.
"bca"The bias-corrected and accelerated bootstrap,
the default and the specific recommendation of Kelley and
Pornprasertmanit (2016). Where the percentile interval reads its
limits directly off the empirical quantiles of the bootstrap
estimates, BCa adjusts the two quantile positions for median bias
(estimated from the bootstrap distribution) and for the rate at
which the estimator's variance changes with the parameter (the
acceleration, estimated by the jackknife), making it second-order
accurate where the percentile interval is first-order accurate
(DiCiccio & Efron, 1996). The adjusted quantile positions sit
farther into the tails than the percentile interval uses, which
is why the default B = 10000 is larger than the customary
2000; reduce B for exploration, not for a reported
analysis.
"percentile"Percentile bootstrap: the interval limits are the empirical quantiles of the bootstrap estimates.
"bootstrap_se", "bootstrap_se_logistic"Wald
intervals using the bootstrap standard deviation as a standard
error, built on the logit scale for the _logistic variant
so the endpoints respect [0, 1].
"none"Return only the point estimate.
Comparison with other packages. The psych package's
omega fits a Schmid-Leiman hierarchical factor
model on continuous (or treated-as-continuous) items and does not
implement categorical omega in the sense of Green and Yang (2009).
For ordered-categorical items reliability_omega_categorical is the
appropriate choice; polychoric provides
polychoric correlation estimation as a separate tool.
DiCiccio, T. J., & Efron, B. (1996). Bootstrap confidence intervals. Statistical Science, 11(3), 189--228.
Efron, B., & Tibshirani, R. J. (1993). An introduction to the bootstrap. New York, NY: Chapman & Hall/CRC.
Green, S. B., & Yang, Y. (2009). Reliability of summed item scores using structural equation modeling: An alternative to coefficient alpha. Psychometrika, 74, 155--167. tools:::Rd_expr_doi("10.1007/s11336-008-9099-3")
Kelley, K., & Cheng, Y. (2012). Estimation of and confidence interval formation for reliability coefficients of homogeneous measurement instruments. Methodology, 8, 39--50. tools:::Rd_expr_doi("10.1027/1614-2241/a000036")
Kelley, K., & Pornprasertmanit, S. (2016). Confidence intervals for population reliability coefficients: Evaluation of methods, recommendations, and software for composite measures. Psychological Methods, 21, 69--92. tools:::Rd_expr_doi("10.1037/a0040086")
Maxwell, S. E., Delaney, H. D., & Kelley, K. (2027). Designing experiments and analyzing data: A model comparison perspective (4th ed.). Routledge.
Millsap, R. E., & Yun-Tein, J. (2004). Assessing factorial invariance in ordered-categorical measures. Multivariate Behavioral Research, 39(3), 479--515. tools:::Rd_expr_doi("10.1207/s15327906mbr3903_4")
Muthén, B. (1984). A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators. Psychometrika, 49(1), 115--132.
Terry, L. J., & Kelley, K. (2012). Sample size planning for composite reliability coefficients: Accuracy in parameter estimation via narrow confidence intervals. British Journal of Mathematical and Statistical Psychology, 65, 371--401. tools:::Rd_expr_doi("10.1111/j.2044-8317.2011.02030.x")
reliability (general wrapper),
reliability_omega (use for continuous items),
reliability_kr20 (for dichotomous items),
omega,
polychoric.
Other reliability:
cohen_kappa(),
diagnosis_agreement,
fleiss_kappa(),
icc(),
reliability(),
reliability_H(),
reliability_alpha(),
reliability_kr20(),
reliability_omega()
set.seed(113)
# Six 5-category items with a single latent factor.
N <- 200
J <- 6
loadings <- rep(0.7, J)
eta <- rnorm(N)
latent <- outer(eta, loadings) +
matrix(rnorm(N * J), N, J) %*% diag(sqrt(1 - loadings^2))
items <- apply(latent, 2, function(x)
as.integer(cut(x, breaks = c(-Inf, -1.5, -0.5, 0.5, 1.5, Inf),
labels = FALSE)))
colnames(items) <- paste0("y", seq_len(J))
# Default: point estimate only, with a message naming the call that
# produces the recommended interval.
reliability_omega_categorical(data = items)
# The same items treated as continuous, for contrast. With five
# categories and thresholds spread across the latent scale the two
# coefficients nearly agree on these data; the gap widens as the
# categories get coarser and as the thresholds move into the tails,
# which is where the categorical coefficient is worth its cost.
reliability_omega(data = items)
Run the code above in your browser using DataLab