ci_omega_squared: Confidence Interval for Omega Squared (Effect Size for ANOVA)

Description

Computes the point estimate and an exact, noncentrality-based confidence interval for the population omega squared ($\omega^2$), the proportion of variance in the dependent variable accounted for by a fixed effect. Accepts either the raw ANOVA summary (F, effect df, error df, total N) or a fitted aov / lm object, in which case the function returns a row per effect (partial $\omega^2$ in factorial designs).

Usage

ci_omega_squared(
  object = NULL,
  F_value = NULL,
  df_effect = NULL,
  df_error = NULL,
  N = NULL,
  conf_level = 0.95,
  alpha_lower = NULL,
  alpha_upper = NULL
)

Value

A data.frame with one row per effect and the columns

effect, omega_squared (point estimate), lower_limit,

upper_limit, F_value, df_effect, df_error, and

N. When the raw-argument interface is used, effect is

"overall".

Arguments

object: Optional. A fitted aov or lm object. When supplied, the function loops over the non-Residuals rows of anova(object) and returns one row per effect.
F_value: Observed F-value from the fixed-effects ANOVA (ignored if object is supplied).
df_effect: Numerator degrees of freedom for the effect (ignored if object is supplied).
df_error: Error (residual) degrees of freedom (ignored if object is supplied).
N: Total sample size (ignored if object is supplied; nobs(object) is used instead).
conf_level: Desired confidence coverage; default 0.95. Used only when alpha_lower and alpha_upper are both NULL.
alpha_lower, alpha_upper: Optional Type I error on the lower and upper side. If both are NULL, a symmetric interval at conf_level is used. If both are supplied, conf_level is recomputed as 1 - alpha_lower - alpha_upper.

Author

Ken Kelley [email protected]

Details

Point estimate. The function reports the usual sample omega squared, which for a one-way design can be written as $$\hat{\omega}^2 = \frac{\mathit{SS}_{\text{effect}} - df_{\text{effect}} \cdot \mathit{MS}_{\text{error}}}{\mathit{SS}_{\text{total}} + \mathit{MS}_{\text{error}}} = \frac{df_{\text{effect}} (F - 1)}{df_{\text{effect}} (F - 1) + N}$$ (Hays, 1994; Keppel, 1991). For factorial designs the same formula applied per effect yields partial omega squared (Olejnik & Algina, 2003); values below zero are truncated to zero.

Confidence interval. The CI is constructed by Steiger's (2004, Proposition 1) confidence interval transformation principle: a CI for the noncentrality parameter $\lambda$ of the F distribution is obtained (via ci_nc_F) and then mapped through $$\omega^2_{\text{bound}} = \frac{\lambda_{\text{bound}}}{\lambda_{\text{bound}} + N}.$$ When the lower CI on $\lambda$ is not identified (i.e., the observed F is below the one-sided critical value), the lower limit on $\omega^2$ is set to 0, matching the convention used in ci_pvaf. In a one-way design, the interval produced here is identical to the CI for $\eta^2$ from ci_pvaf; the two estimands coincide in the population and differ only in their sample estimators (an implication of the confidence interval transformation principle of Steiger, 2004).

Sums of squares in factorial designs. When a fitted model is supplied, the function reads the F-values from anova(), which in base R uses Type I (sequential) sums of squares. For balanced designs, Types I, II, and III give identical F-values; for unbalanced designs they differ. If Type II or III F-values are required, compute them with e.g.\ car::Anova(object, type = 3) and pass the relevant F / df into the raw-argument interface.

References

Fleishman, A. I. (1980). Confidence intervals for correlation ratios. Educational and Psychological Measurement, 40(3), 659--670.

Hays, W. L. (1994). Statistics (5th ed.). Fort Worth, TX: Harcourt Brace College Publishers.

Kelley, K. (2007). Confidence intervals for standardized effect sizes: Theory, application, and implementation. Journal of Statistical Software, 20(8), 1--24. tools:::Rd_expr_doi("10.18637/jss.v020.i08")

Kelley, K., & Preacher, K. J. (2012). On effect size. Psychological Methods, 17, 137--152. tools:::Rd_expr_doi("10.1037/a0028086")

Keppel, G. (1991). Design and analysis: A researcher's handbook (3rd ed.). Prentice Hall.

Maxwell, S. E., Delaney, H. D., & Kelley, K. (2027). Designing experiments and analyzing data: A model comparison perspective (4th ed.). Routledge. (See Chapter 3 on $\eta^2$, Chapter 7 on factorial designs, and Chapter 11 on generalized $\eta^2$ for within-subjects designs.)

Olejnik, S., & Algina, J. (2003). Generalized eta and omega squared statistics: Measures of effect size for some common research designs. Psychological Methods, 8(4), 434--447. tools:::Rd_expr_doi("10.1037/1082-989X.8.4.434")

Steiger, J. H. (2004). Beyond the F test: Effect size confidence intervals and tests of close fit in the analysis of variance and contrast analysis. Psychological Methods, 9(2), 164--182. tools:::Rd_expr_doi("10.1037/1082-989X.9.2.164")

Examples

Run this code

# 1. Raw-argument interface. Bargman's (1970) example, also used in
#        Venables (1975), Fleishman (1980), and Steiger (2004): a 5-group
#        one-way ANOVA with 11 subjects per group, observed F = 11.221.
ci_omega_squared(F_value = 11.221, df_effect = 4, df_error = 50, N = 55)

# Same example with a 90% confidence interval.
ci_omega_squared(
  F_value = 11.221, df_effect = 4, df_error = 50, N = 55,
  conf_level = 0.90
)

# 2. One way ANOVA from a fitted model: mean IQ gain differs across
#        the six grades of the pygmalion data (N = 310).
fit_one <- aov(iq_gain ~ factor(grade), data = pygmalion)
ci_omega_squared(fit_one)

# 3. Two-factor ANOVA: partial omega squared per effect for the
#        manipulated expectancy treatment and the measured grade
#        classification (pygmalion data, N = 310). The treatment by
#        grade interaction is weak here (F = 1.19), so the additive
#        model is used.
fit_additive <- aov(iq_8 ~ treatment + factor(grade), data = pygmalion)
ci_omega_squared(fit_additive)

Run the code above in your browser using DataLab