emmeans (version 1.4.8)

eff_size: Calculate effect sizes and confidence bounds thereof

Description

Standardized effect sizes are typically calculated using pairwise differences of estimates, divided by the SD of the population providing the context for those effects. This function calculates effect sizes from an emmGrid object, and confidence intervals for them, accounting for uncertainty in both the estimated effects and the population SD.

Usage

eff_size(object, sigma, edf, method = "pairwise", ...)

Arguments

object

an emmGrid object, typically one defining the EMMs to be contrasted. If instead, class(object) == "emm_list", such as is produced by emmeans(model, pairwise ~ treatment), a message is displayed; the contrasts already therein are used; and method is replaced by "identity".

sigma

numeric scalar, value of the population SD.

edf

numeric scalar that specifies the equivalent degrees of freedom for the sigma. This is a way of specifying the uncertainty in sigma, in that we regard our estimate of sigma^2 as being proportional to a chi-square random variable with edf degrees of freedom. (edf should not be confused with the df argument that may be passed via ... to specify the degrees of freedom to use in \(t\) statistics and confidence intervals.)

method

the contrast method to use to define the effects. This is passed to contrast after the elements of object are scaled.

...

Additional arguments passed to contrast

Value

an emmGrid object containing the effect sizes

Computation

This function uses calls to regrid to put the estimated marginal means (EMMs) on the log scale. Then an extra element is added to this grid for the log of sigma and its standard error (where we assume that sigma is uncorrelated with the log EMMs). Then a call to contrast subtracts log{sigma} from each of the log EMMs, yielding values of log(EMM/sigma). Finally, the results are re-gridded back to the original scale and the desired contrasts are computed using method. In the log-scaling part, we actually rescale the absolute values and keep track of the signs.

Details

Any by variables specified in object will remain in force in the returned effects, unless overridden in the optional arguments.

For models having a single random effect, such as those fitted using lm; in that case, the stats::sigma and stats::df.residual functions may be useful for specifying sigma and edf. For models with more than one random effect, sigma may be based on some combination of the random-effect variances.

Specifying edf can be rather unintuitive but is also relatively uncritical; but the smaller the value, the wider the confidence intervals for effect size. The value of sqrt(2/edf) can be interpreted as the relative accuracy of sigma; for example, with edf = 50, \(\sqrt(2/50) = 0.2\), meaning that sigma is accurate to plus or minus 20 percent. Note in an example below, we tried two different edf values as kind of a bracketing/sensitivity-analysis strategy. A value of Inf is allowable, in which case you are assuming that sigma is known exactly. Obviously, this narrows the confidence intervals for the effect sizes -- unrealistically if in fact sigma is unknown.

Examples

# NOT RUN {
fiber.lm <- lm(strength ~ diameter + machine, data = fiber)

emm <- emmeans(fiber.lm, "machine")
eff_size(emm, sigma = sigma(fiber.lm), edf = df.residual(fiber.lm))

# or equivalently:
eff_size(pairs(emm), sigma(fiber.lm), df.residual(fiber.lm), method = "identity")


### Mixed model example:
if (require(nlme)) {

  Oats.lme <- lme(yield ~ Variety + factor(nitro), 
                  random = ~ 1 | Block / Variety,
                  data = Oats)
                  
  # Combine variance estimates
  VarCorr(Oats.lme)
  totSD <- sqrt(214.4724 + 109.6931 + 162.5590)
  # I figure edf is somewhere between 5 (Blocks df) and 51 (Resid df)
  
  emmV <- emmeans(Oats.lme, ~ Variety)
  print(eff_size(emmV, sigma = totSD, edf = 5))
  print(eff_size(emmV, sigma = totSD, edf = 51))
}

# Multivariate model for the same data:
 MOats.lm <- lm(yield ~ Variety, data = MOats)
 eff_size(emmeans(MOats.lm, "Variety"), 
          sigma = sqrt(mean(sigma(MOats.lm)^2)),   # RMS of sigma()
          edf = df.residual(MOats.lm))

# These results illustrate a sobering message that effect sizes are often
# not nearly as accurate as you may think.
# }