cliff_delta: Cliff's $\delta$ Ordinal Effect Size

Description

Computes Cliff's (1993) $\delta$ statistic for two independent groups, the difference between the probability that a randomly drawn observation from group 1 exceeds one from group 2 and the reverse probability, together with an analytic confidence interval built from the U-statistic variance (Cliff, 1996). Most R implementations of Cliff's $\delta$ fall back to a bootstrap CI; the analytic CI here is faster, deterministic, and exact in the large-sample limit.

Usage

cliff_delta(group_1, group_2, conf_level = 0.95)

Value

A data.frame with rows for the point estimate

cliff_delta and the lower/upper CI bounds. The output also reports the proportion of pairs with $y_1 > y_2$, the proportion with $y_1 < y_2$, and the proportion of ties.

Arguments

group_1, group_2: Numeric vectors of observations in the two groups. Ordinal data are fine; the statistic uses only ranks.
conf_level: Confidence level for the CI. Default 0.95.

Author

Ken Kelley [email protected]

Details

Definition. Cliff's $\delta$ is $$\delta \;=\; \Pr(Y_1 > Y_2) - \Pr(Y_1 < Y_2) \;=\; 2 \cdot A - 1,$$ where $A$ is the Vargha-Delaney (2000) statistic. The sample estimator is $\hat\delta = (\#\{(i,j): y_{1i} > y_{2j}\} - \#\{(i,j): y_{1i} < y_{2j}\}) / (n_1 n_2)$. Ties contribute zero to both counts. $\delta$ ranges over $[-1, 1]$, with 0 indicating no stochastic dominance.

Analytic CI. The asymptotic variance of $\hat\delta$ is (Cliff, 1993; restated as Feng & Cliff, 2004, Equation 2, p. 323) $$\mathrm{Var}(\hat\delta) \;=\; \frac{(n_2 - 1) \sigma^2_{d_1} + (n_1 - 1) \sigma^2_{d_2} + \sigma^2_d}{n_1 n_2},$$ where $\sigma^2_{d_i}$ is the variance of the per-observation dominance scores within each group. (Feng & Cliff's printed equation transposes the $(n_1 - 1)$ and $(n_2 - 1)$ coefficients, which matters only for unequal group sizes; the pairing above is the correct one, checked by simulation against the empirical variance of $\hat\delta$.) The CI is constructed on the Fisher-style $\mathrm{arctanh}$-transformed scale and back-transformed to respect the bounded range of $\delta$ (analogous to Fisher's Z CI for Pearson $r$). Feng & Cliff (2004, Equation 5, p. 324) recommend an alternative asymmetric interval that models the dependence of the variance on $\delta$; the two constructions agree to first order.

Connection to other measures. Cliff's $\delta$ is a linear transformation of the Vargha-Delaney (2000) $A$ statistic ($\delta = 2A - 1$) and of the Mann-Whitney $U$ statistic ($U / (n_1 n_2) = A$). It is the ordinal analog of the common-language effect size cles and is preferable when bivariate normality is implausible (skewed outcomes, ordinal scales).

References

Cliff, N. (1993). Dominance statistics: Ordinal analyses to answer ordinal questions. Psychological Bulletin, 114(3), 494--509. tools:::Rd_expr_doi("10.1037/0033-2909.114.3.494")

Cliff, N. (1996). Ordinal methods for behavioral data analysis. Lawrence Erlbaum.

Feng, D., & Cliff, N. (2004). Monte Carlo evaluation of ordinal d with improved confidence interval. Journal of Modern Applied Statistical Methods, 3(2), 322--332. tools:::Rd_expr_doi("10.22237/jmasm/1099267560")

Long, J. D., Feng, D., & Cliff, N. (2003). Ordinal analysis of behavioral data. In I. B. Weiner (Ed.), Handbook of psychology, Vol. 2: Research methods (pp.\ 635--661). Wiley.

Vargha, A., & Delaney, H. D. (2000). A critique and improvement of the CL common language effect size statistics of McGraw and Wong. Journal of Educational and Behavioral Statistics, 25(2), 101--132. tools:::Rd_expr_doi("10.3102/10769986025002101")

Examples

Run this code

# 1. Two groups of different sizes, no ties:
set.seed(113)
a <- rnorm(30, mean = 0, sd = 1)
b <- rnorm(40, mean = 0.5, sd = 1)
cliff_delta(a, b)

# 2. With ties (ordinal data):
o1 <- c(1, 2, 2, 3, 3, 3, 4, 4, 5)
o2 <- c(2, 3, 3, 4, 4, 5, 5, 5)
cliff_delta(o1, o2)

# 3. Robust to right skew. Cliff's delta on the raw, untransformed
#    drinking outcome from the Smith, Meyers, and Delaney (1998)
#    trial, comparing the Community Reinforcement Approach (CRA)
#    against standard care. Because the statistic uses only ranks it
#    needs no normalizing transformation of the heavily skewed
#    outcome, unlike the standardized mean difference.
data(drinks_trial)
cra <- drinks_trial$drinks_per_week[drinks_trial$treatment == "CRA"]
std <- drinks_trial$drinks_per_week[drinks_trial$treatment == "Standard"]
cliff_delta(cra, std)

Run the code above in your browser using DataLab

cliff_delta: Cliff's \(\delta\) Ordinal Effect Size