Let \((X_k,Y_k), k=1,\ldots,n\), be complete paired observations.
Chatterjee's rank correlation is directed: \(\xi(X,Y)\) measures how well
\(Y\) is functionally determined by \(X\). It is not a symmetric
association measure, so in matrix mode
$$\mathrm{result}_{ij} = \xi(X_{\cdot i}, X_{\cdot j})$$
need not equal \(\mathrm{result}_{ji}\). The row variable is always the
sorting/predictor variable and the column variable is the response/ranked
variable.
For a given directed pair, observations are sorted by \(X\). Let
\(r_i\) be the number of response values \(Y_j \le Y_i\), and let
\(l_i\) be the number of response values \(Y_j \ge Y_i\), evaluated for
the \(i\)-th observation in sorted-by-\(X\) order. The tied-response
finite-sample estimator is
$$
\xi_n = 1 -
\frac{n \sum_{i=1}^{n-1} |r_{i+1} - r_i|}
{2 \sum_{i=1}^n l_i (n - l_i)}.
$$
If all response values are equal, the denominator is zero and the estimate
is NA. When there are no ties in \(Y\), this reduces to the familiar
rank-difference expression
$$
\xi_n = 1 -
\frac{3 \sum_{i=1}^{n-1}|r_{i+1}-r_i|}{n^2 - 1}.
$$
The default bias_correction = "none" returns Chatterjee's original
finite-sample estimator. This raw statistic is not forced to attain one
under perfect deterministic dependence. In continuous data without ties, its
finite-sample upper bound is
$$
\xi_{\max} = \xi_n(Y,Y) = \frac{n - 2}{n + 1}.
$$
For example, when \(n = 20\), the largest possible raw value is
\(18 / 21 \approx 0.857\). This is a property of the estimator rather
than an implementation error. Use
bias_correction = "upper_bound" when you want the bias-reduced
Dalitz-Arning-Goebbels normalisation, which rescales by \(\xi_n(Y,Y)\).
Ties in the sorting variable \(X\) are handled before computing adjacent
rank differences. Chatterjee's definition uses random tie breaking, provided
here by tie_method = "random". For reproducible diagnostics or when
the input order should define tied-\(X\) ordering, use
tie_method = "first". Ties in the response variable \(Y\) are
handled by the general \(r_i,l_i\) formula above and do not require random
tie breaking.
Confidence intervals. The ordinary n-out-of-n bootstrap is not
used for Chatterjee's coefficient. Both available intervals use
m-out-of-n subsampling without replacement, consistent with the bootstrap
family considered for Chatterjee's rank correlation.
With ci_method = "dette_kroll", subsamples of size \(m\) are drawn
without replacement and the limiting standard deviation is estimated as
$$\widehat{\sigma} = \sqrt{m}\,\mathrm{sd}(\xi_m^*).$$
The reported interval is the normal interval
$$
\xi_n \pm z_{1-\alpha/2}\widehat{\sigma}/\sqrt{n},
\quad \alpha = 1-\texttt{conf\_level}.
$$
If m = NULL, this method uses \(m=\lfloor\sqrt{n}\rfloor\), bounded
to \([2,n-1]\).
With ci_method = "n_choose_m", subsamples are also drawn without
replacement, but the interval is obtained by inverting the empirical
quantiles of the centred and scaled statistic
$$T^* = \sqrt{m}(\xi_m^* - \xi_n).$$
If \(q_{\alpha/2}\) and \(q_{1-\alpha/2}\) are the bootstrap quantiles,
the basic m-out-of-n limits are
$$
\xi_n - q_{1-\alpha/2}/\sqrt{n}
\quad\text{and}\quad
\xi_n - q_{\alpha/2}/\sqrt{n}.
$$
If finite Monte Carlo quantiles fall entirely on one side of zero, the
inversion is anchored at zero so that the reported interval contains the
observed estimate; the bounds are not clipped to the population parameter
range. If m = NULL, this method uses
\(m=\mathrm{round}(2\sqrt{n})\), bounded to \([2,n-1]\), following the
implementation rule used by Dalitz, Arning and Goebbels.
Which CI method should be used? The Dette-Kroll interval is the
more conservative default for small to moderate complete-case sample sizes
in this implementation, because it uses the m-out-of-n bootstrap only to
estimate a normal-approximation standard error. The non-parametric
n-choose-m interval is useful for larger samples when a direct
m-out-of-n bootstrap interval is preferred, but Dalitz, Arning and Goebbels
report that this interval may need fairly large \(n\) to approach nominal
coverage in some settings. Therefore ci_method = "auto" uses
"dette_kroll" when pair-specific n_complete <=
large_sample_cutoff and "n_choose_m" when
n_complete > large_sample_cutoff. Increase
bootstrap_reps for final analyses to reduce Monte Carlo error, and
consider setting m explicitly when a study protocol requires a
fixed subsample size.
Computation. For complete finite matrices without confidence
intervals, the C++ backend precomputes each column's sorting order and
response ranks and reuses them across directed pairs. This costs
\(O(p\,n\log n + p^2 n)\) time and \(O(pn + p^2)\) memory. Pairwise
missing-data evaluation and bootstrap confidence intervals recompute on the
relevant complete-case samples and are correspondingly more expensive.