dcor: Pairwise Distance Correlation (dCor)

Description

Computes conventional pairwise distance correlation for numeric columns using a high-performance C++ backend. Distance correlation is a nonnegative sample measure of dependence that can capture nonlinear and nonmonotonic relationships. By default, dcor() returns the conventional sample distance correlation $R_n$; set squared = TRUE to return $R_n^2$.

Usage

dcor(
  data,
  squared = FALSE,
  na_method = c("error", "pairwise", "complete"),
  p_value = FALSE,
  n_threads = getOption("matrixCorr.threads", 1L),
  output = c("matrix", "sparse", "edge_list"),
  threshold = 0,
  diag = TRUE,
  ...
)
# S3 method for dcor
print(
  x,
  digits = 4,
  n = NULL,
  topn = NULL,
  max_vars = NULL,
  width = NULL,
  show_ci = NULL,
  ...
)
# S3 method for dcor
plot(
  x,
  title = NULL,
  low_color = "white",
  high_color = "steelblue1",
  value_text_size = 4,
  show_value = TRUE,
  ...
)
# S3 method for dcor
summary(
  object,
  n = NULL,
  topn = NULL,
  max_vars = NULL,
  width = NULL,
  show_ci = NULL,
  ...
)
# S3 method for summary.dcor
print(
  x,
  digits = NULL,
  n = NULL,
  topn = NULL,
  max_vars = NULL,
  width = NULL,
  show_ci = NULL,
  ...
)

Value

A symmetric correlation result where the (i, j) entry is conventional distance correlation $R_n$, or $R_n^2$ when squared = TRUE. When p_value = TRUE, the inference attribute contains bcdcor, statistic, parameter, and p_value; the t-test uses the signed U-centred statistic in inference$bcdcor.

Invisibly returns x.

A ggplot object representing the heatmap.

Arguments

data: A numeric matrix or a data frame with at least two numeric columns. All non-numeric columns are dropped. Columns must be numeric.
squared: Logical; if FALSE (default), return $R_n$. If TRUE, return conventional squared distance correlation $R_n^2$.
na_method: Character scalar controlling missing-data handling. "error" rejects missing, NaN, and infinite values. "pairwise" recomputes each association on its own pairwise complete-case overlap. "complete" performs listwise deletion once across the retained numeric columns.
p_value: Logical (default FALSE). Compatibility option. If TRUE, attach the Székely-Rizzo t-test metadata computed from the signed bias-corrected statistic returned by bcdcor(), not from the conventional $R_n$ values in the main matrix.
n_threads: Integer $\geq 1$. Number of OpenMP threads. Defaults to getOption("matrixCorr.threads", 1L).
output: Output representation for the computed estimates: "matrix", "sparse", or "edge_list".
threshold: Non-negative absolute-value filter for non-matrix outputs: keep entries with abs(value) >= threshold. Must be 0 when output = "matrix".
diag: Logical; whether to include diagonal entries in "sparse" and "edge_list" outputs.
...: Additional arguments passed to ggplot2::theme() or other ggplot2 layers.
x: An object of class summary.dcor.
digits: Integer; number of decimal places to print.
n: Optional row threshold for compact preview output.
topn: Optional number of leading/trailing rows to show when truncated.
max_vars: Optional maximum number of visible columns; NULL derives this from console width.
width: Optional display width; defaults to getOption("width").
show_ci: One of "yes" or "no".
title: Plot title. Default is "Distance correlation heatmap".
low_color: Colour for zero correlation. Default is "white".
high_color: Colour for strong correlation. Default is "steelblue1".
value_text_size: Font size for displaying values. Default is 4.
show_value: Logical; if TRUE (default), overlay numeric values on the heatmap tiles.
object: An object of class dcor.

Difference between <code>dcor()</code> and <code>bcdcor()</code>

dcor() returns conventional nonnegative distance correlation $R_n$. dcor(squared = TRUE) returns conventional $R_n^2$. bcdcor() returns a U-centred bias-corrected squared distance-correlation statistic that may be negative in finite samples and is used by the bias-corrected dCor t-test. In finite samples, bcdcor(x) is not generally equal to dcor(x)^2.

Author

Thiago de Paula Oliveira

Details

For distances $a_{ij}=|x_i-x_j|$, define the conventional doubly centred distances $$A_{ij}=a_{ij}-\bar a_{i\cdot}-\bar a_{\cdot j}+\bar a_{\cdot\cdot}.$$ Define $B_{ij}$ analogously for $y$. The sample squared distance covariance is $$V_n^2(x,y)=\frac{1}{n^2}\sum_{i,j} A_{ij}B_{ij},$$ with $V_n^2(x,x)$ and $V_n^2(y,y)$ defined analogously. The conventional squared distance correlation is $$R_n^2(x,y)= \frac{V_n^2(x,y)}{\sqrt{V_n^2(x,x)V_n^2(y,y)}}.$$ dcor() returns $R_n=\sqrt{R_n^2}$ unless squared = TRUE.

Under the required finite first-moment conditions, population distance correlation is zero if and only if the variables are independent. This population property should not be read as a sample-level test by itself.

References

Szekely, G. J., Rizzo, M. L., & Bakirov, N. K. (2007). Measuring and testing dependence by correlation of distances. Annals of Statistics, 35(6), 2769-2794.

Szekely, G. J., & Rizzo, M. L. (2013). The distance correlation t-test of independence. Journal of Multivariate Analysis, 117, 193-213.

Examples

Run this code

# \donttest{
# Conventional distance correlation
set.seed(42)
n <- 200
x <- rnorm(n)
y <- 0.7 * x + sqrt(1 - 0.7^2) * rnorm(n)
X <- cbind(x = x, y = y)

# dcor() returns conventional distance correlation R.
# squared = TRUE returns conventional R^2.
dcor(X)
dcor(X, squared = TRUE)

D <- dcor(X)
D2 <- dcor(X, squared = TRUE)

D2["x", "y"]
D["x", "y"]^2

# Pearson can be close to zero for this symmetric nonlinear relationship,
# while distance correlation remains positive.
set.seed(123)
n <- 300
x <- rnorm(n)
y <- x^2 + rnorm(n, sd = 0.25)
X <- cbind(x = x, y = y)

cor(x, y)
dcor(X)

# Pairwise distance-correlation matrix across columns
if (requireNamespace("MASS", quietly = TRUE)) {
  set.seed(7)
  p <- 5
  n <- 150
  rho <- 0.6
  Sigma <- rho^abs(outer(seq_len(p), seq_len(p), "-"))

  X <- MASS::mvrnorm(
    n,
    mu = rep(0, p),
    Sigma = Sigma
  )
  colnames(X) <- paste0("V", seq_len(p))

  D <- dcor(X)
  D[1:3, 1:3]
}

# Backward-compatible inference through dcor()
fit <- dcor(cbind(x = x, y = y), p_value = TRUE)
fit

# The displayed matrix contains conventional dCor.
# The attached t-test is based on the signed bias-corrected statistic.
inf <- attr(fit, "inference")
inf$bcdcor
inf$p_value

# Missing-data modes
X_miss <- cbind(
  a = c(1, 2, NA, 4, 5, 6),
  b = c(2, 3, 4, 5, NA, 7),
  c = c(6, 5, 4, 3, 2, 1)
)

dcor(X_miss, na_method = "complete")
dcor(X_miss, na_method = "pairwise")

# Edge-list output for larger matrices
set.seed(11)
X <- matrix(rnorm(500), nrow = 100, ncol = 5)
colnames(X) <- paste0("V", seq_len(ncol(X)))

dcor(
  X,
  output = "edge_list",
  threshold = 0.2,
  diag = FALSE
)
# }

Run the code above in your browser using DataLab