np.reg.test: Nonparametric Tests of Regression Coefficients

Description

Assuming a linear model of the form $$Y = \alpha + X \beta + \epsilon$$ or $$Y = \alpha + X \beta + Z \gamma + \epsilon$$ this function implements permutation tests of $H_0: \beta = \beta_0$ where $\beta_0$ is the user-specified null vector.

Usage

np.reg.test(x, y, z = NULL, method = NULL,
            beta = NULL, homosced = FALSE, lambda = 0, 
            R = 9999, parallel = FALSE, cl = NULL,
            perm.dist = TRUE, na.rm = TRUE)

Value

statistic: Test statistic value.
p.value: p-value for testing $H_0: \beta = \beta_0$.
perm.dist: Permutation distribution of statistic.
method: Permutation method.
null.value: Null hypothesis value for $\beta$.
homosced: Homoscedastic errors?
lambda: Ridge parameters.
R: Number of resamples.
exact: Exact permutation test? See Note.
coefficients: Least squares estimates of $\alpha$, $\beta$, and $\gamma$ (if applicable).
univariate: Univariate test statistic value for $j$-th variable (for multivariate inputs).
adj.p.value: Adjusted p-value for testing significance of $j$-th variable (for multivariate inputs).

Arguments

x: Matrix of predictor variables (n by p).
y: Response vector or matrix (n by m).
z: Optional matrix of nuisance variables (n by q).
method: Permutation method. See Details.
beta: Null hypothesis value for $\beta$ (p by m). Defaults to matrix of zeros.
homosced: Are the $\epsilon$ terms homoscedastic? If FALSE (default), a robust Wald test statistic is used. Otherwise the classic $F$ test statistic is used.
lambda: Scalar or vector of ridge parameter(s). Defaults to vector of zeros.
R: Number of resamples for the permutation test (positive integer).
parallel: Logical indicating if the parallel package should be used for parallel computing (of the permutation distribution). Defaults to FALSE, which implements sequential computing.
cl: Cluster for parallel computing, which is used when parallel = TRUE. Note that if parallel = TRUE and cl = NULL, then the cluster is defined as makeCluster(2L) to use two cores. To make use of all available cores, use the code cl = makeCluster(detectCores()).
perm.dist: Logical indicating if the permutation distribution should be returned.
na.rm: If TRUE (default), the arguments x and y (and z if provided) are passed to the na.omit function to remove cases with missing data.

Author

Nathaniel E. Helwig <helwig@umn.edu>

Multivariate Tests

If the input y is a matrix with $m > 1$ columns, the multivariate test statistic is defined as statistic = max(univariate) given that the univariate test statistics are non-negative.

The global null hypothesis (across all $m$ variables) is tested by comparing the observed statistic to the permutation distribution perm.dist. This produces the p.value for testing the global null hypothesis.

The local null hypothesis (separately for each variable) is tested by comparing the univariate test statistic to perm.dist. This produces the adjusted p-values (adj.p.values), which control the familywise Type I error rate across the $m$ tests.

Details

With no nuisance variables in the model (i.e., z = NULL), there are three possible options for the method argument:

Method	Model	perm
$P Y = \alpha + X \beta + \epsilon$	flip	$S Y = \alpha + X \beta + \epsilon$

where $P$ is a permutation matrix and $S$ is a sign-flipping matrix.

With nuisance variables in the model, there are eight possible options for the method argument:

Method	Name	Model
HJ	Huh-Jhun	$P Q' R_z Y = \alpha + Q' R_z X \beta + \epsilon$
KC	Kennedy-Cade	$P R_z Y = \alpha + R_z X \beta + \epsilon$
SW	Still-White	$P R_z Y = \alpha + X \beta + \epsilon$
TB	ter Braak	$(P R_m + H_m) Y = \alpha + X \beta + Z \gamma + \epsilon$
FL	Freedman-Lane	$(P R_z + H_z) Y = \alpha + X \beta + Z \gamma + \epsilon$
MA	Manly	$P Y = \alpha + X \beta + Z \gamma + \epsilon$
OS	O'Gorman-Smith	$Y = \alpha + P R_z X \beta + Z \gamma + \epsilon$
DS	Draper-Stoneman	$Y = \alpha + P X \beta + Z \gamma + \epsilon$

where $P$ is permutation matrix and $Q$ is defined as $R_z = Q Q'$ with $Q'Q = I$.

Note that $H_z$ is the hat matrix for the nuisance variable design matrix, and $R_z = I - H_z$ is the corresponding residual forming matrix. Similarly, $H_m$ and $R_m$ are the hat and residual forming matrices for the full model including the predictor and nuisance variables.

References

DiCiccio, C. J., & Romano, J. P. (2017). Robust permutation tests for correlation and regression coefficients. Journal of the American Statistical Association, 112(519), 1211-1220. doi: 10.1080/01621459.2016.1202117

Draper, N. R., & Stoneman, D. M. (1966). Testing for the inclusion of variables in linear regression by a randomisation technique. Technometrics, 8(4), 695-699. doi: 10.2307/1266641

Freedman, D., & Lane, D. (1983). A nonstochastic interpretation of reported significance levels. Journal of Business and Economic Statistics, 1(4), 292-298. doi: 10.2307/1391660

Helwig, N. E. (2019a). Statistical nonparametric mapping: Multivariate permutation tests for location, correlation, and regression problems in neuroimaging. WIREs Computational Statistics, 11(2), e1457. doi: 10.1002/wics.1457

Helwig, N. E. (2019b). Robust nonparametric tests of general linear model coefficients: A comparison of permutation methods and test statistics. NeuroImage, 201, 116030. doi: 10.1016/j.neuroimage.2019.116030

Huh, M.-H., & Jhun, M. (2001). Random permutation testing in multiple linear regression. Communications in Statistics - Theory and Methods, 30(10), 2023-2032. doi: 10.1081/STA-100106060

Kennedy, P. E., & Cade, B. S. (1996). Randomization tests for multiple regression. Communications in Statistics - Simulation and Computation, 25(4), 923-936. doi: 10.1080/03610919608813350

Manly, B. (1986). Randomization and regression methods for testing for associations with geographical, environmental and biological distances between populations. Researches on Population Ecology, 28(2), 201-218. doi: 10.1007/BF02515450

Nichols, T. E., Ridgway, G. R., Webster, M. G., & Smith, S. M. (2008). GLM permutation: nonparametric inference for arbitrary general linear models. NeuroImage, 41(S1), S72.

O'Gorman, T. W. (2005). The performance of randomization tests that use permutations of independent variables. Communications in Statistics - Simulation and Computation, 34(4), 895-908. doi: 10.1080/03610910500308230

Still, A. W., & White, A. P. (1981). The approximate randomization test as an alternative to the F test in analysis of variance. British Journal of Mathematical and Statistical Psychology, 34(2), 243-252. doi: 10.1111/j.2044-8317.1981.tb00634.x

ter Braak, C. J. F. (1992). Permutation versus bootstrap significance tests in multiple regression and ANOVA. In K. H. Jockel, G. Rothe, & W. Sendler (Eds.), Bootstrapping and related techniques. Lecture notes in economics and mathematical systems, vol 376 (pp. 79-86). Springer.

White, H. (1980). A heteroscedasticity-consistent covariance matrix and a direct test for heteroscedasticity. Econometrica, 48(4), 817-838. doi: 10.2307/1912934

Winkler, A. M., Ridgway, G. R., Webster, M. A., Smith, S. M., & Nichols, T. E. (2014). Permutation inference for the general linear model. NeuroImage, 92, 381-397. doi: 10.1016/j.neuroimage.2014.01.060

Examples

Run this code


######******######   UNIVARIATE   ######******######

###***###   TEST ALL COEFFICIENTS   ###***###

# generate data
set.seed(1)
n <- 10
x <- cbind(rnorm(n), rnorm(n))
y <- rnorm(n)

# Wald test (method = "perm")
set.seed(0)
np.reg.test(x, y)

# F test (method = "perm")
set.seed(0)
np.reg.test(x, y, homosced = TRUE)


###***###   TEST SUBSET OF COEFFICIENTS   ###***###

# generate data
set.seed(1)
n <- 10
x <- rnorm(n)
z <- rnorm(n)
y <- 3 + 2 * z + rnorm(n)

# Wald test (method = "HJ")
set.seed(0)
np.reg.test(x, y, z)

# F test (method = "HJ")
set.seed(0)
np.reg.test(x, y, z, homosced = TRUE)


if (FALSE) {

######******######   MULTIVARIATE   ######******######

###***###   TEST ALL COEFFICIENTS   ###***###

# generate data
set.seed(1)
n <- 10
x <- cbind(rnorm(n), rnorm(n))
y <- matrix(rnorm(n * 3), nrow = n, ncol = 3)

# multivariate Wald test (method = "perm")
set.seed(0)
np.reg.test(x, y)

# multivariate F test (method = "perm")
set.seed(0)
np.reg.test(x, y, homosced = TRUE)


###***###   TEST SUBSET OF COEFFICIENTS   ###***###

# generate data
set.seed(1)
n <- 10
x <- rnorm(n)
z <- rnorm(n)
y <- cbind(1 + 3 * z + rnorm(n),
           2 + 2 * z + rnorm(n),
           3 + 1 * z + rnorm(n))
           
# multivariate Wald test (method = "HJ")
set.seed(0)
np.reg.test(x, y, z)

# multivariate F test (method = "HJ")
set.seed(0)
np.reg.test(x, y, z, homosced = TRUE)

}

Run the code above in your browser using DataLab