metafor (version 1.0-1)

permutest.rma.uni: Permutation Tests for rma.uni Objects

Description

The function carries out permutation tests for objects of class "rma.uni".

Usage

## S3 method for class 'rma.uni':
permutest(x, exact=FALSE, iter=1000, progbar=TRUE, digits=x$digits, \dots)

Arguments

x

an object of class "rma.uni".

exact

logical indicating whether an exact permutation test should be carried out or not (default is FALSE). See Details.

iter

integer to specify the number of iterations for the permutation test when not doing an exact test (default is 1000 iterations).

progbar

logical indicating whether a progress bar should be shown (default is TRUE).

digits

an integer specifying the number of decimal places to which the printed results should be rounded (the default is to take the value from the object).

...

other arguments.

Value

• An object of class "permutest.rma.uni". The object is a list containing the following components:
• pval
  p-value(s) based on the permutation test.
• QMp
  p-value for the omnibus test of coefficients based on the permutation test.
• ...
  some additional elements/values are passed on.
• The results are formated and printed with the print.permutest.rma.uni function.

Details

For models without moderators, the permutation test is carried out by permuting the signs of the observed effect sizes or outcomes. The p-value of the permutation test is then the proportion of times that the test statistic under the permuted data is more extreme than under the actually observed data. See Follmann and Proschan (1999) for more details. For models with moderators, the permutation test is carried out by permuting the rows of the design matrix. The p-value for a particular model coefficient is then equal to the proportion of times that the test statistic for the coefficient under the permuted data is more extreme than under the actually observed data. Similarly, for the omnibus test, the p-value is the proportion of times that the test statistic for the omnibus test is more extreme than the actually observed one. See Higgins and Thompson (2004) for more details. If exact=TRUE, the function will try to carry out an exact permutation test. An exact permutation test requires fitting the model to each possible permutation once. However, the number of possible permutations increases rapidly with the number of outcomes/studies (i.e., $k$), especially with respect to the possible number of permutations of the design matrix. For example, for $k=5$, there are only 120 possible permutations of the design matrix (32 possible permutations of the signs). For $k=8$, there are already 40,320 (256). And for $k=10$, there are 3,628,800 (1024). Therefore, going through all possible permutations may become infeasible. Instead of using an exact permutation test, one can set exact=FALSE. In that case, the function approximates the exact permutation-based p-value(s) by going through a smaller number (as specified by the iter argument) of random permutations. Therefore, running the function twice on the same data will then yield (slightly) different p-values. Setting iter sufficiently large ensures that the results become stable.

References

Follmann, D. A. & Proschan, M. A. (1999) Valid inference in random effects meta-analysis. Biometrics, 55, 732--737. Higgins, J. P. T. & Thompson, S. G. (2004) Controlling the risk of spurious findings from meta-regression. Statistics in Medicine, 23, 1663--1682.

See Also

rma.uni, print.permutest.rma.uni

Examples

### load BCG vaccine data
data(dat.bcg)

### calculate log relative risks and corresponding sampling variances
dat <- escalc(measure="RR", ai=tpos, bi=tneg, ci=cpos, di=cneg, data=dat.bcg)
dat <- cbind(dat.bcg, dat)

### random-effects model
res <- rma(yi, vi, data=dat, method="REML")

### permutation test (approximate and exact)
permutest(res)
permutest(res, exact=TRUE)

### mixed-effects model with two moderators (absolute latitude and publication year)
res1 <- rma(yi, vi, mods=cbind(ablat, year), data=dat, method="REML")

### permutation test (only approximate; exact not feasible)
permutest(res)