Performs one and two sample permutation t-tests on vectors of data.
perm.t.test(x, ...)# S3 method for default
perm.t.test(x, y = NULL,
alternative = c("two.sided", "less", "greater"),
mu = 0, paired = FALSE, var.equal = FALSE,
conf.type = "pivot", conf.level = 0.95, R = 9999, symmetric = TRUE,
permStat = FALSE, useCombn = FALSE, ...)
# S3 method for formula
perm.t.test(formula, data, subset, na.action, ...)
A list with class "perm.htest" (derived from class htest)
containing the following components:
the value of the t-statistic.
the degrees of freedom for the t-statistic.
the p-value for the test.
the (Monte-Carlo) permutation p-value for the test.
a confidence interval for the mean appropriate to the specified alternative hypothesis.
a (Monte-Carlo) permutation percentile confidence interval for the mean appropriate to the specified alternative hypothesis.
the estimated mean or difference in means depending on whether it was a one-sample test or a two-sample test.
(Monte-Carlo) permutation estimate.
the specified hypothesized value of the mean or mean difference depending on whether it was a one-sample test or a two-sample test.
the standard error of the mean (difference), used as denominator in the t-statistic formula.
(Monte-Carlo) permutation standard error.
a character string describing the alternative hypothesis.
a character string indicating what type of t-test was performed.
a character string giving the name(s) of the data.
NULL or numeric vector with permutation results.
a (non-empty) numeric vector of data values.
an optional (non-empty) numeric vector of data values.
a character string specifying the alternative
hypothesis, must be one of "two.sided" (default),
"greater" or "less". You can specify just the initial
letter.
a number indicating the true value of the mean (or difference in means if you are performing a two sample test).
a logical indicating whether you want a paired t-test.
a logical variable indicating whether to treat the
two variances as being equal. If TRUE then the pooled
variance is used to estimate the variance otherwise the Welch
(or Satterthwaite) approximation to the degrees of freedom is used.
a character string specifying which type of confidence interval to compute; see details below.
confidence level of the interval.
number of (Monte-Carlo) permutations.
a logical variable indicating whether to assume symmetry
in the two-sided test. If TRUE then the symmetric permutation p value
otherwise the equal-tail permutation p value is computed.
a formula of the form lhs ~ rhs where lhs
is a numeric variable giving the data values and rhs a factor
with two levels giving the corresponding groups.
an optional matrix or data frame (or similar: see
model.frame) containing the variables in the
formula formula. By default the variables are taken from
environment(formula).
an optional vector specifying a subset of observations to be used.
a function which indicates what should happen when
the data contain NAs. Defaults to
getOption("na.action").
logical; should the permutation results for the test statistic
be returned. If TRUE, function h0plot can be used to plot the
distribution of the results.
logical; use all combinations instead of permutations in the case of the two-sample t-test; see Details.
further arguments to be passed to or from methods.
The implemented test corresponds to the proposal of Chapter 15 of Efron and Tibshirani (1993) for equal variances as well as Janssen (1997) respectively Chung and Romano (2013) for unequal variances.
Janssen and Pauls (2003) state in Example 5 (c) about Bootstrap two-sample tests: "From the previous comments it is clear that permutation tests should always be preferred for two-sample problems."
The function returns permutation p values and confidence intervals
as well as the results ot the t-test without permutations.
If useCombn is TRUE in the two-sample case, then all combinations
instead of all permutations are used. This option only has an effect for
small sample sizes when the number of all combinations/permutations is smaller
than R. In principle, the results should be identical in this case.
However, the difference in the number of permutations/combinations can have a
minor effect on p values, confidence intervals as well as the distribution of
the test statistic.
The function computes four types of permutation confidence intervals:
pivotPivot-Inverted Test Confidence Interval (Garthwaite (1996), Good (2005)). Constructed by inverting the studentized permutation test while treating the sample permutation distribution as fixed (pivotal). It provides a fast, stable, and smooth approximation of test inversion.
exactExact Inverted Test Confidence Interval (Chung and Romano (2013)). Obtained by exact test inversion where the permutation standard error is dynamically re-computed under each null hypothesis shift. It guarantees exact coverage under the i.i.d. assumption and asymptotic validity under heteroscedasticity, though extreme outliers may result in conservative (wide) bounds.
studStudentized (Bootstrap-t) Permutation Confidence Interval (Hall (1988), Janssen (1997)). Uses the quantiles of the studentized permutation distribution centered at 0 and scales them by the observed standard error.
percPercentile Permutation Confidence Interval (Efron (1981), Efron and Tibshirani (1993)). Derived directly from the empirical quantiles of the unstudentized permutation distribution of the effect size.
allAll types of confidence intervals are computed.
The formula interface is only applicable for the 2-sample tests.
alternative = "greater" is the alternative that x has a
larger mean than y.
If paired is TRUE then both x and y must
be specified and they must be the same length. Missing values are
silently removed (in pairs if paired is TRUE). If
var.equal is TRUE then the pooled estimate of the
variance is used. By default, if var.equal is FALSE
then the variance is estimated separately for both groups and the
Welch modification to the degrees of freedom is used.
If the input data are effectively constant (compared to the larger of the two means) an error is generated.
B. Efron (1981). Nonparametric standard errors and confidence intervals. The Canadian Journal of Statistics, 9(2), 139-158.
B. Efron, R.J. Tibshirani. An Introduction to the Bootstrap. Chapman and Hall/CRC 1993.
P.H. Garthwaite (1996). Confidence intervals from randomization tests. Biometrics, 52(4), 1387-1393.
P. Good (2005). Permutation, Parametric, and Bootstrap Tests of Hypotheses (3rd ed.). Springer.
P. Hall (1986). On the bootstrap and confidence intervals. The Annals of Statistics, 14(4), 1431-1452.
A. Janssen (1997). Studentized permutation tests for non-i.i.d, hypotheses and the generalized Behrens-Fisher problem. Statistics and Probability Letters, 36, 9-21.
A. Janssen, T. Pauls. How do bootstrap and permutation tests work?. Ann. Statist., 31(3), 768-806.
E. Chung, J.P. Romano (2013). Exact and asymptotically robust permutation tests. The Annals of Statistics, 41(2), 484-507.
t.test, meanCI, meanDiffCI,
boot.t.test
require(graphics)
t.test(1:10, y = c(7:20)) # P = .00001855
perm.t.test(1:10, y = c(7:20), conf.type = "all")
t.test(1:10, y = c(7:20, 200)) # P = .1245 -- NOT significant anymore
perm.t.test(1:10, y = c(7:20, 200), conf.type = "pivot")
perm.t.test(1:10, y = c(7:20, 200), conf.type = "exact")
perm.t.test(1:10, y = c(7:20, 200), conf.type = "stud")
perm.t.test(1:10, y = c(7:20, 200), conf.type = "perc") # p < 1e-04, but CI covers 0
## Traditional interface
with(mtcars, t.test(mpg[am == 0], mpg[am == 1]))
with(mtcars, perm.t.test(mpg[am == 0], mpg[am == 1]))
## Formula interface
t.test(mpg ~ am, data = mtcars)
perm.t.test(mpg ~ am, data = mtcars)
## all permutations vs combinations
perm.t.test(1:3, y = 2:5, useCombn = TRUE)
perm.t.test(1:3, y = 2:5, useCombn = FALSE)
Run the code above in your browser using DataLab