Learn R Programming

LearnNonparam

Overview

This R package implements several non-parametric tests in chapters 1-5 of Higgins (2004), including tests for one sample, two samples, k samples, paired comparisons, blocked designs, trends and association. Built with Rcpp for efficiency and R6 for flexible, object-oriented design, it provides a unified framework for performing or creating custom permutation tests.

Installation

Install the stable version from CRAN:

install.packages("LearnNonparam")

Install the development version from Github:

# install.packages("remotes")
remotes::install_github("qddyy/LearnNonparam")

Usage

library(LearnNonparam)
  • Construct a test object

    • from some R6 class directly
    t <- Wilcoxon$new(n_permu = 1e6)
    • using the pmt (permutation test) wrapper
    # recommended for a unified API
    t <- pmt("twosample.wilcoxon", n_permu = 1e6)
  • Provide it with samples

    set.seed(-1)
    
    t$test(rnorm(10, 1), rnorm(10, 0))
  • Check the results

    t$statistic
    t$p_value
    options(digits = 3)
    
    t$print()
    ggplot2::theme_set(ggplot2::theme_minimal())
    
    t$plot(style = "ggplot2", binwidth = 1) # or ggplot2::autoplot(t, binwidth = 1)
  • Modify some settings and observe the change

    t$type <- "asymp"
    t$p_value

See pmts() for tests implemented in this package.

pmts()
keyclasstest
onesample.quantileQuantileQuantile Test
onesample.cdfCDFInference on Cumulative Distribution Function
twosample.differenceDifferenceTwo-Sample Test Based on Mean or Median
twosample.wilcoxonWilcoxonTwo-Sample Wilcoxon Test
twosample.ansariAnsariBradleyAnsari-Bradley Test
twosample.siegelSiegelTukeySiegel-Tukey Test
twosample.rmdRatioMeanDevianceRatio Mean Deviance Test
distribution.ksKolmogorovSmirnovTwo-Sample Kolmogorov-Smirnov Test
distribution.kuiperKuiperTwo-Sample Kuiper Test
distribution.cvmCramerVonMisesTwo-Sample Cramer-Von Mises Test
distribution.adAndersonDarlingTwo-Sample Anderson-Darling Test
association.corrCorrelationTest for Association Between Paired Samples
paired.signSignTwo-Sample Sign Test
paired.differencePairedDifferencePaired Comparison Based on Differences
ksample.onewayOneWayOne-Way Test for Equal Means
ksample.kwKruskalWallisKruskal-Wallis Test
ksample.jtJonckheereTerpstraJonckheere-Terpstra Test
multcomp.studentizedStudentizedMultiple Comparison Based on Studentized Statistic
rcbd.onewayRCBDOneWayOne-Way Test for Equal Means in RCBD
rcbd.friedmanFriedmanFriedman Test
rcbd.pagePagePage Test
table.chisqChiSquareChi-Square Test on Contingency Table

Extending

define_pmt allows users to define new permutation tests. Take the two-sample Wilcoxon test as an example:

t_custom <- define_pmt(
    # this is a two-sample permutation test
    method = "twosample",
    statistic = function(x, y) {
        # (optional) pre-calculate certain constants that remain invariant during permutation
        m <- length(x)
        n <- length(y)
        # return a closure to calculate the test statistic
        function(x, y) sum(x) / m - sum(y) / n
    },
    # reject the null hypothesis when the test statistic is too large or too small
    rejection = "<>", n_permu = 1e5
)

For R >= 4.4.0, the quickr package can be used to accelerate statistic. However, this results in repeated crossings of the R-Fortran boundary and makes pre-calculation of constants impossible.

t_quickr <- define_pmt(
    method = "twosample", rejection = "<>", n_permu = 1e5,
    statistic = function(x, y) {
        sum(x) / length(x) - sum(y) / length(y)
    },
    quickr = TRUE
)

In cases where both pre-calculation and computational efficiency are required, the statistic can be written in C++. Leveraging Rcpp sugars and C++14 features, only minor modifications are needed to make it compatible with C++ syntax.

t_cpp <- define_pmt(
    method = "twosample", rejection = "<>", n_permu = 1e5,
    statistic = "[](const auto& x, const auto& y) {
        auto m = x.length();
        auto n = y.length();
        return [=](const auto& x, const auto& y) {
            return sum(x) / m - sum(y) / n;
        };
    }"
)

It’s easy to check that t_custom, t_quickr and t_cpp are equivalent:

x <- rnorm(10, mean = 0)
y <- rnorm(10, mean = 5)
set.seed(0)
t_custom$test(x, y)$print()
set.seed(0)
t_quickr$test(x, y)$print()
set.seed(0)
t_cpp$test(x, y)$print()

Performance

coin is a commonly used R package for performing permutation tests. Below is a benchmark:

library(coin)

data <- c(x, y)
group <- factor(c(rep("x", length(x)), rep("y", length(y))))

options(LearnNonparam.pmt_progress = FALSE)
benchmark <- microbenchmark::microbenchmark(
    pure_R = t_custom$test(x, y),
    quickr = t_quickr$test(x, y),
    Rcpp = t_cpp$test(x, y),
    coin = wilcox_test(data ~ group, distribution = approximate(nresample = 1e5, parallel = "no"))
)
benchmark

It can be seen that C++ brings significantly better performance than pure R, which enables it to even surpass the coin package in its no-parallelization setting. However, all tests in this package are currently written in pure R with no plans for migration to C++ in the future. This is because the primary goal of this package is not to maximize performance but to offer a flexible framework for permutation tests.

References

Higgins, J. J. 2004. An Introduction to Modern Nonparametric Statistics. Duxbury Advanced Series. Brooks/Cole.

Copy Link

Version

Install

install.packages('LearnNonparam')

Monthly Downloads

189

Version

1.3.0

License

GPL (>= 2)

Issues

Pull Requests

Stars

Forks

Maintainer

Yan Du

Last Published

September 30th, 2025

Functions in LearnNonparam (1.3.0)

MultipleComparison

MultipleComparison Class
PermuTest

PermuTest Class
KruskalWallis

Kruskal-Wallis Test
KSampleTest

KSampleTest Class
Kuiper

Two-Sample Kuiper Test
Page

Page Test
KolmogorovSmirnov

Two-Sample Kolmogorov-Smirnov Test
PairedDifference

Paired Comparison Based on Differences
OneWay

One-Way Test for Equal Means
OneSampleTest

OneSampleTest Class
Quantile

Quantile Test
SiegelTukey

Siegel-Tukey Test
Table2.1.1

Test Scores
Sign

Two-Sample Sign Test
Studentized

Multiple Comparison Based on Studentized Statistic
RCBDTest

RCBDTest Class
RatioMeanDeviance

Ratio Mean Deviance Test
Table1.1.1

Sodium Contents
Table1.2.1

Cycles Until Failure
Table3.2.3

Saltiness Scores
Table3.3.1

Percentages of Clay
RCBDOneWay

One-Way Test for Equal Means in RCBD
Table4.1.3

Cholesterol Reduction
Table4.4.3

Yield Data
Table2.3.1

Runoff Minutes
Table5.1.2

Heterophils and Lymphocytes
Table3.4.1

Phosphorus Contents
Table4.1.1

Caloric Intake
Table4.5.3

Randomized Complete Block with Ties
Table5.2.2

Scores of Projects
Table2.6.2

Cerium Amounts
Table2.8.1

Ounces Of Beverage
Table5.4.2

Satisfaction with Pain-Relief Treatment
pmt

Syntactic Sugar for Object Construction
TwoSampleAssociationTest

TwoSampleAssociationTest Class
TwoSampleDistributionTest

TwoSampleDistributionTest Class
Wilcoxon

Two-Sample Wilcoxon Test
TwoSampleTest

TwoSampleTest Class
TwoSamplePairedTest

TwoSamplePairedTest Class
Table3.1.2

Normal Samples
TwoSampleLocationTest

TwoSampleLocationTest Class
Table2.6.1

Hours Until Recharge
Table3.2.2

Logarithms of Bacteria Counts
Friedman

Friedman Test
Difference

Two-Sample Test Based on Mean or Median
ChiSquare

Chi-Square Test on Contingency Table
ContingencyTableTest

ContingencyTableTest Class
Correlation

Test for Association Between Paired Samples
JonckheereTerpstra

Jonckheere-Terpstra Test
CramerVonMises

Two-Sample Cramer-Von Mises Test
AndersonDarling

Two-Sample Anderson-Darling Test
AnsariBradley

Ansari-Bradley Test
CDF

Inference on Cumulative Distribution Function