Learn R Programming

LearnNonparam

Overview

This R package implements several non-parametric tests in chapters 1-5 of Higgins (2004), including tests for one sample, two samples, k samples, paired comparisons, blocked designs, trends and association. Built with Rcpp for efficiency and R6 for flexible, object-oriented design, it provides a unified framework for performing or creating custom permutation tests.

Installation

Install the stable version from CRAN:

install.packages("LearnNonparam")

Install the development version from Github:

# install.packages("remotes")
remotes::install_github("qddyy/LearnNonparam")

Usage

library(LearnNonparam)
  • Construct a test object

    • from some R6 class directly
    t <- Wilcoxon$new(n_permu = 1e6)
    • using the pmt (permutation test) wrapper
    # recommended for a unified API
    t <- pmt("twosample.wilcoxon", n_permu = 1e6)
  • Provide it with samples

    set.seed(-1)
    
    t$test(rnorm(10, 1), rnorm(10, 0))
  • Check the results

    t$statistic
    t$p_value
    options(digits = 3)
    
    t$print()
    ggplot2::theme_set(ggplot2::theme_minimal())
    
    t$plot(style = "ggplot2", binwidth = 1)
  • Modify some settings and observe the change

    t$type <- "asymp"
    t$p_value
keyclasstest
onesample.quantileQuantileQuantile Test
onesample.cdfCDFInference on Cumulative Distribution Function
twosample.differenceDifferenceTwo-Sample Test Based on Mean or Median
twosample.wilcoxonWilcoxonTwo-Sample Wilcoxon Test
twosample.scoresumScoreSumTwo-Sample Test Based on Sum of Scores
twosample.ansariAnsariBradleyAnsari-Bradley Test
twosample.siegelSiegelTukeySiegel-Tukey Test
twosample.rmdRatioMeanDevianceRatio Mean Deviance Test
twosample.ksKolmogorovSmirnovTwo-Sample Kolmogorov-Smirnov Test
ksample.onewayOneWayOne-Way Test for Equal Means
ksample.kwKruskalWallisKruskal-Wallis Test
ksample.jtJonckheereTerpstraJonckheere-Terpstra Test
multcomp.studentizedStudentizedMultiple Comparison Based on Studentized Statistic
paired.signSignTwo-Sample Sign Test
paired.differencePairedDifferencePaired Comparison Based on Differences
rcbd.onewayRCBDOneWayOne-Way Test for Equal Means in RCBD
rcbd.friedmanFriedmanFriedman Test
rcbd.pagePagePage Test
association.corrCorrelationTest for Association Between Paired Samples
table.chisqChiSquareChi-Square Test on Contingency Table

Extending

define_pmt allows users to define new permutation tests. Take the two-sample Wilcoxon test as an example:

t_custom <- define_pmt(
    # this is a two-sample permutation test
    inherit = "twosample",
    statistic = function(x, y) {
        # (optional) pre-calculate certain constants that remain invariant during permutation
        m <- length(x)
        n <- length(y)
        # return a closure to calculate the test statistic
        function(x, y) sum(x) / m - sum(y) / n
    },
    # reject the null hypothesis when the test statistic is too large or too small
    rejection = "lr", n_permu = 1e5
)

Also, the statistic can be written in C++. Leveraging Rcpp sugars and C++14 features, only minor modifications are needed to make it compatible with C++ syntax.

t_cpp <- define_pmt(
    inherit = "twosample", rejection = "lr", n_permu = 1e5,
    statistic = "[](const auto& x, const auto& y) {
        auto m = x.length();
        auto n = y.length();
        return [=](const auto& x, const auto& y) {
            return sum(x) / m - sum(y) / n;
        };
    }"
)

It’s easy to check that t_custom and t_cpp are equivalent:

x <- rnorm(10, mean = 0)
y <- rnorm(10, mean = 5)
set.seed(0)
t_custom$test(x, y)$print()
set.seed(0)
t_cpp$test(x, y)$print()

Performance

coin is a commonly used R package for performing permutation tests. Below is a benchmark:

library(coin)

data <- c(x, y)
group <- factor(c(rep("x", length(x)), rep("y", length(y))))

options(LearnNonparam.pmt_progress = FALSE)
benchmark <- microbenchmark::microbenchmark(
    R = t_custom$test(x, y),
    Rcpp = t_cpp$test(x, y),
    coin = wilcox_test(data ~ group, distribution = approximate(nresample = 1e5, parallel = "no"))
)
benchmark

It can be seen that C++ brings significantly better performance than pure R, even surpassing the coin package (under sequential execution). However, all tests in this package are currently written in R with no plans for migration to C++ in the future. This is because the primary goal of this package is not to maximize performance but to offer a flexible framework for permutation tests.

References

Higgins, J. J. 2004. An Introduction to Modern Nonparametric Statistics. Duxbury Advanced Series. Brooks/Cole.

Copy Link

Version

Install

install.packages('LearnNonparam')

Monthly Downloads

249

Version

1.2.8

License

GPL (>= 2)

Issues

Pull Requests

Stars

Forks

Maintainer

Yan Du

Last Published

April 29th, 2025

Functions in LearnNonparam (1.2.8)

RatioMeanDeviance

Ratio Mean Deviance Test
Table2.3.1

Runoff Minutes
PermuTest

PermuTest Class
OneWay

One-Way Test for Equal Means
MultipleComparison

MultipleComparison Class
Quantile

Quantile Test
Table2.6.1

Hours Until Recharge
Table3.2.3

Saltiness Scores
Table3.3.1

Percentages of Clay
RCBDTest

RCBDTest Class
OneSampleTest

OneSampleTest Class
Page

Page Test
SiegelTukey

Siegel-Tukey Test
ScoreSum

Two-Sample Test Based on Sum of Scores
Wilcoxon

Two-Sample Wilcoxon Test
PairedDifference

Paired Comparison Based on Differences
Studentized

Multiple Comparison Based on Studentized Statistic
Table1.2.1

Cycles Until Failure
Table2.1.1

Test Scores
Table4.5.3

Randomized Complete Block with Ties
Table4.4.3

Yield Data
Table4.1.3

Cholesterol Reduction
pmt

Syntactic Sugar for Object Construction
Table1.1.1

Sodium Contents
TwoSampleAssociationTest

TwoSampleAssociationTest Class
Table3.1.2

Normal Samples
Table3.4.1

Phosphorus Contents
Table3.2.2

Logarithms of Bacteria Counts
Table4.1.1

Caloric Intake
Table5.1.2

Heterophils and Lymphocytes
TwoSamplePairedTest

TwoSamplePairedTest Class
Table2.8.1

Ounces Of Beverage
TwoSampleTest

TwoSampleTest Class
Sign

Two-Sample Sign Test
Table5.2.2

Scores of Projects
Table5.4.2

Satisfaction with Pain-Relief Treatment
Table2.6.2

Cerium Amounts
TwoSampleLocationTest

TwoSampleLocationTest Class
ContingencyTableTest

ContingencyTableTest Class
Difference

Two-Sample Test Based on Mean or Median
AnsariBradley

Ansari-Bradley Test
ChiSquare

Chi-Square Test on Contingency Table
CDF

Inference on Cumulative Distribution Function
KolmogorovSmirnov

Two-Sample Kolmogorov-Smirnov Test
Correlation

Test for Association Between Paired Samples
KSampleTest

KSampleTest Class
Friedman

Friedman Test
JonckheereTerpstra

Jonckheere-Terpstra Test
RCBDOneWay

One-Way Test for Equal Means in RCBD
KruskalWallis

Kruskal-Wallis Test