pPerm: Calculate permutation p-value Calculate perutation p-value of actual model performance vs null hypothesis distribution. `pPerm` will calculate the cumulative (1-tailed) probability of `actual` belonging to `permutation_distribution`. `side` is guessed by actual value compared to median(permutation_distribution). Test is performed on original data OR ranked for non-parametric statistics.

Description

Calculate permutation p-value Calculate perutation p-value of actual model performance vs null hypothesis distribution. `pPerm` will calculate the cumulative (1-tailed) probability of `actual` belonging to `permutation_distribution`. `side` is guessed by actual value compared to median(permutation_distribution). Test is performed on original data OR ranked for non-parametric statistics.

Usage

pPerm(
  actual,
  permutation_distribution,
  side = c("smaller", "greater"),
  type = "t",
  extend = 0.1
)

Value

p-value

Arguments

actual: Actual model performance (e.g. misclassifications or Q2)
permutation_distribution: Null hypothesis distribution from permutation test (same metric as `actual`)
side: Smaller or greater than (automatically guessed if omitted) (Q2 and AUC is a "greater than" test, whereas misclassifications is "smaller than")
type: one of ('t','non',"smooth","ecdf","rank")
extend: extend how much it extend

Examples

Run this code

data("freelive2")
actual <- sample(YR2, 1)
permutation_distribution <- YR2
pPerm(actual, permutation_distribution)

Run the code above in your browser using DataLab