cutpointr (version 1.0.2)

boot_test: Test for equivalence of a metric

Description

This function performs a significance test based on the bootstrap results of cutpointr to test whether a chosen metric is equal between subgroups or between two cutpointr objects. The test statistic is calculated as the standardized difference of the metric between groups. If x contains subgroups, the test is run on all possible pairings of subgroups. An additional adjusted p-value is returned in that case.

Usage

boot_test(x, y = NULL, variable = "AUC", in_bag = TRUE, correction = "holm")

Arguments

x

A cutpointr object with bootstrap results

y

If x does not contain subgroups another cutpointr object

variable

The variable for testing

in_bag

Whether the in-bag or out-of-bag results should be used for testing

correction

The type of correction for multiple testing. Possible values are as in p.adjust.methods

Value

A data.frame (a tibble) with the columns test_var, p, d, sd_d, z and in_bag. If a grouped cutpointr object was tested, the additional columns subgroup1, subgroup2 and p_adj are returned.

Details

The variable name is looked up in the columns of the bootstrap results where the suffixes _b and _oob indicate in-bag and out-of-bag estimates, respectively (controlled via the in_bag argument). Possible values are optimal_cutpoint, AUC, acc, sensitivity, specificity, and the metric that was selected in cutpointr. Note that there is no "out-of-bag optimal cutpoint", so when selecting variable = optimal_cutpoint the test will be based on the in-bag data.

The test statistic is calculated as z = (t1 - t2) / sd(t1 - t2) where t1 and t2 are the metric values on the full sample and sd(t1 - t2) is the standard deviation of the differences of the metric values per bootstrap repetition. The test is two-sided.

If two cutpointr objects are compared and the numbers of bootstrap repetitions differ, the smaller number will be used.

See Also

Other main cutpointr functions: add_metric(), boot_ci(), cutpointr(), multi_cutpointr(), predict.cutpointr(), roc()

Examples

# NOT RUN {
library(cutpointr)
library(dplyr)
set.seed(734)
cp_f <- cutpointr(suicide %>% filter(gender == "female"), dsi, suicide,
  boot_runs = 1000, boot_stratify = TRUE)
set.seed(928)
cp_m <- cutpointr(suicide %>% filter(gender == "male"), dsi, suicide,
  boot_runs = 1000, boot_stratify = TRUE)
# No significant differences:
boot_test(cp_f, cp_m, AUC, in_bag = TRUE)
boot_test(cp_f, cp_m, sum_sens_spec, in_bag = FALSE)

set.seed(135)
cp <- cutpointr(suicide, dsi, suicide, gender, boot_runs = 1000,
  boot_stratify = TRUE)
# Roughly same result as above:
boot_test(cp, variable = AUC, in_bag = TRUE)
boot_test(cp, variable = sum_sens_spec, in_bag = FALSE)
# }
# NOT RUN {
# }