trinROC (version 0.7)

boot.test: Bootstrap test for three-class ROC data

Description

A statistical test function to assess three-class ROC data. It can be used for assessment of a single classifier or comparison of two independent / correlated classifiers, using the Bootstrap test.

Usage

boot.test(
  x1,
  y1,
  z1,
  x2 = 0,
  y2 = 0,
  z2 = 0,
  dat = NULL,
  paired = FALSE,
  n.boot = 1000,
  conf.level = 0.95,
  alternative = c("two.sided", "less", "greater")
)

Value

A list of class "htest" containing the following components:

statistic

the value of the Z-statistic.

p.value

the p-value for the test.

estimate

a data frame containing the estimated parameters from Classifier 1 and Classifier 2 (if specified).

null.value

a character expressing the null hypothesis.

alternative

a character string describing the alternative hypothesis.

method

a character string indicating what type of extended Metz--Kronman test was performed.

data.name

a character string giving the names of the data.

Summary

A data frame representing the number of NA's as well as the means and the standard deviations per class.

Sigma

The covariance matrix of the VUS.

Arguments

x1, y1, z1

Non-empty numeric vectors of data from the healthy, intermediate and diseased class from Classifier 1.

x2, y2, z2

Numeric vectors of data from the healthy, intermediate and diseased class from Classifier 2, only needed in a comparison of two classifiers.

dat

A data frame of the following structure: The first column represents a factor with three levels, containing the true class membership of each measurement. The levels are ordered according to the convention of higher values for more severe disease status. The second column contains all measurements obtained from Classifier 1 (in the case of single marker assessment). In the case of comparison of two markers, column three contains the measurementss from the Classifier.

paired

A logical indicating whether data arose from a paired setting. If TRUE, each class must have equal sample size for both classifiers.

n.boot

An integer incicating the number of bootstrap replicates sampled to obtain the variance of the VUS. Default is 1000.

conf.level

confidence level of the interval. A numeric value between (0,1) yielding the significance level \(\alpha=1-\code{conf.level}\).

alternative

character string specifying the alternative hypothesis, must be one of "two.sided" (default), "greater" or "less". You can specify just the initial letter. For two sided test, notice \(H0: Z = (VUS_1-VUS_2) / (Var(VUS_1)+Var(VUS_2)-2Cov(VUS_1,VUS_2))^{0.5}\).

Details

Based on the reference standard, the Bootstrap test assesses the discriminatory power of classifiers by comparing the volumes under the ROC surfaces (VUS). It distinguishes between single classifier assessment, where a classifier is compared to the chance plane with VUS=1/6, and comparison between two classifiers. The latter case tests the equality between VUS_1 and VUS_2. The data can arise in a unpaired or paired setting. If paired is TRUE, a correlation is introduced which has to be taken into account. Therefore the sets of the two classifiers have to have classwise equal size. The data can be input as the data frame dat or as single vectors x1, y1, z1, .... The implemented methods to evaluate the VUS and var(VUS), cov(vus.1,vus.2) are based on the empirical model assumptions and resampling techniques. This means, there are no underlying distributions assumed in any of the classes.

References

Nakas, C. T. and C. T. Yiannoutsos (2004). Ordered multiple-class ROC analysis with continuous measurements. Statistics in Medicine, 23(22), 3437–3449.

See Also

trinROC.test, trinVUS.test.

Examples

data(cancer)
data(krebs)

# investigate a single marker:
boot.test(dat = krebs[,c(1,2)], n.boot=500)

# result is equal to:
x1 <- with(krebs, krebs[trueClass=="healthy", 2])
y1 <- with(krebs, krebs[trueClass=="intermediate", 2])
z1 <- with(krebs, krebs[trueClass=="diseased", 2])
boot.test(x1, y1, z1, n.boot=500) 

# comparison of marker 2 and 6:
boot.test(dat = krebs[,c(1,2,5)], paired = TRUE) 

# result is equal to:
x2 <- with(krebs, krebs[trueClass=="healthy", 5])
y2 <- with(krebs, krebs[trueClass=="intermediate", 5])
z2 <- with(krebs, krebs[trueClass=="diseased", 5])
boot.test(x1, y1, z1, x2, y2, z2, paired = TRUE)