chemmodlab (version 2.0.0)

PerfCurveTest: Perform a hypothesis test for the difference between two performance curves.

Description

PerfCurveTest takes score vectors for two scoring algorithms and an activity vector. A performance curve is created for the two scoring algorithms and hypothesis tests are performed at the selected testing fractions.

Usage

PerfCurveTest(
  S1,
  S2,
  X,
  r,
  metric = "rec",
  method = "EmProc",
  type = "pointwise",
  plus = T,
  pool = F,
  alpha = 0.05,
  h = NULL,
  seed = 111,
  mc.rep = 1e+05
)

Arguments

S1

a vector of scores for scoring algorithm 1.

S2

a vector of scores for scoring algorithm 2.

X

a vector of activities.

r

a vector of testing fractions.

metric

the performance curve to use. Options are recall ("rec") and precision ("prec").

method

the method to use. Recall options are c("EmProc", "binomial", "JZ ind", "mcnemar", "binomial ind"). Precision options are c("EmProc", "binomial", "JZ ind", "stouffer", "binomial ind").

type

specifies whether a point-wise confidence interval ("pointwise") or a confidence band ("band") should be constructed.

plus

should plus correction be applied to the confidence intervals?

pool

use pooling for hypothesis tests? Only relevant to "EmProc".

alpha

the significance level.

h

the bandwidth for the local regression estimator of Lambda. If NULL, uses the default plugin estimator.

seed

the random seed.

mc.rep

the number of Monte Carlo replicates to use for the sup-t method.