raptools (version 1.23.0)

CI.raplot: Statistical metrics with confidence intervals

Description

The CI.raplot function produces summary metrics for risk assessment. Outputs the NRI, IDI, weighted NRI and category Free NRI all for those with events and those without events. Also the AUCs of the two models and the comparison (DeLong) between AUCs. Output includes confidence intervals. Uses statistics.raplot. Displayed graphically by raplot.

Usage

CI.raplot(
  x1,
  x2 = NULL,
  y = NULL,
  t = NULL,
  NRI_return = FALSE,
  conf.level = 0.95,
  n.boot = 1000,
  dp = 3
)

Value

A list with the following elements:

meta_data

A data.frame with thresholds, confidence interval, number of bootstraps, input data type and decimal places.

Metrics

Point estimates of the statistical metrics (see function docs).

Each_bootstrap_metrics

List of per-bootstrap metric results.

Summary_metrics

A table of summary metrics with confidence intervals (e.g. Total, Events, Non-events, NRI, IDI, AUCs, Brier scores, etc.).

Arguments

x1

Either a logistic regression fitted using glm (base package) or lrm (rms package) or calculated probabilities (eg through a logistic regression model) of the baseline model. Must be between 0 & 1

x2

Either a logistic regression fitted using glm (base package) or lrm (rms package) or calculated probabilities (eg through a logistic regression model) of the new (alternative) model. Must be between 0 & 1

y

Binary of outcome of interest. Must be 0 or 1 (if fitted models are provided this is extracted from the fit which for an rms fit must have x = TRUE, y = TRUE).

t

The risk threshold(s) for groups. eg t<-c(0,0.1,1) is a two group model with a threshold of 0.1 & t<-c(0,0.1,0.3,1) is a three group model with thresholds at 0.1 and 0.3.

NRI_return

If NRI statistics are required (default = FALSE).

conf.level

The confidence interval expressed as a fraction of 1 (ie 0.95 is the 95% confidence interval )

n.boot

The number of "bootstraps" to use. Performance slows down with more bootstraps. For trialling result, use a low number (eg 5), for accuracy use a large number (eg 2000)

dp

The number of decimal places to display

References

Pencina, M. J., D'Agostino, R. B., & Vasan, R. S. (2008). Evaluating the added stats::predictive ability of a new marker: From area under the ROC curve to reclassification and beyond. Statistics in Medicine, 27(2), 157-172. doi:10.1002/sim.2929

Examples

# Quick example with subset of data and fewer bootstraps
data(data_risk)
data_subset <- data_risk[1:100, ]  # Use first 100 rows for speed
complete_cases <- complete.cases(data_subset)
data_clean <- data_subset[complete_cases, ]
y <- data_clean$outcome 
x1 <- data_clean$baseline
x2 <- data_clean$new
t <- c(0, 0.19, 1) 
output <- CI.raplot(x1, x2, y, t, conf.level = 0.95, n.boot = 10, dp = 2)

# \donttest{
# Full dataset example with more bootstraps
data(data_risk)
complete_cases <- complete.cases(data_risk)
data_clean <- data_risk[complete_cases, ]
y <- data_clean$outcome 
x1 <- data_clean$baseline
x2 <- data_clean$new
t <- c(0, 0.19, 1) 
output <- CI.raplot(x1, x2, y, t, conf.level = 0.95, n.boot = 1000, dp = 2) 
# }