ROCR integration: Generating a prediction object for ROCR

Description

Evaluate structure learning accuracy with ROCR. This function views the arcs in a bn.strength object as a set of predictions and the arcs in a true reference graph as a set of labels, and produces a prediction object from the ROCR package. This facilitates evaluation of structure learning with traditional machine learning metrics such as ROC curves and AUC.

Usage

# S3 method for bn.strength
as.prediction(x, true, ..., consider.direction = TRUE)

Value

An object of class prediction from the ROCR package.

Arguments

x: an object of class bn.strength returned by boot.strength(), representing learning results targeting the object of class bn specified by the true argument.
true: an object of class bn, the target of structure learning.
...: additional arguments, currently ignored.
consider.direction: a boolean value. If TRUE an arc's prediction value is set to the product of its strength and direction values in x (interpreted as the probability an arc is both present and has the specified direction). If FALSE the arc's prediction value is set to its strength value.

Author

Robert Ness

Details

One way of evaluating the overall performance of a network structure learning algorithm is to evaluate how well it detects individual arcs. as.prediction() takes each pair of nodes in a ground truth network and labels them with a 1 if an arc exists between them and 0 if not. It uses the arc presence probabilities in a bn.strength object returned by boot.strength() as the predictions.

Examples

Run this code

if (FALSE) {
library(ROCR)

modelstring = paste0("[HIST|LVF][CVP|LVV][PCWP|LVV][HYP][LVV|HYP:LVF][LVF]",
  "[STKV|HYP:LVF][ERLO][HRBP|ERLO:HR][HREK|ERCA:HR][ERCA][HRSA|ERCA:HR][ANES]",
  "[APL][TPR|APL][ECO2|ACO2:VLNG][KINK][MINV|INT:VLNG][FIO2][PVS|FIO2:VALV]",
  "[SAO2|PVS:SHNT][PAP|PMB][PMB][SHNT|INT:PMB][INT][PRSS|INT:KINK:VTUB][DISC]",
  "[MVS][VMCH|MVS][VTUB|DISC:VMCH][VLNG|INT:KINK:VTUB][VALV|INT:VLNG][ACO2|VALV]",
  "[CCHL|ACO2:ANES:SAO2:TPR][HR|CCHL][CO|HR:STKV][BP|CO:TPR]")
true.dag = model2network(modelstring)
strength = boot.strength(alarm, R = 200, m = 30, algorithm = "hc")
pred = as.prediction(strength, true.dag)
perf = performance(pred, "tpr", "fpr")
plot(perf, main = "Arc Detection")
performance(pred, "auc")
}

Run the code above in your browser using DataLab