BiDAG (version 2.1.4)

samplecomp: Performance assessment of sampling algorithms against a known Bayesian network

Description

This function compute 8 different metrics of structure fit of an object of classes orderMCMC and partitionMCMC to the ground truth DAG (or CPDAG). First posterior probabilities of single edges are calculated based on a sample stores in the object of class orderMCMC or partitionMCMC. This function computes structure fit of each of the consensus graphs to the ground truth one based on a defined range of posterior thresholds. Computed metrics include: TP, FP, TPR, FPR, FPRn, FDR, SHD. See metrics description in see also compareDAGs.

Usage

samplecomp(
  MCMCchain,
  truedag,
  p = c(0.99, 0.95, 0.9, 0.8, 0.7, 0.6, 0.5, 0.4, 0.3, 0.2),
  pdag = TRUE,
  burnin = 0.2,
  trans = TRUE
)
# S3 method for samplecomp
plot(x, ..., vars = c("FP", "TP"), type = "b", col = "blue", showp = NULL)
# S3 method for samplecomp
print(x, ...)
# S3 method for samplecomp
summary(object, ...)

Value

an object if class samplesim, a matrix with the number of rows equal to the number of elements in 'p', and 8 columns reporting for the consensus graphss (corresponfing to each of the values in 'p') the number of true positive edges ('TP'), the number of false positive edges ('FP'), the number of false negative edges ('FN'), the true positive rate ('TPR'), the structural Hamming distance ('SHD'), false positive rate ('FPR'), false discovery rate ('FDR') and false positive rate normalized by TP+FN ('FPRn').

Arguments

MCMCchain

an object of class partitionMCMC or orderMCMC, representing the output of structure sampling function partitionMCMC or orderMCMC (the latter when parameter chainout=TRUE;

truedag

ground truth DAG which generated the data used in the search procedure; represented by an object of class graphNEL

p

a vector of numeric values between 0 and 1, defining posterior probabilities according to which the edges of assessed structures are drawn, please note very low barriers can lead to very dense structures; by default \(p=c(0.99, 0.95, 0.9, 0.8, 0.7, 0.6, 0.5, 0.4, 0.3, 0.2)\)

pdag

logical, if TRUE (default) all DAGs in the MCMCchain are first converted to equivalence class (CPDAG) before the averaging

burnin

number between 0 and 1, indicates the percentage of the samples which will be the discarded as `burn-in' of the MCMC chain; the rest of the samples will be used to calculate the posterior probabilities; 0.2 by default

trans

logical, for DBNs indicates if model comparions are performed for transition structure; when trans equals FALSE the comparison is performed for initial structures of estimated models and the ground truth DBN; for usual BNs the parameter is disregarded

x

object of class 'samplecomp'

...

ignored

vars

a tuple of variables which will be used for 'x' and 'y' axes; possible values: "SHD", "TP", "FP", "TPR", "FPR", "FPRn", "FDR"

type

type of line in the plot; "b" by default

col

colour of line in the plotl; "blue" by default

showp

logical, defines if points are labelled with the posterior threshold corresponding to the assessed model

object

object of class 'samplecomp'

Author

Polina Suter

Examples

gsim.score<-scoreparameters("bge", gsim)
if (FALSE) {
MAPestimate<-learnBN(gsim.score,"orderIter",scoreout=TRUE)
ordersample<-sampleBN(gsim.score, "order", scoretable=getSpace(MAPestimate))
samplecomp(ordersample, gsimmat)
}