NEArender (version 1.5)

benchmark: Benchmark networks using Network Enrichment Analysis (NEA)

Description

Tests the ability of a given network to perform well in a network enrichment analysis. It executes a series of multiple individual tests: for each member gene of a pathway or another functional set calculates the network enrichment score against other members of the same gene set. This procedure gives true positive and false negative test results. In order to complement it with false positives and true negatives, the same is done for randomly picked genes (with matching node connectivity values) against the same functional sets. The two vectors allow plotting a ROC curve where at each sequential cutoff represents a ratio of true positive vs. false positive predictions. This approach (first presented in Merid et al. (2012)) is an alternative to the trivial counting edges shared between different networks and is superior to the latter because: 1) the analysis can be done without knowing the "true" reference network, 2) benchmarks can be context-dependent by using domain-specific test sets (e.g. cancer, diabetes etc.), 3) one can compare more than two networks at a time, and 4) given dense global networks and due to the use of multi-gene sets, presence or absence of particular links is unlikely to affect the overall result.

Usage

benchmark(NET, GS, gs.gene.col = 2, gs.group.col = 3, net.gene1.col = 1,
  net.gene2.col = 2, echo = 1, graph = FALSE, na.replace = 0,
  mask = ".", minN = 0, coff.z = 1.965, coff.fdr = 0.1,
  Parallelize = 1)

Arguments

NET

A network to benchmark. See Details in nea.render.

GS

a test set, typically a set of pathways with known members.

gs.gene.col

number of the column containing GS genes (only needed if GS is submitted as a text file)

gs.group.col

number of the column containing group IDs (only needed if GS is submitted as a text file)

net.gene1.col

number of the column containing first nodes of each network edge (only needed if NET is submitted as a text file)

net.gene2.col

number of the column containing second nodes of each network edge (only needed if NET is submitted as a text file)

echo

if messages about execution progress should appear

graph

Plot the ROC curve immediately. Alternatively, the returned list is plotted afterwards by roc. In the latter case, it could be a combined list of lists for multiple test sets and networks which are then plotted as separate curves (see Examples).

na.replace

replace NA values. Default=0, i.e. do not replace.

mask

when the test set contains various GSs, they can be used selectively by applying a mask. The mask follows the regular expression synthax, since fixed=FALSE in grep.

minN

the minimal number of network edges that must connect a tested member with the GS genes for the test to be considered positive. (Default:0).

coff.z

a parameter to roc.

coff.fdr

to make significance levels comparable between different curves, the point where FDR=coff.fdr will be labeled with a circle (think of TP/FP ratio at this level).

Parallelize

The number of CPU cores to be used for the step "Counting actual links" (while the other steps are sufficiently fast). The option is not supported in Windows.

Value

An object, i.e. a list of three equal-length vectors from a prediction object of ROCR package (prediction@cutoffs, prediction@fp, prediction@tp) and the point that matches coff.fdr. These are needed to plot a ROC curve for the given network and test set by using roc.

Details

The function would either plot a ROC curve for the analyzed network, or return an object with the following slots from function prediction (package ROCR): tp, vector of true positives; fp, vector of false positives; tn, vector of true negatives; fn, vector of false negatives; cutoffs, z-score cutoffs from nea.render; cross.z, a z-score value which corresponds to FDR=coff.fdr (will be denoted with a special marker at the curve);

References

http://www.biomedcentral.com/1471-2105/15/308

See Also

roc, nea.render

Examples

# NOT RUN {
data(can.sig.go);
fpath <- can.sig.go
gs.list <- import.gs(fpath, Lowercase = 1, col.gene = 2, col.set = 3);
data(net.kegg)
netpath <- net.kegg
net <- import.net(netpath)
# }
# NOT RUN {
b0 <- benchmark (NET = net,
 GS = gs.list, 
 echo=1, graph=TRUE, na.replace = 0, mask = ".", minN = 0,
 coff.z = 1.965, coff.fdr = 0.1, Parallelize=2);
# }
# NOT RUN {
## Benchmark a number of networks on GO terms and KEGG pathways separately, using masks:
b1 <- NULL;
for (mask in c("kegg_", "go_")) {
b1[[mask]] <- NULL;
for (file.net in c("netpath")) {
# a series of networks can be put here: c("netpath1", "netpath2", "netpath3")
net <- import.net(netpath, col.1 = 1, col.2 = 2, Lowercase = 1, echo = 1)
b1[[mask]][[file.net]] <- benchmark (NET = net, GS = gs.list, echo=1, 
graph=FALSE, na.replace = 0, mask = mask, minN = 0,  Parallelize=1);
}}
par(mfrow=c(2,1));
roc(b1[["kegg_"]], coff.z = 2.57,main="kegg_");
roc(b1[["go_"]], coff.z = 2.57,main="go_");
# }