netbenchmark (version 1.4.2)

experiments.bench: Noise sensitivity benchmark

Description

For a given vector of character of the names of wrapper functions that compute a network inference methods, experiments.bench performs a number of experiments sensitivity test. It makes use of five different big gene datasets subsampling them to generate different datasets.num of the network with different number of experiments.

Usage

experiments.bench(methods = "all.fast", datasources.names = "all",
        experiments = c(20, 50, 150), eval = "AUPR",
        no.topedges = 20, datasets.num = 3, local.noise = 20,
        global.noise = 0, noiseType = "normal", sym = TRUE,
        seed = NULL, verbose= TRUE)

Arguments

methods

A vector of characters containing the names of network inference algorithms wrappers to be compared (default: "all.fast").

datasources.names

A vector of characters containing the names of network datasets to be included in the benchmark (default: "all").

experiments

A vector to set the number of experiments to test the methods (default=c(20,50,150)).

eval

The name of the evaluation metric among the following ones: "no.truepos", "AUROC" or "AUPR" (default : "AUPR") - see evaluate.

no.topedges

Float specifying the percentage number of links to be considered in the evaluation (default: 20).

datasets.num

Number of repetitions in the noise evaluation, for each method and each dataset and each noise intensity (default: 3).

local.noise

Integer specifying the desired percentage of local noise to be added at each of the subsampled datasets (default: 20) - see datasource.subsample.

global.noise

Integer specifying the desired percentage of global noise to be added at each of the subsampled datasets (default: 20) - see datasource.subsample.

noiseType

Character specifying the type of the noise to be added: "normal" or "lognormal" (default: "normal") - see datasource.subsample.

sym

Logical specifying if the evaluation is symmetric (default: TRUE) - see evaluate.

seed

A single value, interpreted as an integer to specify seeds, useful for creating simulations that can be reproduced (default: NULL) - see set.seed.

verbose

Logical specifying if the code should provide a log about what the function is doing (default: TRUE).

Value

• experiments.bench returns a list with three elements:
  1. A data.frame which is the result table containing the number of true positives as an evaluation measure. It evaluates each algorithm specified atmethodsat each one of the specifieddatasources.nameswith different noise intensities.
  2. A data.frame which is the corresponding pvalue table of the corresponding statistical test for each one of thedatasets.numbetween the best algorithm and the others.
  3. The seed of the random number generators that allows the replication of the results.

Details

The argument methods accepts "all.fast" and "all" (case insensitive) as a parameters:

• "all.fast" performs network inference with "aracne", "c3net", "clr", "GeneNet", "mutual ranking", "mrnet", "pcit"
• "all" performs network inference with "aracne", "c3net", "clr", "GeneNet", "Genie3", "mutual ranking", "mrnet", "mrnetb", "pcit"

It evaluates the first no.topedges % of the possible links inferred by each algorithm at each dataset.

Two different types of noises are added independently:

• "Local": the standard deviation of the noise is different for each variable.local.noisespecifies the percentage for each variable ($\pm 20 %$).
• "Global": the standard deviation of the noise is the same for the whole dataset.global.noisespecifies the percentage of the mean standard deviation of all the variables ($\pm 20 %$).

The distribution of noise is set with noiseType, it is possible to choose between "normal" (rnorm) and "lognormal" (rlnorm). The argument noiseType can be a single character, this specifies the same distribution for both "Local" and "Global" noise, it also can be a vector of characters with two elements, the former specifies the distribution of "Local" noise and the later the distribution of "Global" noise.

See Also

netbenchmark, noise.bench

Examples

results <- experiments.bench(datasources.names="toy",
    datasets.num=2,methods="all.fast",experiments=c(20,40))