OmicsMarkeR (version 1.4.2)

perm.class: Monte Carlo Permutation of Model Performance

Description

Applies Monte Carlo permutations to user specified models. The user can either use the results from fs.stability or provide specified model parameters.

Usage

perm.class(fs.model = NULL, X, Y, method, k.folds = 5, metric = "Accuracy", nperm = 10, allowParallel = FALSE, create.plot = FALSE, verbose = TRUE, ...)

Arguments

fs.model

Object containing results from fs.stability

X

A scaled matrix or dataframe containing numeric values of each feature

Y

A factor vector containing group membership of samples

method

A string of the model to be fit. Available options are "plsda" (Partial Least Squares Discriminant Analysis), "rf" (Random Forest), "gbm" (Gradient Boosting Machine), "svm" (Support Vector Machines), "glmnet" (Elastic-net Generalized Linear Model), and "pam" (Prediction Analysis of Microarrays)

k.folds

How many and what fractions of dataset held-out for prediction (i.e. 3 = 1/3, 10 = 1/10, etc.)

metric

Performance metric to assess. Available options are "Accuracy", "Kappa", and "ROC.AUC".

nperm

Number of permutations, default nperm = 10

allowParallel

Logical argument dictating if parallel processing is allowed via foreach package. Default allowParallel = FALSE

create.plot

Logical argument whether to create a distribution plot of permuation results.

verbose

Logical argument whether output printed automatically in 'pretty' format. Default create.plot = FALSE

...

Extra arguments that the user would like to apply to the models

Value

p.value

Resulting p-value of permuation test

References

Guo Y., et. al. (2010) Sample size and statistical power considerations in high-dimensionality data settings: a comparative study of classification algorithms. BMC Bioinformatics 11:447.

Examples

dat.discr <- create.discr.matrix(
    create.corr.matrix(
        create.random.matrix(nvar = 50, 
                             nsamp = 100, 
                             st.dev = 1, 
                             perturb = 0.2)),
    D = 10
)

vars <- dat.discr$discr.mat
groups <- dat.discr$classes

fits <- fs.stability(vars, 
                     groups, 
                     method = c("plsda", "rf"), 
                     f = 10, 
                     k = 3, 
                     k.folds = 10, 
                     verbose = 'none')

perm.class(fits, vars, groups, "rf", k.folds=5,
           metric="Accuracy", nperm=10)