Constraint based feature selection algorithms: SES: Feature selection algorithm for identifying multiple minimal, statistically-equivalent and equally-predictive feature signatures. MMPC: Feature selection algorithm for identifying minimal feature subsets.

Description

SES algorithm follows a forward-backward filter approach for feature selection in order to provide minimal, highly-predictive, statistically-equivalent, multiple feature subsets of a high dimensional dataset. See also Details. MMPC algorithm follows the same approach without generating multiple feature subsets.

Usage

SES(target, dataset, max_k = 3, threshold = 0.05, test = NULL,
 user_test = NULL, hash = FALSE, hashObject = NULL, robust = FALSE, ncores = 1)
 
MMPC(target, dataset, max_k = 3, threshold = 0.05, test = NULL,
 user_test = NULL, hash = FALSE, hashObject = NULL, robust = FALSE, ncores = 1, 
 backward = FALSE)

Arguments

target

The class variable. Provide either a string, an integer, a numeric value, a vector, a factor, an ordered factor or a Surv object. See also Details.

dataset

The data-set; provide either a data frame or a matrix (columns = variables , rows = samples). Alternatively, provide an ExpressionSet (in which case rows are samples and columns are features, see bioconductor for details).

max_k

The maximum conditioning set to use in the conditional indepedence test (see Details). Integer, default value is 3.

threshold

Threshold (suitable values in [0,1]) for assessing p-values significance. Default value is 0.05.

test

The conditional independence test to use. Default value is NULL. See also CondIndTests.

user_test

A user-defined conditional independence test (provide a closure type object). Default value is NULL. If this is defined, the "test" argument is ignored.

hash

A boolean variable which indicates whether (TRUE) or not (FALSE) to store the statistics calculated during SES execution in a hash-type object. Default value is FALSE. If TRUE a hashObject is produced.

hashObject

A List with the hash objects generated in a previous run of SES. Each time SES runs with "hash=TRUE" it produces a list of hashObjects that can be re-used in order to speed up next runs of SES. Important: the generated hashObjects should be used only

robust

A boolean variable which indicates whether (TRUE) or not (FALSE) to use a robust version of the statistical test if it is available. It takes more time than a non robust version but it is suggested in case of outliers. Default value is FALSE.

ncores

How many cores to use. This plays an important role if you have tens of thousands of variables or really large sample sizes and tens of thousands of variables and a regression based test which requires numerical optimisation. In other cases it will not ma

backward

If TRUE, the backward (or symmetry correction) phase will be implemented. This removes any falsely included variables in the parents and children set of the target variable and it will slow down the algorithm. Bear in mind that the target becomes predicto

Value

The output of the algorithm is an object of the class 'SESoutput' for SES or 'MMPCoutput' for MMPC including:
selectedVarsThe selected variables, i.e., the signature of the target variable.
selectedVarsOrderThe order of the selected variables according to increasing pvalues.
queuesA list containing a list (queue) of equivalent features for each variable included in selectedVars. An equivalent signature can be built by selecting a single feature from each queue. Featured only in SES.
signaturesA matrix reporting all equivalent signatures (one signature for each row). Featured only in SES.
hashObjectThe hashObject caching the statistic calculted in the current run.
pvaluesFor each feature included in the dataset, this vector reports the strength of its association with the target in the context of all other variables. Particularly, this vector reports the max p-values foudn when the association of each variable with the target is tested against different conditional sets. Lower values indicate higher association.
statsThe statistics corresponding to "pvalues" (higher values indicates higher association).
max_kThe max_k option used in the current run.
thresholdThe threshold option used in the current run.
runtimeThe run time of the algorithm. A numeric vector. The first element is the user time, the second element is the system time and the third element is the elapsed time.
testThe character name of the statistic test used.
robThe value of the robust option, either TRUE or FALSE.
summary(x=SESoutput)Summary view of the SESoutput object.
plot(object=SESoutput, mode="all")Plots the generated pvalues (using barplot) of the current SESoutput object in comparison to the threshold. Argument mode can be either "all" or "partial" for the first 500 pvalues of the object.

Details

The SES function implements the Statistically Equivalent Signature (SES) algorithm as presented in "Tsamardinos, Lagani and Pappas, HSCBB 2012" http://www.mensxmachina.org/publications/discovering-multiple-equivalent-biomarker-signatures/ The MMPC function mplements the MMPC algorithm as presented in "Tsamardinos, Brown and Aliferis. The max-min hill-climbing Bayesian network structure learning algorithm" http://www.dsl-lab.org/supplements/mmhc_paper/paper_online.pdf For faster computations in the internal SES functions, install the suggested package "gRbase". The max_k option: the maximum size of the conditioning set to use in the conditioning independence test. Larger values provide more accurate results, at the cost of higher computational times. When the sample size is small (e.g., $<50$ observations)="" the="" max_k="" parameter="" should="" be="" $\leq="" 5$,="" otherwise="" conditional="" independence="" test="" may="" not="" able="" to="" provide="" reliable="" results.="" if="" dataset="" contains="" missing="" (na)="" values,="" they="" will="" automatically="" replaced="" by="" current="" variable="" (column)="" mean="" value="" with="" an="" appropriate="" warning="" user="" after="" execution.="" target="" is="" a="" single="" integer="" or="" string,="" it="" has="" corresponds="" column="" number="" name="" of="" feature="" in="" dataset.="" any="" other="" case="" that="" contained="" 'test'="" argument="" defined="" as="" null="" "auto"="" and="" user_test="" then="" algorithm="" selects="" best="" based="" on="" type="" data.="" particularly:=""

if target is a factor, the multinomial or the binary logistic regression is used. If the target has two values only, binary logistic regression will be used.

if target is a ordered factor, the ordered logit regression is used in the logistic test.

if target is a numerical vector and the dataset is a matrix or a data.frame with continuous variables, the Fisher conditional independence test is used. If the dataset is a data.frame and there are categorical variables, linear regression is used.

if target is discrete numerical (counts), the poisson regression conditional independence test is used. If there are only two values, the binary logistic regression is to be used.

if target is a Surv object, the Survival conditional independence test is used.

if target is a matrix with at least 2 columns, the multivariate linear regression is used.

Conditional independence test functions to be pass through the user_test argument should have the same signature of the included test. See testIndFisher for an example. For all the available conditional independence tests that are currently included on the package, please see CondIndTests. If two or more p-values are below the machine epsilon (.Machine$double.eps which is equal to 2.220446e-16), all of them are set to 0. To make the comparison or the ordering feasible we use the logarithm of the p-value. The max-min heuristic though, requires comparison and an ordering of the p-values. Hence, all conditional independence tests calculate the logarithm of the p-value. If there are missing values in the dataset (predictor variables) columnwise imputation takes place. The median is used for the continuous variables and the mode for categorical variables. It is a naive and not so clever method. For this reason the user is encouraged to make sure his data contain no missing values. If you have percentages, in the (0, 1) interval, they are automatically mapped into $R$ by using the logit transformation. If you set the test to testIndBeta, beta regression is used. If you have compositional data, positive multivariate data where each vector sums to 1, with NO zeros, they are also mapped into the Euclidean space using the additive log-ratio (multivariate logit) transformation (Aitchison, 1986). If you use testIndSpearman (argument "test"), the ranks of the data calculated and those are used in the caclulations. This speeds up the whole procedure.

References

I. Tsamardinos, V. Lagani and D. Pappas (2012). Discovering multiple, equivalent biomarker signatures. In proceedings of the 7th conference of the Hellenic Society for Computational Biology & Bioinformatics - HSCBB12. Tsamardinos, Brown and Aliferis (2006). The max-min hill-climbing Bayesian network structure learning algorithm. Machine learning, 65(1), 31-78.

Examples

Run this code

set.seed(123)
#require(gRbase) #for faster computations in the internal functions
require(hash)

#simulate a dataset with continuous data
dataset <- matrix(runif(1000 * 1000, 1, 100), ncol = 1000)

#define a simulated class variable 
target <- 3 * dataset[, 10] + 2 * dataset[, 200] + 3 * dataset[, 20] + rnorm(1000, 0, 5)

#define some simulated equivalences
dataset[, 15] <- dataset[,10] + rnorm(1000, 0, 2)
dataset[, 10] <- dataset[ ,10] + rnorm(1000, 0, 2)
dataset[, 250] <- dataset[,200] + rnorm(1000, 0, 2) 
dataset[, 230] <- dataset[,200] + rnorm(1000, 0, 2)

require("hash", quietly = TRUE) 
#run the SES algorithm
sesObject <- SES(target , dataset, max_k = 5, threshold = 0.05, test = "testIndFisher", 
hash = TRUE, hashObject = NULL);
#print summary of the SES output
summary(sesObject);
#plot the SES output
plot(sesObject, mode = "all");
#get the queues with the equivalences for each selected variable
sesObject@queues
#get the generated signatures
sesObject@signatures;
#get the run time
sesObject@runtime;

#re-run the SES algorithm with the same or different configuration 
#under the hash-based implementation of retrieving the statistics
#in the SAME dataset (!important)
#hashObj <- sesObject@hashObject;
#sesObject2 <- SES(target, dataset, max_k = 2, threshold = 0.01, test = "testIndFisher", 
#hash = TRUE, hashObject = hashObj);
#retrieve the results: summary, plot, sesObject2@...)
#summary(sesObject2)
#get the run time
#sesObject2@runtime;

#MMPCObject <- MMPC(target , dataset , max_k=3 , threshold=0.05 , test="testIndFisher", 
#hash = FALSE, hashObject=NULL);
#MMPCObject@selectedVars
#MMPCObject@runtime

Run the code above in your browser using DataLab

State of Data and AI Literacy Report 2025