MoTBFs (version 2.0)

motbf.cv: Cross-validation for MoTBFs

Description

Perform a k-fold cross validation for Bayesian networks of class MoTBF.

Usage

motbf.cv(
  data,
  dag,
  loss,
  k = 10,
  target = NULL,
  seed = NULL,
  fit.args = NULL,
  loss.args = NULL,
  foldsIndex = NULL,
  ...
)

Value

An object of class "motbf.fit.cv". This is a list of "k" elements, each of them containing the results of each fold. More specifically, each element contains:

test

a "data.frame" of the subset used to fit the model.

fitted

an object of class "motbf.fit", i.e., the model fitted.

loss

the loss value computed for the fold. If "pred" is chosen for the argument "loss", then each element of the "motbf.fit.cv" object also contains:

predicted

a vector of predictions for the target variable.

observed

a vector of observed records of the target variable.

Arguments

data

an object of class "data.frame", which can contain continuous and discrete variables.

dag

a network of the class "bn", "graphNEL" or "network".

loss

a character string indicating which loss function should be used. Currently, two options are available: 'logl', for the log-likelihood of the model; and 'pred', for the predictive error. See details.

k

an integer indicating the number of folds to split the data set. If k = 0, the train and test sets are the same data; if k = 1, hold-out validation is carried out, i.e., the data set is split in train (80% by default) and test (20% by default); finally, if k >=2, k-fold cross validation is carried out.

target

a character string indicating which node is the target. This argument might be NULL if the loss function chosen is the log-likelihood of the model ('logl').

seed

an integer to specify the seed. The k-folds are created randomly, so one might expect slightly different results unless 'seed' is used.

fit.args

a list containing optional arguments used to fit the models. These arguments must be those accepted by function motbf.fit, i.e., 'numIntervals' (4), 'POTENTIAL_TYPE' ('MOP'), 'maxParam' (NULL), 's' (NULL), 'priorData' (NULL) or 'scale' (TRUE). If fit.args is left NULL, the default values (in brackets) for those arguments will be used.

loss.args

a list containing optional arguments related to the loss functions. Currently available arguments are: 'loss.matrix' and 'percentage_test'. See details.

foldsIndex

a list containing the test indexes for each fold of cross-validation. If is not null, k and seed are ignored.

...

Additional arguments. Not used currently.

Details

If the basis function is MTE, the loss function is the log-likelihood of the model. On the other hand, if the basis function if MOP, the loss function might be either the predictive error ('pred') or the the log-likelihood of the model ('logl').

Details on the loss argument:

'logl'

The log-likelihood of the model is computed. This measure is available for both basis functions, MTE and MOP.

'pred'

This option is only available for MOPs. The predictive error (root mean squared error) or classification accuracy is computed as the loss function, depending on the nature of the target variable (continuous or discrete, respectively). The program will guess which type the target variable is and compute the corresponding measure.

Details on the loss.args argument. Currently, two arguments can be specified within this list:

'p.test'

Only used if k = 1. This argument specifies the proportion of the data set that goes to the test set (between 0 and 1).

'loss.matrix'

A squared matrix used to compute a weighted classification accuracy for discrete targets. This matrix is multiplied by the confusion matrix element by element, and the resulting matrix is used to compute the classification accuracy. The loss matrix allows to increase the penalty of some user-specified errors. If the loss matrix provided is a constant matrix of ones, the result is the standard classification accuracy. Note that the diagonal of the loss matrix is regarded as a reward, while the off-diagonal is regarded as a cost.

Examples

#################
### EXAMPLE 1 ###
#################
## Perform 2-fold cross validation using the default model arguments
## and the log-likelihood as loss function

# Load data
data(ecoli)
ecoli <- ecoli[,-c(1,9)]

# Learn DAG
dag <- LearningHC(ecoli)

# Run cross validation
cv = motbf.cv(data = ecoli, dag, k = 2, loss = 'logl')
cv

# \donttest{
#################
### EXAMPLE 2 ###
#################
## Choose different arguments to fit the model parameters
fit.args = list(numIntervals = 3, POTENTIAL_TYPE = 'MOP', maxParam = 4)

# Run cross validation using the classification accuracy as loss function
cv = motbf.cv(data = ecoli, dag, k = 2, loss = 'pred', target = 'lip', 
  fit.args = fit.args)
cv
summary(cv)

#################
### EXAMPLE 3 ###
#################

## Specify a loss matrix to increase the penalty of classification errors
lossFunctionMatrix = matrix(c(c(1,2), c(3, 1)),nrow = 2, ncol = 2, byrow = TRUE)

# Run cross validation using the weighted classification accuracy as loss function
cv = motbf.cv(data = ecoli, dag, k = 2, loss = 'pred', target = 'lip', 
  loss.args = list(loss.matrix = lossFunctionMatrix))
cv
summary(cv)
  
#################
### EXAMPLE 4 ###
#################

# Run cross validation using the root mean squared error as loss function
cv = motbf.cv(data = ecoli, dag, k = 2, loss = 'pred', target = 'mcg')
cv
# }