specmine (version 4.0.0)

train_models_performance: Train multiple models and compare their performance

Description

Train a set of models, collect their resampling performance, optionally compute variable importance, and store fitted models and tuning summaries.

Usage

train_models_performance(
  dataset,
  models,
  column.class,
  validation,
  num.folds = 10,
  num.repeats = 10,
  tunelength = 10,
  tunegrid = NULL,
  metric = NULL,
  summary.function = "default",
  class.in.metadata = TRUE,
  compute.varimp = TRUE
)

Value

A list containing model performance, variable importance, full tuning results, best tuning settings, optional confusion matrices, and final fitted models.

Arguments

dataset

A dataset object.

models

A character vector with model names accepted by caret::train().

column.class

The metadata column containing the class labels.

validation

Validation method used in training.

num.folds

Number of folds used in resampling.

num.repeats

Number of repeats used in repeated resampling.

tunelength

Number of tuning levels evaluated by caret.

tunegrid

Optional list of tuning grids, one per model.

metric

Optional performance metric used for model selection.

summary.function

Summary function, or "default" to select the package default.

class.in.metadata

Logical; if TRUE, class labels are taken from metadata.

compute.varimp

Logical; if TRUE, variable importance is computed.

Examples

if (FALSE) {
datamat <- matrix(
  rnorm(24),
  nrow = 4,
  dimnames = list(paste0("v", 1:4), paste0("s", 1:6))
)
metadata <- data.frame(class = factor(c("A", "A", "A", "B", "B", "B")))
dataset <- list(data = datamat, metadata = metadata)
train_models_performance(
  dataset,
  models = c("rpart"),
  column.class = "class",
  validation = "cv",
  compute.varimp = FALSE
)
}