caret (version 5.17-7)

train: Fit Predictive Models over Different Tuning Parameters


This function sets up a grid of tuning parameters for a number of classification and regression routines, fits each model and calculates a resampling based performance measure.


train(x, ...)

## S3 method for class 'default': train(x, y, method = "rf", preProcess = NULL, ..., weights = NULL, metric = ifelse(is.factor(y), "Accuracy", "RMSE"), maximize = ifelse(metric == "RMSE", FALSE, TRUE), trControl = trainControl(), tuneGrid = NULL, tuneLength = 3)

## S3 method for class 'formula': train(form, data, ..., weights, subset, na.action, contrasts = NULL)


a data frame containing training data where samples are in rows and features are in columns.
a numeric or factor vector containing the outcome for each sample.
A formula of the form y ~ x1 + x2 + ...
Data frame from which variables specified in formula are preferentially to be taken.
a numeric vector of case weights. This argument will only affect models that allow case weights.
An index vector specifying the cases to be used in the training sample. (NOTE: If given, this argument must be named.)
A function to specify the action to be taken if NAs are found. The default action is for the procedure to fail. An alternative is na.omit, which leads to rejection of cases with missing values on any required variable. (NOTE: If given, this argument must
a list of contrasts to be used for some or all of the factors appearing as variables in the model formula.
a string specifying which classification or regression model to use. Possible values are: ada, avNNet, bag, bagEarth, bagFDA, bayesglm, bdk, blackboost
arguments passed to the classification or regression routine (such as randomForest). Errors will occur if values for tuning parameters are passed here.
a string vector that defines an pre-processing of the predictor data. Current possibilities are "BoxCox", "YeoJohnson", "expoTrans", "center", "scale", "range", "knnImpute", "bagImpute", "pca", "ica" and "spatialSign". The default is no pre-processing. Se
a string that specifies what summary metric will be used to select the optimal model. By default, possible values are "RMSE" and "Rsquared" for regression and "Accuracy" and "Kappa" for classification. If custom performance metrics are used (via the
a logical: should the metric be maximized or minimized?
a list of values that define how this function acts. See trainControl. (NOTE: If given, this argument must be named.)
a data frame with possible tuning values. The columns are named the same as the tuning parameters in each method preceded by a period (e.g. .decay, .lambda). Also, a function can be passed to tuneGrid with arguments called len<
an integer denoting the number of levels for each tuning parameters that should be generated by createGrid. (NOTE: If given, this argument must be named.)


  • A list is returned of class train containing:
  • methodthe chosen model.
  • modelTypean identifier of the model type.
  • resultsa data frame the training error rate and values of the tuning parameters.
  • bestTunea data frame with the final parameters.
  • callthe (matched) function call with dots expanded
  • dotsa list containing any ... values passed to the original call
  • metrica string that specifies what summary metric will be used to select the optimal model.
  • controlthe list of control parameters.
  • preProcesseither NULL or an object of class preProcess
  • finalModelan fit object using the best parameters
  • trainingDataa data frame
  • resampleA data frame with columns for each performance metric. Each row corresponds to each resample. If leave-one-out cross-validation or out-of-bag estimation methods are requested, this will be NULL. The returnResamp argument of trainControl controls how much of the resampled results are saved.
  • perfNamesa character vector of performance metrics that are produced by the summary function
  • maximizea logical recycled from the function arguments.
  • yLimitsthe range of the training set outcomes.
  • timesa list of execution times: everything is for the entire call to train, final for the final model fit and, optionally, prediction for the time to predict new samples (see trainControl)


  • \cpkg
  • \bpkg




  • #1
  • #1


train can be used to tune models by picking the complexity parameters that are associated with the optimal resampling statistics. For particular model, a grid of parameters (if any) is created and the model is trained on slightly different data for each candidate combination of tuning parameters. Across each data set, the performance of held-out samples is calculated and the mean and standard deviation is summarized for each combination. The combination with the optimal resampling statistic is chosen as the final model and the entire training set is used to fit a final model.

A variety of models are currently available. The lists below enumerate the models and the values of the method argument, as well as the complexity parameters used by train.


  • Method Value:bagfrom packagecaretwith tuning parametervars(dual use)
  • Method Value:bagEarthfrom packagecaretwith tuning parameters:nprune,degree(dual use)
  • Method Value:bagFDAfrom packagecaretwith tuning parameters:degree,nprune(classification only)
  • Method Value:logicBagfrom packagelogicFSwith tuning parameters:ntrees,nleaves(dual use)
  • Method Value:treebagfrom packageipredwith no tuning parameters (dual use)

Bayesian Methods

  • Method Value:nbfrom packageklaRwith tuning parameters:fL,usekernel(classification only)
  • Method Value:vbmpRadialfrom packagevbmpwith tuning parameterestimateTheta(classification only)

Boosted Trees

  • Method Value:adafrom packageadawith tuning parameters:iter,maxdepth,nu(classification only)
  • Method Value:blackboostfrom packagemboostwith tuning parameters:maxdepth,mstop(dual use)
  • Method Value:bstTreefrom packagebstwith tuning parameters:nu,maxdepth,mstop(dual use)
  • Method Value:C5.0from packageC50with tuning parameters:winnow,model,trials(classification only)
  • Method Value:gbmfrom packagegbmwith tuning parameters:interaction.depth,n.trees,shrinkage(dual use)

Boosting (Non-Tree)

  • Method Value:bstLsfrom packagebstwith tuning parameters:mstop,nu(dual use)
  • Method Value:bstSmfrom packagebstwith tuning parameters:nu,mstop(dual use)
  • Method Value:gamboostfrom packagemboostwith tuning parameters:prune,mstop(dual use)
  • Method Value:glmboostfrom packagemboostwith tuning parameters:prune,mstop(dual use)
  • Method Value:logitBoostfrom packagecaToolswith tuning parameternIter(classification only)

Elastic Net

  • Method Value:glmnetfrom packageglmnetwith tuning parameters:alpha,lambda(dual use)

Flexible Discriminant Analysis (MARS basis)

  • Method Value:fdafrom packagemdawith tuning parameters:nprune,degree(classification only)

Gaussian Processes

  • Method Value:gaussprLinearfrom packagekernlabwith no tuning parameters (dual use)
  • Method Value:gaussprPolyfrom packagekernlabwith tuning parameters:degree,scale(dual use)
  • Method Value:gaussprRadialfrom packagekernlabwith tuning parametersigma(dual use)

Generalized additive model

  • Method Value:gamfrom packagemgcvwith tuning parameters:select,method(dual use)
  • Method Value:gamLoessfrom packagegamwith tuning parameters:degree,span(dual use)
  • Method Value:gamSplinefrom packagegamwith tuning parameterdf(dual use)

Generalized linear model

  • Method Value:glmfrom packagestatswith no tuning parameters (dual use)
  • Method Value:bayesglmfrom packagearmwith no tuning parameters (dual use)
  • Method Value:glmStepAICfrom packageMASSwith no tuning parameters (dual use)

Heteroscedastic Discriminant Analysis

  • Method Value:hdafrom packagehdawith tuning parameters:newdim,lambda,gamma(classification only)

High Dimensional Discriminant Analysis

  • Method Value:hddafrom packageHDclassifwith tuning parameters:model,threshold(classification only)

Independent Component Regression

  • Method Value:icrfrom packagecaretwith tuning parametern.comp(regression only)

K Nearest Neighbor

  • Method Value:knnfrom packagecaretwith tuning parameterk(dual use)

Learned Vector Quantization

  • Method Value:lvqfrom packageclasswith tuning parameters:size,k(classification only)

Linear Discriminant Analysis

  • Method Value:ldafrom packageMASSwith no tuning parameters (classification only)
  • Method Value:lda2from packageMASSwith tuning parameterdimen(classification only)
  • Method Value:Lindafrom packagerrcovwith no tuning parameters (classification only)
  • Method Value:rrldafrom packagerrldawith tuning parameters:lambda,alpha(classification only)
  • Method Value:sdafrom packagesdawith tuning parameterdiagonal(classification only)
  • Method Value:sddaLDAfrom packageSDDAwith no tuning parameters (classification only)
  • Method Value:sldafrom packageipredwith no tuning parameters (classification only)
  • Method Value:stepLDAfrom packageklaRwith tuning parameters:direction,maxvar(classification only)

Linear Least Squares

  • Method Value:leapBackwardfrom packageleapswith tuning parameternvmax(regression only)
  • Method Value:leapForwardfrom packageleapswith tuning parameternvmax(regression only)
  • Method Value:leapSeqfrom packageleapswith tuning parameternvmax(regression only)
  • Method Value:lmfrom packagestatswith no tuning parameters (regression only)
  • Method Value:lmStepAICfrom packageMASSwith no tuning parameters (regression only)
  • Method Value:rlmfrom packageMASSwith no tuning parameters (regression only)

Logic Regression

  • Method Value:logforestfrom packageLogForestwith no tuning parameters (classification only)
  • Method Value:logregfrom packageLogicRegwith tuning parameters:treesize,ntrees(dual use)

Logistic Model Trees

  • Method Value:LMTfrom packageRWekawith tuning parameteriter(classification only)

Logistic/Multinomial Regression

  • Method Value:multinomfrom packagennetwith tuning parameterdecay(classification only)
  • Method Value:plrfrom packagestepPlrwith tuning parameters:cp,lambda(classification only)

Mixture Discriminant Analysis

  • Method Value:mdafrom packagemdawith tuning parametersubclasses(classification only)
  • Method Value:smdafrom packagesparseLDAwith tuning parameters:R,lambda,NumVars(classification only)

Multivariate Adaptive Regression Spline

  • Method Value:earthfrom packageearthwith tuning parameters:nprune,degree(dual use)
  • Method Value:gcvEarthfrom packageearthwith tuning parameterdegree(dual use)

Nearest Shrunken Centroids

  • Method Value:pamfrom packagepamrwith tuning parameterthreshold(classification only)

Neural Networks

  • Method Value:avNNetfrom packagecaretwith tuning parameters:size,bag,decay(dual use)
  • Method Value:mlpfrom packageRSNNSwith tuning parametersize(dual use)
  • Method Value:mlpWeightDecayfrom packageRSNNSwith tuning parameters:decay,size(dual use)
  • Method Value:neuralnetfrom packageneuralnetwith tuning parameters:layer2,layer1,layer3(regression only)
  • Method Value:nnetfrom packagennetwith tuning parameters:size,decay(dual use)
  • Method Value:pcaNNetfrom packagecaretwith tuning parameters:size,decay(dual use)
  • Method Value:qrnnfrom packageqrnnwith tuning parameters:penalty,bag,n.hidden(regression only)

Partial Least Squares

  • Method Value:gplsfrom packagegplswith tuning parameterK.prov(classification only)
  • Method Value:kernelplsfrom packageplswith tuning parameterncomp(dual use)
  • Method Value:plsfrom packageplswith tuning parameterncomp(dual use)
  • Method Value:simplsfrom packageplswith tuning parameterncomp(dual use)
  • Method Value:splsfrom packagesplswith tuning parameters:eta,kappa,K(dual use)
  • Method Value:widekernelplsfrom packageplswith tuning parameterncomp(dual use)

Penalized Discriminant Analysis

  • Method Value:pdafrom packagemdawith tuning parameterlambda(classification only)
  • Method Value:pda2from packagemdawith tuning parameterdf(classification only)

Penalized Linear Models

  • Method Value:enetfrom packageelasticnetwith tuning parameters:fraction,lambda(regression only)
  • Method Value:fobafrom packagefobawith tuning parameters:lambda,k(regression only)
  • Method Value:krlsPolyfrom packageKRLSwith tuning parameters:lambda,degree(regression only)
  • Method Value:krlsRadialfrom packageKRLSwith tuning parameters:sigma,lambda(regression only)
  • Method Value:larsfrom packagelarswith tuning parameterfraction(regression only)
  • Method Value:lars2from packagelarswith tuning parameterstep(regression only)
  • Method Value:lassofrom packageelasticnetwith tuning parameterfraction(regression only)
  • Method Value:penalizedfrom packagepenalizedwith tuning parameters:lambda1,lambda2(regression only)
  • Method Value:relaxofrom packagerelaxowith tuning parameters:lambda,phi(regression only)
  • Method Value:ridgefrom packageelasticnetwith tuning parameterlambda(regression only)

Principal Component Regression

  • Method Value:pcrfrom packageplswith tuning parameterncomp(regression only)

Projection Pursuit Regression

  • Method Value:pprfrom packagestatswith tuning parameternterms(regression only)

Quadratic Discriminant Analysis

  • Method Value:qdafrom packageMASSwith no tuning parameters (classification only)
  • Method Value:QdaCovfrom packagerrcovwith no tuning parameters (classification only)
  • Method Value:sddaQDAfrom packageSDDAwith no tuning parameters (classification only)
  • Method Value:stepQDAfrom packageklaRwith tuning parameters:maxvar,direction(classification only)

Radial Basis Function Networks

  • Method Value:rbffrom packageRSNNSwith tuning parametersize(dual use)
  • Method Value:rbfDDAfrom packageRSNNSwith tuning parameternegativeThreshold(classification only)

Random Forests

  • Method Value:Borutafrom packageBorutawith tuning parametermtry(dual use)
  • Method Value:cforestfrom packagepartywith tuning parametermtry(dual use)
  • Method Value:ORFlogfrom packageobliqueRFwith tuning parametermtry(classification only)
  • Method Value:ORFplsfrom packageobliqueRFwith tuning parametermtry(classification only)
  • Method Value:ORFridgefrom packageobliqueRFwith tuning parametermtry(classification only)
  • Method Value:ORFsvmfrom packageobliqueRFwith tuning parametermtry(classification only)
  • Method Value:parRFfrom packagerandomForestwith tuning parametermtry(dual use)
  • Method Value:qrffrom packagequantregForestwith tuning parametermtry(regression only)
  • Method Value:rffrom packagerandomForestwith tuning parametermtry(dual use)
  • Method Value:rFernsfrom packagerFernswith tuning parameterdepth(classification only)
  • Method Value:RRFfrom packageRRFwith tuning parameters:mtry,coefReg,coefImp(dual use)
  • Method Value:RRFglobalfrom packageRRFwith tuning parameters:coefReg,mtry(dual use)

Recursive Partitioning

  • Method Value:C5.0Treefrom packageC50with no tuning parameters (classification only)
  • Method Value:ctreefrom packagepartywith tuning parametermincriterion(dual use)
  • Method Value:ctree2from packagepartywith tuning parametermaxdepth(dual use)
  • Method Value:evtreefrom packageevtreewith tuning parameteralpha(dual use)
  • Method Value:J48from packageRWekawith tuning parameterC(classification only)
  • Method Value:nodeHarvestfrom packagenodeHarvestwith tuning parameters:maxinter,mode(dual use)
  • Method Value:obliqueTreefrom packageobliqueTreewith tuning parameters:variable.selection,oblique.splits(dual use)
  • Method Value:partDSAfrom packagepartDSAwith tuning,MPD(dual use)
  • Method Value:rpartfrom packagerpartwith tuning parametercp(dual use)
  • Method Value:rpart2from packagerpartwith tuning parametermaxdepth(dual use)

Regularized Discriminant Analysis

  • Method Value:rdafrom packageklaRwith tuning parameters:lambda,gamma(classification only)

Relevance Vector Machines

  • Method Value:rvmLinearfrom packagekernlabwith no tuning parameters (regression only)
  • Method Value:rvmPolyfrom packagekernlabwith tuning parameters:scale,degree(regression only)
  • Method Value:rvmRadialfrom packagekernlabwith tuning parametersigma(regression only)

ROC Curves

  • Method Value:roccfrom packageroccwith tuning parameterxgenes(classification only)

Rule-Based Models

  • Method Value:C5.0Rulesfrom packageC50with no tuning parameters (classification only)
  • Method Value:cubistfrom packageCubistwith tuning parameters:committees,neighbors(regression only)
  • Method Value:JRipfrom packageRWekawith tuning parameterNumOpt(classification only)
  • Method Value:M5from packageRWekawith tuning parameters:rules,pruned,smoothed(regression only)
  • Method Value:M5Rulesfrom packageRWekawith tuning parameters:pruned,smoothed(regression only)
  • Method Value:OneRfrom packageRWekawith no tuning parameters (classification only)
  • Method Value:PARTfrom packageRWekawith tuning parameters:pruned,threshold(classification only)

Self-Organizing Maps

  • Method Value:bdkfrom packagekohonenwith tuning parameters:topo,ydim,xweight,xdim(dual use)
  • Method Value:xyffrom packagekohonenwith tuning parameters:xdim,ydim,topo,xweight(dual use)

Sparse Linear Discriminant Analysis

  • Method Value:PenalizedLDAfrom packagepenalizedLDAwith tuning parameters:K,lambda(classification only)
  • Method Value:sparseLDAfrom packagesparseLDAwith tuning parameters:lambda,NumVars(classification only)

Supervised Principal Components

  • Method Value:superpcfrom packagesuperpcwith tuning parameters:threshold,n.components(regression only)

Support Vector Machines

  • Method Value:lssvmRadialfrom packagekernlabwith tuning parametersigma(classification only)
  • Method Value:svmLinearfrom packagekernlabwith tuning parameterC(dual use)
  • Method Value:svmPolyfrom packagekernlabwith tuning parameters:degree,scale,C(dual use)
  • Method Value:svmRadialfrom packagekernlabwith tuning parameters:C,sigma(dual use)
  • Method Value:svmRadialCostfrom packagekernlabwith tuning parameterC(dual use)

By default, the function createGrid is used to define the candidate values of the tuning parameters. The user can also specify their own. To do this, a data fame is created with columns for each tuning parameter in the model. The column names must be the same as those listed in the table above with a leading dot. For example, ncomp would have the column heading .ncomp. This data frame can then be passed to createGrid.

In some cases, models may require control arguments. These can be passed via the three dots argument. Note that some models can specify tuning parameters in the control objects. If specified, these values will be superseded by those given in the createGrid argument.

The formula interface to train will always convert factor variables to dummy variables. For several models (rpart, rf, gbm, treebag, nb, J48, PART, JRip, OneR, ctree, cforest, bag, cubist, C5.0, C5.0Tree, C5.0Rules and custom) factor predictors variables can be passed directly to the underlying modeling function using the interface train(x, y). In these cases, it is possible for the models to treat factor variables in a manner different than most (i.e. not as a decomposed set of dummy variables).

The web page has more details and examples related to this function.

train can be used with "explicit parallelism", where different resamples (e.g. cross-validation group) and models can be split up and run on multiple machines or processors. By default, train will use a single processor on the host machine. As of version 4.99 of this package, the framework used for parallel processing uses the foreach package. To run the resamples in parallel, the code for train does not change; prior to the call to train, a parallel backend is registered with foreach (see the examples below).


Kuhn (2008), ``Building Predictive Models in R Using the caret'' (

See Also

trainControl, update.train, modelLookup, createGrid, createFolds


Run this code
## Classification Example

TrainData <- iris[,1:4]
TrainClasses <- iris[,5]

knnFit1 <- train(TrainData, TrainClasses,
                 method = "knn",
                 preProcess = c("center", "scale"),
                 tuneLength = 10,
                 trControl = trainControl(method = "cv"))

knnFit2 <- train(TrainData, TrainClasses,
                 method = "knn",
                 preProcess = c("center", "scale"),
                 tuneLength = 10, 
                 trControl = trainControl(method = "boot"))

nnetFit <- train(TrainData, TrainClasses,
                 method = "nnet",
                 preProcess = "range", 
                 tuneLength = 2,
                 trace = FALSE,
                 maxit = 100)

## Regression Example


lmFit <- train(medv ~ . + rm:lstat,
               data = BostonHousing, 

rpartFit <- train(medv ~ .,
                  data = BostonHousing,
                  tuneLength = 9)

## Example with a custom metric

madSummary <- function (data,
                        lev = NULL,
                        model = NULL) 
  out <- mad(data$obs - data$pred, 
             na.rm = TRUE)  
  names(out) <- "MAD"

robustControl <- trainControl(summaryFunction = madSummary)
marsGrid <- expand.grid(.degree = 1,
                        .nprune = (1:10) * 2)

earthFit <- train(medv ~ .,
                  data = BostonHousing, 
                  tuneGrid = marsGrid,
                  metric = "MAD",
                  maximize = FALSE,
                  trControl = robustControl)

## Parallel Processing Example via multicore package

## library(doMC)
## registerDoMC(2)

## NOTE: don't run models form RWeka when using
### multicore. The session will crash.

## The code for train() does not change:
usingMC <-  train(medv ~ .,
                  data = BostonHousing, 

## or use:
## library(doMPI) or 
## library(doSMP) and so on

Run the code above in your browser using DataLab