Learn R Programming

MachineShop: Machine Learning Models and Tools for R

Description

MachineShop is a meta-package for statistical and machine learning with a unified interface for model fitting, prediction, performance assessment, and presentation of results. Support is provided for predictive modeling of numerical, categorical, and censored time-to-event outcomes and for resample (bootstrap, cross-validation, and split training-test sets) estimation of model performance. This vignette introduces the package interface with a survival data analysis example, followed by supported methods of variable specification; applications to other response variable types; available performance metrics, resampling techniques, and graphical and tabular summaries; and modeling strategies.

Features

  • Unified and concise interface for model fitting, prediction, and performance assessment.
  • Support for 53+ models from 28 R packages, including model specifications from the parsnip package.
  • Dynamic model parameters.
  • Ensemble modeling with stacked regression and super learners.
  • Modeling of response variables types: binary factors, multi-class nominal and ordinal factors, numeric vectors and matrices, and censored time-to-event survival.
  • Model specification with traditional formulas, design matrices, and flexible pre-processing recipes.
  • Resample estimation of predictive performance, including cross-validation, bootstrap resampling, and split training-test set validation.
  • Parallel execution of resampling algorithms.
  • Choices of performance metrics: accuracy, areas under ROC and precision recall curves, Brier score, coefficient of determination (R2), concordance index, cross entropy, F score, Gini coefficient, unweighted and weighted Cohen’s kappa, mean absolute error, mean squared error, mean squared log error, positive and negative predictive values, precision and recall, and sensitivity and specificity.
  • Graphical and tabular performance summaries: calibration curves, confusion matrices, partial dependence plots, performance curves, lift curves, and model-specific and permutation-based variable importance.
  • Model tuning over automatically generated grids and with exhaustive and random grid searches, Bayesian optimization, particle swarm optimization, quasi-Newton BFGS optimization, simulated annealing, and support for user-defined optimization functions.
  • Model selection and comparisons for any combination of models and model parameter values.
  • Recursive feature elimination.
  • User-definable models and performance metrics.

Getting Started

Installation

# Current release from CRAN
install.packages("MachineShop")

# Development version from GitHub
# install.packages("devtools")
devtools::install_github("brian-j-smith/MachineShop")

# Development version with vignettes
devtools::install_github("brian-j-smith/MachineShop", build_vignettes = TRUE)

Documentation

Once installed, the following R commands will load the package and display its help system documentation. Online documentation and examples are available at the MachineShop website.

library(MachineShop)

# Package help summary
?MachineShop

# Vignette
RShowDoc("UserGuide", package = "MachineShop")

Copy Link

Version

Install

install.packages('MachineShop')

Monthly Downloads

399

Version

3.9.0

License

GPL-3

Maintainer

Brian Smith

Last Published

June 9th, 2025

Functions in MachineShop (3.9.0)

GAMBoostModel

Gradient Boosting with Additive Models
KNNModel

Weighted k-Nearest Neighbor Model
GLMBoostModel

Gradient Boosting with Linear Models
GLMModel

Generalized Linear Model
LDAModel

Linear Discriminant Analysis Model
LARSModel

Least Angle Regression, Lasso and Infinitesimal Forward Stagewise Models
FDAModel

Flexible and Penalized Discriminant Analysis Models
GBMModel

Generalized Boosted Regression Model
ICHomes

Iowa City Home Sales Dataset
GLMNetModel

GLM Lasso or Elasticnet Model
NNetModel

Neural Network Model
NaiveBayesModel

Naive Bayes Classifier Model
ModelSpecification

Model Specification
ModelFrame

ModelFrame Class
LMModel

Linear Models
MLModel

MLModel and MLModelFunction Class Constructors
MDAModel

Mixture Discriminant Analysis Model
MachineShop-package

MachineShop: Machine Learning Models and Tools
MLControl

Resampling Controls
MLMetric

MLMetric Class Constructor
RPartModel

Recursive Partitioning and Regression Tree Models
ParameterGrid

Tuning Parameters Grid
PLSModel

Partial Least Squares Model
QDAModel

Quadratic Discriminant Analysis Model
ParsnipModel

Parsnip Model
RangerModel

Fast Random Forest Model
SVMModel

Support Vector Machine Models
RFSRCModel

Fast Random Forest (SRC) Model
SelectedModel

Selected Model
SelectedInput

Selected Model Inputs
RandomForestModel

Random Forest Model
TreeModel

Classification and Regression Tree Models
POLRModel

Ordered Logistic or Probit Regression Model
SuperModel

Super Learner Model
StackedModel

Stacked Regression Model
TunedInput

Tuned Model Inputs
as.data.frame

Coerce to a Data Frame
dependence

Partial Dependence
as.MLModel

Coerce to an MLModel
TunedModel

Tuned Model
TuningGrid

Tuning Grid Control
diff

Model Performance Differences
calibration

Model Calibration
case_weights

Extract Case Weights
XGBModel

Extreme Gradient Boosting Models
expand_params

Model Parameters Expansion
as.MLInput

Coerce to an MLInput
SurvMatrix

SurvMatrix Class Constructors
SurvRegModel

Parametric Survival Model
combine

Combine MachineShop Objects
confusion

Confusion Matrix
expand_steps

Recipe Step Parameters Expansion
lift

Model Lift Curves
inputs

Model Inputs
metricinfo

Display Performance Metric Information
expand_modelgrid

Model Tuning Grid Expansion
expand_model

Model Expansion Over Tuning Parameters
fit

Model Fitting
print

Print MachineShop Objects
extract

Extract Elements of an Object
quote

Quote Operator
predict

Model Prediction
plot

Model Performance Plots
metrics

Performance Metrics
recipe_roles

Set Recipe Roles
reexports

Objects exported from other packages
models

Models
modelinfo

Display Model Information
set_strata

Resampling Stratification Control
settings

MachineShop Settings
set_monitor

Training Parameters Monitoring Control
resample

Resample Estimation of Model Performance
set_predict

Resampling Prediction Control
rfe

Recursive Feature Elimination
set_optim

Tuning Parameter Optimization
response

Extract Response Variable
performance_curve

Model Performance Curves
performance

Model Performance Metrics
step_kmeans

K-Means Clustering Variable Reduction
step_spca

Sparse Principal Components Analysis Variable Reduction
step_kmedoids

K-Medoids Clustering Variable Selection
varimp

Variable Importance
t.test

Paired t-Tests for Model Comparisons
unMLModelFit

Revert an MLModelFit Object
summary

Model Performance Summaries
step_lincomp

Linear Components Variable Reduction
step_sbf

Variable Selection by Filtering
CForestModel

Conditional Random Forest Model
DiscreteVariate

Discrete Variate Constructors
BlackBoostModel

Gradient Boosting with Regression Trees
AdaBagModel

Bagging with Classification Trees
BARTModel

Bayesian Additive Regression Trees Model
C50Model

C5.0 Decision Trees and Rule-Based Model
BARTMachineModel

Bayesian Additive Regression Trees Model
EarthModel

Multivariate Adaptive Regression Splines Model
AdaBoostModel

Boosting with Classification Trees
CoxModel

Proportional Hazards Regression Model