modeltime.resample (version 0.3.0)

modeltime_fit_resamples: Fits Models in a Modeltime Table to Resamples

Description

Resampled predictions are commonly used for:

1. Analyzing accuracy and stability of models

2. As inputs to Ensemble methods (refer to the modeltime.ensemble package)

Usage

modeltime_fit_resamples(object, resamples, control = control_resamples())

Value

A Modeltime Table (mdl_time_tbl) object with a column containing resample results (.resample_results)

Arguments

object

A Modeltime Table

resamples

An rset resample object. Used to generate sub-model predictions for the meta-learner. See timetk::time_series_cv() or rsample::vfold_cv() for making resamples.

control

A tune::control_resamples() object to provide control over the resampling process.

Details

The function is a wrapper for tune::fit_resamples() to iteratively train and predict models contained in a Modeltime Table on resample objects. One difference between tune::fit_resamples() and modeltime_fit_resamples() is that predictions are always returned (i.e. control = tune::control_resamples(save_pred = TRUE)). This is needed for ensemble_model_spec().

Resampled Prediction Accuracy

Calculating Accuracy Metrics on models fit to resamples can help to understand the model performance and stability under different forecasting windows. See modeltime_resample_accuracy() for getting resampled prediction accuracy for each model.

Ensembles

Fitting and Predicting Resamples is useful in creating Stacked Ensembles using modeltime.ensemble::ensemble_model_spec(). The sub-model cross-validation predictions are used as the input to the meta-learner model.

Examples

library(tidymodels)
library(modeltime)
library(timetk)
library(magrittr)

# Make resamples
resamples_tscv <- training(m750_splits) %>%
    time_series_cv(
        assess      = "2 years",
        initial     = "5 years",
        skip        = "2 years",
        # Normally we do more than one slice, but this speeds up the example
        slice_limit = 1
    )

# \donttest{
# Fit and generate resample predictions
m750_models_resample <- m750_models %>%
    modeltime_fit_resamples(
        resamples = resamples_tscv,
        control   = control_resamples(verbose = TRUE)
    )

# A new data frame is created from the Modeltime Table
#  with a column labeled, '.resample_results'
m750_models_resample
# }