MARSS (version 3.11.3)

accuracy.marssMLE: Return accuracy metrics

Description

This is a method for the generic accuracy function in the forecast package. The forecast package is not part of the MARSS package and you will need to load it separately (See examples). Alternatively you can use accuracy.marssMLE() or accuracy.marssPredict().

The measures calculated are:

• ME: Mean Error

• RMSE: Root Mean Squared Error

• MAE: Mean Absolute Error

• MPE: Mean Percentage Error

• MAPE: Mean Absolute Percentage Error

• MASE: Mean Absolute Scaled Error

• ACF1: Autocorrelation of errors at lag 1.

The MASE calculation is scaled using MAE of the training set naive forecasts which are simply \(\mathbf{y}_{t-1}\).

For the training data, the metrics are shown for the one-step-ahead predictions by default (type="ytt1"). This is the prediction of \(\mathbf{y}_t\) conditioned on the data up to \(t-1\) (and the model estimated from all the data). With type="ytT", you can compute the metrics for the fitted ytT, which is the expected value of new data at \(t\) conditioned on all the data. type does not affect test data (forecasts are past the end of the training data).

Usage

accuracy.marssPredict(f, x, test=NULL, type="ytt1", verbose=FALSE, ...)
accuracy.marssMLE(f, x, test=NULL, type="ytt1", verbose=FALSE, ...)

Arguments

f

A marssMLE or marssPredict object

x

A matrix or data frame with data to test against the h steps of a forecast.

test

Which time steps in training data (data model fit to) to compute accuracy for.

type

type="ytt1" is the one-step-ahead predictions. type="ytT" is the fitted ytT predictions. The former are standardly used for training data prediction metrics.

verbose

Show metrics for each time series of data.

...

Not used.

References

Hyndman, R.J. and Koehler, A.B. (2006) "Another look at measures of forecast accuracy". International Journal of Forecasting, 22(4), 679-688. Hyndman, R.J. and Athanasopoulos, G. (2018) "Forecasting: principles and practice", 2nd ed., OTexts, Melbourne, Australia. Section 3.4 "Evaluating forecast accuracy". https://otexts.org/fpp2/accuracy.html.

Examples

# NOT RUN {
dat <- t(harborSeal)
dat <- dat[c(2, 11, 12),]
train.dat <- dat[,1:12]
fit <- MARSS(train.dat, model = list(Z = factor(c("WA", "OR", "OR"))))

library(forecast)
accuracy(fit)

# Compare to test data set
fr <- predict(fit, n.ahead=10)
test.dat <- dat[,13:22]
accuracy(fr, x=test.dat)
# }