ts.eval(trues, preds, stats = if (is.null(train.y)) c("mae","mse","rmse","mape") else c("mae","mse","rmse","mape","nmse","nmae","theil"), train.y = NULL)train.y
contains a numeric vector of target variable values (see below).
"mae": mean absolute error, which is calculated as sum(|t_i - p_i|)/N, where t's are the true values and p's are the predictions, while N is supposed to be the size of both vectors.
"mse": mean squared error, which is calculated as sum( (t_i - p_i)^2 )/N
"rmse": root mean squared error that is calculated as sqrt(mse)
The remaining measures ("mape", "nmse", "nmae" and "theil") are relative
measures, the three later
comparing the performance of the model with a baseline. They are
unit-less measures with values always greater than 0. In the case of
"nmse", "nmae" and "theil" the values are expected to be in the interval [0,1]
though occasionaly scores can overcome 1, which means that your model
is performing worse than the baseline model. The baseline used in our
implementation for metrics "nmse" and "nmae" is a constant model that always predicts the average
target variable value, estimated using the values of this variable on
the training data (data used to obtain the model that generated the
predictions), which should be
given in the parameter train.y. The baseline used for
calculating the Theil coefficient ("theil") is the model that predicts
for time t+1 the value of the time series on time t, i.e. the last
known value. The relative error measure
"mape" does not require a baseline. It simply calculates the average
percentage difference between the true values and the
predictions.
These measures are calculated as follows:
"mape": sum(|(t_i - p_i) / t_i|)/N
"nmse": sum( (t_i - p_i)^2 ) / sum( (t_i - AVG(Y))^2 ), where AVG(Y)
is the average of the values provided in vector train.y
"nmae": sum(|t_i - p_i|) / sum(|t_i - AVG(Y)|)
"theil": sum( (t_i - p_i)^2 ) / sum( (t_i - t_[i-1])^2 ), where t_[i-1] is the last known value of the series when we are trying to forecast the value t_i
class.eval
## A few example uses of the function
tr <- rnorm(1000)
true <- rnorm(50)
preds <- rnorm(50)
ts.eval(true,preds)
ts.eval(true,preds,train.y=tr)
ts.eval(true,preds,stats='theil',train.y=tr)
Run the code above in your browser using DataLab