Perform nonparametric multiple comparisons, across columns, using the Friedman and the post-hoc Nemenyi tests.
nemenyi(data, conf.level = 0.95, sort = c(TRUE, FALSE),
plottype = c("vline", "none", "mcb", "vmcb", "line", "matrix"),
select = NULL, labels = NULL, ...)
an array that includes values to be compared for several treatments (in columns) for several observations (rows), of size n x k. For example, if these are forecast errors, different methods should be in columns and errors for different time series or forecast origins in rows.
the confidence level used for the comparison. Default is 0.95.
if TRUE
, then function sorts the outputted values of mean ranks. If plots are request, this is forced to TRUE
.
type of plot to produce:
"none"
: no plot.
"mcb"
: Multiple Comparison with the Best style plot.
"vmcb"
: vertical MCB plot.
"line"
: summarised line plot.
"vline"
: vertical line plot.
"matrix"
: complete matrix visualisation.
highlight selected treatment (column). Number 1 to k. Use NULL for no highlighting.
optional labels for models. If NULL column names of data
will be used.
additional arguments passed to the plot
function.
Return object of class nemenyi
and contains:
means
: mean rank of each treatment.
intervals
: intervals within there is no evidence of significance difference according to the Nemenyi test at requested confidence level.
fpavl
: Friedman test p-value.
fH
: Friedman test hypothesis outcome.
cd
: Nemenyi critical distance. Output intervals
is calculate as means
+/- cd
.
conf.level
: confidence level used for testing.
k
: number of treatments (columns).
n
: number of observations (rows).
The tests are deailed by Hollander, M., Wolfe, D.A. and Chicken, E. (2014) Nonparametric Statistical Methods. 3rd Edition, John Wiley & Sons, Inc., New York.
The line plot is introduced here and a first example of its use, along with a short description is provided by Kourentzes, N. (2013). Intermittent demand forecasts with neural networks. International Journal of Production Economics, 143(1), 198-206.
The matrix plot is introduced by Kourentzes, N., & Athanasopoulos, G. (2018). Cross-temporal coherent forecasts for Australian tourism (No. 24/18). Monash University, Department of Econometrics and Business Statistics.
The MCB plot is described by Koning, A. J., Franses, P. H., Hibon, M., & Stekler, H. O. (2005). The M3 competition: Statistical tests of the results. International Journal of Forecasting, 21(3), 397-409.
# NOT RUN {
x <- matrix( rnorm(50*4,mean=0,sd=1), 50, 4)
x[,2] <- x[,2]+1
x[,3] <- x[,3]+0.7
x[,4] <- x[,4]+0.5
colnames(x) <- c("Method A","Method B","Method C - long name","Method D")
nemenyi(x,conf.level=0.95,plottype="vline")
# }
Run the code above in your browser using DataLab