plotEvol: Plot for the evolution of model selection algorithm in funGp

Description

This method displays the evolution of the quality of the configurations evaluated along the iterations, by the model selection algorithm in the fgpm_factory function. For each iteration, the performance statistic of all the evaluated models is printed, along with the corresponding median of the group. The plot also includes the global maximum, which corresponds to the best performance statistic obtained up to the current iteration. In this plot, it is typical to have some points falling relatively far from the maximum, even after multiple iterations. This happens mainly because we have multiple categorical features, whose alteration might change the performance statistic in a nonsmooth way. On the other hand, the points that fall bellow zero usually correspond to models whose hyperparameters were hard to optimize. This occurs sporadically during the log-likelihood optimization for Gaussian processes, due to the non-linearity of the objective function. As long as the maximum keeps improving and the median remains close to it, none of the two aforementioned phenomena is matter for worries. Both of them respond to the mechanism of exploration implemented in the algorithm, which makes it able to progressively move towards better model configurations.

Usage

# S4 method for Xfgpm
plotEvol(x.model, ...)

Arguments

x.model

an object of class '>Xfgpm containing the output of the model selection algorithm in fgpm_factory.

...

additional arguments affecting the plot. The following typical graphics parameters are valid entries: xlim, ylim, xlab, ylab, main.

Value

None.

References

Betancourt, J., Bachoc, F., and Klein, T., and Gamboa, F. (2020), Technical Report: "Ant Colony Based Model Selection for Functional-Input Gaussian Process Regression. Ref. D3.b (WP3.2)". RISCOPE project. [HAL]

Betancourt, J., Bachoc, F., and Klein, T. (2020), R Package Manual: "Gaussian Process Regression for Scalar and Functional Inputs with funGp - The in-depth tour". RISCOPE project. [HAL]

Examples

Run this code

# NOT RUN {
# generating input and output data
set.seed(100)
n.tr <- 2^5
sIn <- expand.grid(x1 = seq(0,1,length = n.tr^(1/5)), x2 = seq(0,1,length = n.tr^(1/5)),
                   x3 = seq(0,1,length = n.tr^(1/5)), x4 = seq(0,1,length = n.tr^(1/5)),
                   x5 = seq(0,1,length = n.tr^(1/5)))
fIn <- list(f1 = matrix(runif(n.tr*10), ncol = 10), f2 = matrix(runif(n.tr*22), ncol = 22))
sOut <- fgp_BB7(sIn, fIn, n.tr)
# }
# NOT RUN {
# optimizing the model structure with fgpm_factory (~5 seconds)
xm <- fgpm_factory(sIn = sIn, fIn = fIn, sOut = sOut)

# assessing the evolution of the algorithm
plotEvol(xm)
# }
# NOT RUN {
# }