mlr (version 2.10)

generateHyperParsEffectData: Generate hyperparameter effect data.

Description

Generate cleaned hyperparameter effect data from a tuning result or from a nested cross-validation tuning result. The object returned can be used for custom visualization or passed downstream to an out of the box mlr method, plotHyperParsEffect.

Usage

generateHyperParsEffectData(tune.result, include.diagnostics = FALSE,
  trafo = FALSE, partial.dep = FALSE)

Arguments

tune.result

[TuneResult | ResampleResult] Result of tuneParams (or resample ONLY when used for nested cross-validation). The tuning result (or results if the output is from nested cross-validation), also containing the optimizer results. If nested CV output is passed, each element in the list will be considered a separate run, and the data from each run will be included in the dataframe within the returned HyperParsEffectData.

include.diagnostics

[logical(1)] Should diagnostic info (eol and error msg) be included? Default is FALSE.

trafo

[logical(1)] Should the units of the hyperparameter path be converted to the transformed scale? This is only useful when trafo was used to create the path. Default is FALSE.

partial.dep

[logical(1)] Should partial dependence be requested based on converting to reg task? This sets a flag so that we know to use partial dependence downstream. This should most likely be set to TRUE if 2 or more hyperparameters were tuned simultaneously. Partial dependence should always be requested when more than 2 hyperparameters were tuned simultaneously. Setting to TRUE will cause plotHyperParsEffect to automatically plot partial dependence when called downstream. Default is FALSE.

Value

[HyperParsEffectData] Object containing the hyperparameter effects dataframe, the tuning performance measures used, the hyperparameters used, a flag for including diagnostic info, a flag for whether nested cv was used, a flag for whether partial dependence should be generated, and the optimization algorithm used.

Examples

## Not run: ------------------------------------
# # 3-fold cross validation
# ps = makeParamSet(makeDiscreteParam("C", values = 2^(-4:4)))
# ctrl = makeTuneControlGrid()
# rdesc = makeResampleDesc("CV", iters = 3L)
# res = tuneParams("classif.ksvm", task = pid.task, resampling = rdesc,
# par.set = ps, control = ctrl)
# data = generateHyperParsEffectData(res)
# plt = plotHyperParsEffect(data, x = "C", y = "mmce.test.mean")
# plt + ylab("Misclassification Error")
# 
# # nested cross validation
# ps = makeParamSet(makeDiscreteParam("C", values = 2^(-4:4)))
# ctrl = makeTuneControlGrid()
# rdesc = makeResampleDesc("CV", iters = 3L)
# lrn = makeTuneWrapper("classif.ksvm", control = ctrl,
#                       resampling = rdesc, par.set = ps)
# res = resample(lrn, task = pid.task, resampling = cv2,
#                extract = getTuneResult)
# data = generateHyperParsEffectData(res)
# plotHyperParsEffect(data, x = "C", y = "mmce.test.mean", plot.type = "line")
## ---------------------------------------------