performanceEstimation (version 1.1.0)

getIterationsInfo: Obtaining the information returned by a workflow when applied to a task, on a particular iteration of the estimation process or on all iterations

Description

In the estimation process workflows are applied many times to different train+test samples of each task. We call these repetitions, the iterations of the estimation process. On each of these executions of the workflows, they typically return not only the predictions for the test set but also some extra information, in the form of a list. This function allows you to inspect this extra information

Usage

getIterationsInfo(obj, workflow = 1, task = 1, rep, fold, it)

Arguments

obj

A ComparisonResults object

workflow

A string with the ID of a workflow (it can also be an integer). It defaults to 1 (the first workflow of the estimation experiment)

task

A string with the ID of a task (it can also be an integer). It defaults to 1 (the first task of the estimation experiment)

rep

An integer representing the repetition, which allows you to identify the iteration you want to inspect. You need to specify either this argument together with the argument fold, or only the argument it

fold

An integer representing the fold, which allows you to identify the iteration you want to inspect. You need to specify either this argument together with the argument rep, or only the argument it

it

An integer representing the iteration you want to inspect. Alternatively, for cross validation experiments, you may instead specify the repetition id and the fold id (arguments rep and fold, respectivily)

Value

A list with the information returned by the workflow on the selected iteration or a list containing as many components as there are iterations (i.e. a list of lists) in case you have opted for obtaining all iterations results

References

Torgo, L. (2014) An Infra-Structure for Performance Estimation and Experimental Comparison of Predictive Models in R. arXiv:1412.0436 [cs.MS] http://arxiv.org/abs/1412.0436

See Also

getIterationsPreds, getScores, performanceEstimation

Examples

## Not run: 
# ## Estimating MSE for 3 variants of both
# ## regression trees and SVMs, on  two data sets, using one repetition
# ## of 10-fold CV
# library(e1071)
# data(swiss)
# 
# ## running the estimation experiment
# res <- performanceEstimation(
#   PredTask(Infant.Mortality ~ .,swiss),
#   workflowVariants(learner="svm",
#                    learner.pars=list(cost=c(1,10),gamma=c(0.01,0.5))),
#   EstimationTask("mse",method=CV(nReps=2,nFolds=5))
#   )
# 
# ## Get the iterations scores of svm.v2 on swiss
# getIterationsInfo(res,"svm.v2","swiss.Infant.Mortality",rep=1,fold=2)
# ## this would get the same
# getIterationsInfo(res,"svm.v2","swiss.Infant.Mortality",it=2)
# 
# getIterationsInfo(res,"svm.v2","swiss.Infant.Mortality",rep=2,fold=3)
# ## this would get the same
# getIterationsInfo(res,"svm.v2","swiss.Infant.Mortality",it=8)
# 
# ## Get the results of all iterations
# getIterationsInfo(res,"svm.v1","swiss.Infant.Mortality")
# 
# ## End(Not run)