Compute metrics of the submissions in the history.
compute_metrics(hist_dir = "history", metrics, y_test, ind_quiz, read_fun)string. directory where the history of the submissions are stored. contains one subdirectory per team.
named list of functions. Each function in the list computes
a performance criterion and is defined as: function(y_pred, y_test)
character or numeric vector. the test set output.
indices of y_test in the quiz subset.
function that reads a submission file and returns a vector of predictions.
compute_metrics returns a named list with one named member per team.
Each member is a data.frame where the rows are the submission files sorted by date
and the columns are:
the date of the submission
the file name of the submission
the score obtained on the quiz subset
the score obtained on the test set