exams_eval: Auxiliary Tools for Evaluating Exams

Description

Generation various helper functions for evaluating exams.

Usage

exams_eval(partial = TRUE, negative = FALSE,
  rule = c("false2", "false", "true", "all", "none"))

Value

exams_eval returns a list with the input parameters

partial, negative, and rule along with the following functions:

checkanswer: function with arguments (correct, answer, and tolerance = 0. It checks whether answer (sufficiently) matches correct or not. It returns 1 for correct, -1 for wrong and 0 for not attempted. In case of partial = TRUE, the functions returns a vector for multiple-choice questions.
pointvec: function with argument correct = NULL. It computes the vector of points for correct and wrong answers, respectively.
pointsum: function with arguments (correct, answer, and tolerance = 0. It computes the overall number of points.

Arguments

partial: logical. Should single/multiple-choice answers be evaluated as a whole pattern (partial = FALSE) or should partial credits be assigned to each of the choices (partial = TRUE)?
negative: logical or numeric. Handling of negative points for an exercise, for details see below.
rule: character specifying which rule to use for negative partial credits.

Details

The function exams_eval is a convenience wrapper for specifying various types of evaluation policies. It returns a set of auxiliary functions that may be useful in the evaluation of exams.

Exercises of types "num" or "string" can essentially be just correct or wrong. In the former case they will give 100 percent of all points, in the latter either 0 percent or some negative percentage can be assigned. If negative percentages are used (e.g., negative = 0.25), then it needs to be distinguished between solved incorrectly and not attempted to solve (which should yield 0 percent).

However, for multiple-choice answers the evaluation policy can either pertain to the answer pattern as a whole (which can be correct or wrong, see above) or it can employ a partial credit strategy. In the latter case, each selected correct choice will yield the fraction 1/ncorrect of points. When an incorrect choice is selected, it should lead to negative points. Five strategies are currently implemented: "false" uses 1/nwrong while "false2" uses 1/max(nwrong, 2); "true" uses 1/ncorrect (so that each wrong selection cancels one correct selection); "all" uses 1 (so that a single wrong selection cancels all correct selections); and "none" uses 0 (so that wrong selections have no effect at all). When aggregating the partial percentages, the overall points can become negative. By setting negative a lower bound can be set: negative = TRUE sets no bound while negative = FALSE sets the bound to zero. Any other numeric value could be set as well, e.g., negative = 0.25.

The functions returned by exams_eval internally just distinguish between num, string, and mchoice answers. Thus, if evaluations for schoice or cloze exercises are required, these have to be built by appropriately reusing the building blocks for num/string/mchoice. For example, the components of cloze exercises have to be evaluated individually and then aggregated as desired. Or, if a distinction between mchoice and schoice regarding partial credits is needed, one evaluation has to be set up with partial = TRUE and the other with partial = FALSE. Different evaluations for different item types may be set as in: exams2qti12(..., eval = eval1, schoice = list(eval = eval2)). Then eval = eval1 is used as the default for all exercise types except schoice where eval = eval2 is used.

Thus, exams_eval might not give the complete finished evaluation policy for an entire exam but supplies the most important building blocks for setting this up “by hand”. Internally, exams_eval is also used by exams2moodle, exams2qti12 and exams2blackboard for writing the evaluation specifications in the respective XML specifications.

Examples

Run this code

## binary evaluation policy with solutions being either correct
## or wrong: partial = FALSE, negative = FALSE
ee <- exams_eval(partial = FALSE, negative = FALSE)

## points that can be achieved are 0/1
ee$pointvec()

## checkanswer() returns 1 for correct, -1 for incorrect and 0 for missing answer
ee$checkanswer(1.23, 1.23)
ee$checkanswer(1.23, "1.23")
ee$checkanswer(1.23, "1,23")
ee$checkanswer(1.23, 1.24)
ee$checkanswer(1.23, 1.24, tolerance = 0.01)
ee$checkanswer(1.23, NA)
ee$checkanswer(1.23, NULL)
ee$checkanswer(1.23, "")

## similarly for logical (mchoice/schoice) answers
## (which allows either string or logical specification)
ee$checkanswer("10000", "10000")
ee$checkanswer(c(TRUE, FALSE, FALSE, FALSE, FALSE), c(TRUE, FALSE, FALSE, FALSE, FALSE))
ee$checkanswer(c(TRUE, FALSE, FALSE, FALSE, FALSE), "10000")
ee$checkanswer("10000", "01000")
ee$checkanswer("10000", "11000")

## and analogously for strings
ee$checkanswer("foo", "foo")
ee$checkanswer("foo", "bar")
ee$checkanswer("foo", "")

## obtain points achieved
ee$pointsum("10000", "10000")
ee$pointsum("10000", "01000")
ee$pointsum("10000", "00000")
ee$pointsum("10000", NA)

## ---------------------------------------------------------
## evaluation policy with -25% penalty for wrong answers
ee <- exams_eval(partial = FALSE, negative = -0.25)

## points that can be achieved are 1/-0.25 (or zero)
ee$pointvec()

## obtain points achieved
ee$pointsum("10000", "10000")
ee$pointsum("10000", "01000")
ee$pointsum("10000", "00000")
ee$pointsum("10000", NA)
ee$pointsum(1.23, 1.23)
ee$pointsum(1.23, 2.34)
ee$pointsum(1.23, NA)
ee$pointsum(1.23, 1.24)
ee$pointsum(1.23, 1.24, tolerance = 0.1)

## ---------------------------------------------------------
## default evaluation policy with partial points
## (but without negative points overall)
ee <- exams_eval()

## points that can be achieved are 1/3 (1/#true)
## or -1/2 (1/#false)
ee$pointvec("10101")

## obtain points achieved
ee$pointsum("10101", "10101")
ee$pointsum("10101", "10100")
ee$pointsum("10101", "11100")
ee$pointsum("10101", "01010")
ee$pointsum("10101", "00000")

## show individual answer check
ee$checkanswer("10101", "10101")
ee$checkanswer("10101", "10100")
ee$checkanswer("10101", "11100")
ee$checkanswer("10101", "01010")
ee$checkanswer("10101", "00000")

## numeric/string answers are not affected by partial=TRUE
ee$checkanswer(1.23, 1.23)
ee$pointsum(1.23, 1.23)
ee$checkanswer(1.23, 2.34)
ee$pointsum(1.23, 2.34)

## ---------------------------------------------------------
## evaluation policy with partial points
## (and with up to -25% negative points overall)
ee <- exams_eval(partial = TRUE, negative = -0.25)

## points that can be achieved are 1/3 (1/#true)
## or -1/2 (1/#false)
ee$pointvec("10101")

## obtain points achieved
ee$pointsum("10101", "10101")
ee$pointsum("10101", "01010")
ee$pointsum("10101", "00000")

## show individual answer check
ee$checkanswer("10101", "10101")
ee$checkanswer("10101", "10100")
ee$checkanswer("10101", "11100")
ee$checkanswer("10101", "01010")
ee$checkanswer("10101", "00000")

## numeric/string answers are not affected by partial=TRUE
ee$pointsum(1.23, 1.23)
ee$pointsum(1.23, 2.34)