vdiffr (version 0.3.0)

expect_doppelganger: Does a figure look like its expected output?

Description

expect_doppelganger() takes a figure to check visually.

  • If the figure has yet to be validated, the test is skipped. Call manage_cases() to validate the new figure, so vdiffr knows what to compare against.

  • If the test has been validated, fig is compared to the validated figure. If the plot differs, a failure is issued (except on CRAN, see section on regression testing below).

    Either fix the problem, or call manage_cases() to validate the new figure appearance.

Usage

expect_doppelganger(title, fig, path = NULL, ..., verbose = NULL,
  writer = write_svg)

Arguments

title

A brief description of what is being tested in the figure. For instance: "Points and lines overlap".

If a ggplot2 figure doesn't have a title already, title is applied to the figure with ggtitle().

The title is also used as file name for storing SVG (in a sanitzed form, with special characters converted to "-").

fig

A figure to test. This can be a ggplot object, a recordedplot, or more generally any object with a print method.

For plots that can't be represented as printable objects, you can pass a function. This function must construct the plot and print it.

path

The path where the test case should be stored, relative to the tests/figs/ folder. If NULL (the default), the current testthat context is used to create a subfolder. Supply an empty string "" if you want the figures to be stored in the root folder.

...

Additional arguments passed to testthat::compare() to control specifics of comparison.

verbose

Soft-deprecated. See the debugging section.

writer

A function that takes the plot, a target SVG file, and an optional plot title. It should transform the plot to SVG in a deterministic way and write it to the target file. See write_svg() (the default) for an example.

Regression testing versus Unit testing

Failures to match a validated appearance are only reported when the tests are run locally, on Travis, Appveyor, or any environment where the Sys.getenv("CI") or Sys.getenv("NOT_CRAN") variables are set. Because vdiffr is more of a monitoring than a unit testing tool, it shouldn't cause R CMD check failures on the CRAN machines.

Checking the appearance of a figure is inherently fragile. It is similar to testing for errors by matching exact error messages: these messages are susceptible to change at any time. Similarly, the appearance of plots depends on a lot of upstream code, such as the way margins and spacing are computed. vdiffr uses a special ggplot2 theme that should change very rarely, but there are just too many upstream factors that could cause breakages. For this reason, figure mismatches are not necessarily representative of actual failures.

Visual testing is not an alternative to writing unit tests for the internal data transformations performed during the creation of your figure. It is more of a monitoring tool that allows you to quickly check how the appearance of your figures changes over time, and to manually assess whether changes reflect actual problems in your package.

If you need to override the default vdiffr behaviour on CRAN (not recommended) or Travis (for example to run the tests in a particular builds but not others), set the VDIFFR_RUN_TESTS environment variable to "true" or "false".

Debugging

It is sometimes difficult to understand the cause of a failure. This usually indicates that the plot is not created deterministically. Potential culprits are:

  • Some of the plot components depend on random variation. Try setting a seed.

  • The plot depends on some system library. For instance sf plots depend on libraries like GEOS and GDAL. It might not be possible to test these plots with vdiffr (which can still be used for manual inspection, add a testthat::skip() before the expect_doppelganger() call in that case).

To help you understand the causes of a failure, vdiffr automatically logs the SVG diff of all failures when run under R CMD check. The log is located in tests/vdiffr.Rout.fail and should be displayed on Travis.

You can also set the VDIFFR_LOG_PATH environment variable with Sys.setenv() to unconditionally (also interactively) log failures in the file pointed by the variable.

Examples

Run this code
# NOT RUN {
if (FALSE) {  # Not run

library("ggplot2")

test_that("plots have known output", {
  disp_hist_base <- function() hist(mtcars$disp)
  expect_doppelganger("disp-histogram-base", disp_hist_base)

  disp_hist_ggplot <- ggplot(mtcars, aes(disp)) + geom_histogram()
  expect_doppelganger("disp-histogram-ggplot", disp_hist_ggplot)
})

}
# }

Run the code above in your browser using DataCamp Workspace