tinytest v1.1.0

0

Monthly downloads

0th

Percentile

Lightweight and Feature Complete Unit Testing Framework

Provides a lightweight (zero-dependency) and easy to use unit testing framework. Main features: install tests with the package. Test results are treated as data that can be stored and manipulated. Test files are R scripts interspersed with test commands, that can be programmed over. Fully automated build-install-test sequence for packages. Skip tests when not run locally (e.g. on CRAN). Flexible and configurable output printing. Compare computed output with output stored with the package. Run tests in parallel. Extensible by other packages. Report side effects.

Readme

A brief overview of tinytest

Package setup

A quick way to set things up is as follows.

tinytest::setup_tinytest("pkgdir")

where pkgdir is a package source directory with a valid DESCRIPTION file

The setup is as follows.

  1. Files having names starting with test are in pkg/inst/tinytest, e.g. test_haha.R. Test files are R scripts interspersed with test commands, such as expect_equal(myfunc(1), 0).
  2. tinytest is added to Suggests: in the DESCRIPTION file.
  3. A file named tinytest.R is set up in pkg/tests to make sure that tests will be run by R CMD check.

A nice way to set up a completely new package that passes R CMD check is as follows

pkgKitten::kitten("hihi")
tinytest::setup_tinytest("hihi")

where hihi is the name of the new package.

Interactive package testing

Function description
test_all("pkgdir") run all test files (pkg must be loaded).
build_install_test("pkgdir") build, install, and test in temp dir.
run_test_dir("pkgdir") run all test files in a directory (pkg must be loaded).
run_test_file("testfile") run a single test file (pkg must be loaded).

All functions return an object of class tinytests. Results can be printed to screen, summarized with summary or converted to data frame with as.data.frame for analyses. The option verbose (default: 2) controls showing test progress in the terminal.

Test functions

The syntax of test functions resembles that of testthat. For expectations comparing two results, the first argument represents the observed value while the second argument represents the desired value.

Function description
expect_true Argument must evaluate to TRUE
expect_false Argument must evaluate to FALSE
expect_equal Data and attributes of arguments must be equal
expect_equivalent Data of arguments must be equal
expect_identical Target and current must be identical
expect_null Expression must evaluate to NULL
expect_message Expression must yield a message
expect_warning Expression must yield a warning
expect_error Expression must yield an error
expect_silent Expect no errors, no warnings

For tests in a script there is an alternative syntax in the style of RUnit. For each function of the form expect_lol there is a function of the form checkLol.

Monitor side-effects

Side-effects, such as changing environment variables or changing the working directory can cause hard-to-trace bugs. Add the statement

report_side_effects()

to a test file and certain types of side-effects, if any, are reported.

Alternatively, use the side_effects argument to any of the test runners, for example

test_all("/path/to/package", side_effects=TRUE)

Test results (objects of class tinytests) have two printing modes: a long format and a short, one-line format. Information that is always shown includes:

  • File name and line number of failed test.
  • The test call that resulted in test failure.
  • The type of failure. This can be 'data' (for differences in variable content), 'attr' (for differences in attributes like column names), or 'xcpt' for exceptions (warnings, errors).

In long format, the test call and difference between desired and realized input are shown in full. Global printing options can be set with options(option=value).

Option default description
tt.pr.passes FALSE print passing tests?
tt.pr.limit 10 how many results to print?
tt.pr.nlong 3 how many tests in long format?
tt.pr.color TRUE print colored output?

It is also possible to influence these options using print.tinytest. Colored output is suppressed on systems with a "dumb" terminal.

Run tests for an installed package

For a package called haha that is tested with tinytest, any user that has haha and tinytest installed can run tests as follows.

tinytest::test_package("haha")

Run tests in parallel

Run tests in parallel over files.

tinytest::test_package("haha", ncpu=3)

Or, for more control:

cl <- parallel::makeCluster(4)
parallel::clusterCall(cl, source, "R/functions.R")
test_all(cluster=cl)
stopCluster(cl)

Use extension packages

Add the following to a test file to use assertions exported by checkmate.tinytest.

using(checkmate.tinytest)

Skipping or ignoring tests

Use exit_file() to stop executing a test file, with an optional message.

exit_file("I'm too tired to test today")

Use ignore(testfunction) to run a test but not include the result in the output.

# both tests run, but only second is recorded.
if ( ignore(expect_equal)(1 + 1, 2) ){
  expect_true( 1 > 0 )
}

Note the placement of brackets.

Use at_home() to detect whether a test is running interactively, or via test_package() (i.e. the way R CMD check will run it).

if ( at_home() ){
  # run tests requiring lots of time.
}

The package vignette has some tips on how to use this feature, and how you can set up your package so R CMD check also runs tests protected by at_home() in your environment.

Comparing with data stored on file

Data can be loaded from pkg/inst/tinytest (or subdirectories). A simple test file might look like this.

desired <- read.csv("mycsvoutput.csv", stringsAsFactors=FALSE)
obtained <- compute_my_result()
expect_equal(obtained, desired)

If you wish to publish the package on CRAN, make sure that the files are small enough for the package to be acceptable. See the cran repository policy for explicit bounds on package size. Alternatively you can avoid installing the data and associated test files by adding them to .Rbuildignore.

More information

See the vignette.

vignette("using_tinytest", package="tinytest")

Functions in tinytest

Name Description
using Use an extension package.
run_test_dir Run all tests in a directory
register_tinytest_extension Register or unregister extension functions
run_test_file Run an R file containing tests; gather results
puppy The puppy for a pkgKitten
setup_tinytest Add tinytest to package source directory
build_install_test build, install and test
test_package Test a package during R CMD check or after installation
tinytest Tinytest constructor
ignore Ignore the output of an expectation
report_side_effects Report side effects for expressions in test files
exit_file Stop testing
at_home Detect not on CRANity
expect_equal Express expectations
summary.tinytests Tinytests object
format.tinytest Print a tinytest object
No Results!

Vignettes of tinytest

Name
test_addOne.R
test_hehe.R
test_se.R
tinytest_examples.Rnw
using_tinytest.Rnw
No Results!

Last month downloads

Details

License GPL-3
LazyData no
Type Package
LazyLoad yes
URL https://github.com/markvanderloo/tinytest
BugReports https://github.com/markvanderloo/tinytest/issues
RoxygenNote 6.1.1
Encoding UTF-8
NeedsCompilation no
Packaged 2019-09-26 14:58:33 UTC; mark
Repository CRAN
Date/Publication 2019-09-26 15:20:02 UTC
imports parallel , utils
Contributors

Include our badge in your README

[![Rdoc](http://www.rdocumentation.org/badges/version/tinytest)](http://www.rdocumentation.org/packages/tinytest)