Cache method that accommodates environments, S4 methods, Rasters, & nested caching
The special assign operator %<% is equivalent to Cache. See examples at the end.
Cache(FUN, ..., notOlderThan = NULL, objects = NULL, outputObjects = NULL,
algo = "xxhash64", cacheRepo = NULL, length = 1e+06,
compareRasterFileLength, userTags = c(), digestPathContent,
omitArgs = NULL, classOptions = list(), debugCache = character(),
sideEffect = FALSE, makeCopy = FALSE,
quick = getOption("reproducible.quick", FALSE),
verbose = getOption("reproducible.verbose", FALSE), cacheId = NULL,
useCache = getOption("reproducible.useCache", TRUE), showSimilar = NULL)# S4 method for ANY
Cache(FUN, ..., notOlderThan = NULL, objects = NULL,
outputObjects = NULL, algo = "xxhash64", cacheRepo = NULL,
length = 1e+06, compareRasterFileLength, userTags = c(),
digestPathContent, omitArgs = NULL, classOptions = list(),
debugCache = character(), sideEffect = FALSE, makeCopy = FALSE,
quick = getOption("reproducible.quick", FALSE),
verbose = getOption("reproducible.verbose", FALSE), cacheId = NULL,
useCache = getOption("reproducible.useCache", TRUE), showSimilar = NULL)
lhs %
Either a function or an unevaluated function call (e.g., using
quote.
Arguments of FUN function .
load an artifact from the database only if it was created after notOlderThan.
Character vector of objects to be digested. This is only applicable if there is a list, environment or simList with named objects within it. Only this/these objects will be considered for caching, i.e., only use a subset of the list, environment or simList objects.
Optional character vector indicating which objects to
return. This is only relevant for simList objects
The algorithms to be used; currently available choices are
md5, which is also the default, sha1, crc32,
sha256, sha512, xxhash32, xxhash64 and
murmur32.
A repository used for storing cached objects.
This is optional if Cache is used inside a SpaDES module.
Numeric. If the element passed to Cache is a Path class
object (from e.g., asPath(filename)) or it is a Raster with file-backing, then this will be
passed to digest::digest, essentially limiting the number of bytes
to digest (for speed). This will only be used if quick = FALSE.
Being deprecated; use length.
A character vector with Tags. These Tags will be added to the repository along with the artifact.
Being deprecated. Use quick.
Optional character string of arguments in the FUN to omit from the digest.
Optional list. This will pass into .robustDigest for
specific classes. Should be options that the .robustDigest knows what
to do with.
Character or Logical. Either "complete" or "quick" (uses
partial matching, so "c" or "q" work). TRUE is
equivalent to "complete".
If "complete", then the returned object from the Cache
function will have two attributes, debugCache1 and debugCache2,
which are the entire list(...) and that same object, but after all
.robustDigest calls, at the moment that it is digested using
fastdigest, respectively. This attr(mySimOut, "debugCache2")
can then be compared to a subsequent call and individual items within
the object attr(mySimOut, "debugCache1") can be compared.
If "quick", then it will return the same two objects directly,
without evalutating the FUN(...).
Logical or path. Determines where the function will look for new files following function completion. See Details. NOTE: this argument is experimental and may change in future releases.
Logical. If sideEffect = TRUE, and makeCopy = TRUE,
a copy of the downloaded files will be made and stored in the cacheRepo
to speed up subsequent file recovery in the case where the original copy
of the downloaded files are corrupted or missing. Currently only works when
set to TRUE during the first run of Cache. Default is FALSE.
NOTE: this argument is experimental and may change in future releases.
Logical. If TRUE,
little or no disk-based information will be assessed, i.e., mostly its
memory content. This is relevant for objects of class
Path and Raster currently.
For class Path objects, the file's metadata (i.e., filename and file size)
will be hashed instead of the file contents. If set to FALSE (default),
the contents of the file(s) are hashed.
If quick = TRUE, length is ignored.
NOTE: this argument is experimental and may change in future releases.
Logical. This will output much more information about the internals of Caching, which may help diagnose Caching challenges.
Character string. If passed, this will override the calculated hash of the inputs, and return the result from this cacheId in the cacheRepo. In general, this is not used; however, in some particularly finicky situations where Cache is not correctly detecting unchanged inputs, this can stabilize the return value.
Logical. If FALSE, then the entire Caching mechanism is bypassed
and the function is evaluated as if it was not being Cached.
Default is getOption("reproducible.useCache")),
which is FALSE by default, meaning use the Cache mechanism. This
may be useful to turn all Caching on or off in very complex scripts and
nested functions.
A logical or numeric. Useful for debugging.
If TRUE or 1, then if the Cache
does not find an identical archive in the cacheRepo, it will report (via message)
the next most similar archive, and indicate which argument(s) is/are different.
If a number larger than 1, then it will report the N most similar archived
objects.
A name to assign to.
A function call
As with cache, returns the value of the
function call or the cached version (i.e., the result from a previous call
to this same cached function with identical arguments).
Commonly, Caching is nested, i.e., an outer function is wrapped in a Cache
function call, and one or more inner functions are also wrapped in a Cache
function call. A user can always specify arguments in every Cache function
call, but this can get tedious and can be prone to errors. The normal way that
R handles arguments is it takes the user passed arguments if any, and
default arguments for all those that have no user passed arguments. We have inserted
a middle step. The order or precedence for any given Cache function call is
1. user arguments, 2. inherited arguments, 3. default arguments. At this time,
the top level Cache arguments will propagate to all inner functions unless
each individual Cache call has other arguments specified, i.e., "middle"
nested Cache function calls don't propagate their arguments to further "inner"
Cache function calls. See example.
userTags is unique of all arguments: its values will be appended to the
inherited userTags.
Caching speed may become a critical aspect of a final product. For example,
if the final product is a shiny app, rerunning the entire project may need
to take less then a few seconds at most. There are 3 arguments that affect
Cache speed: quick, length, and
algo. quick is passed to .robustDigest, which currently
only affects Path and Raster* class objects. In both cases, quick
means that little or no disk-based information will be assessed.
If a function has a path argument, there is some ambiguity about what should be done. Possibilities include:
hash the string as is (this will be very system specific, meaning a
Cache call will not work if copied between systems or directories);
hash the basename(path);
hash the contents of the file.
If paths are passed in as is (i.e,. character string), the result will not be predictable.
Instead, one should use the wrapper function asPath(path), which sets the
class of the string to a Path, and one should decide whether one wants
to digest the content of the file (using quick = FALSE),
or just the filename ((quick = TRUE)). See examples.
In general, it is expected that caching will only be used when stochasticity
is not relevant, or if a user has achieved sufficient stochasticity (e.g., via
sufficient number of calls to experiment) such that no new explorations
of stochastic outcomes are required. It will also be very useful in a
reproducible workflow.
If sideEffect is not FALSE, then metadata about any files that
added to sideEffect will be added as an attribute to the cached copy.
Subsequent calls to this function
will assess for the presence of the new files in the sideEffect location.
If the files are identical (quick = FALSE) or their file size is
identical (quick = TRUE), then the cached copy of the function will
be returned (and no files changed). If there are missing or incorrect files,
then the function will re-run. This will accommodate the situation where the
function call is identical, but somehow the side effect files were modified.
If sideEffect is logical, then the function will check the
cacheRepo; if it is a path, then it will check the path. The function will
assess whether the files to be downloaded are found locally
prior to download. If it fails the local test, then it will try to recover from a
local copy if (makeCopy had been set to TRUE the first time
the function was run. Currently, local recovery will only work ifmakeCOpy was
set to TRUE the first time Cache
was run). Default is FALSE.
Caching R objects using cache has five important limitations:
the archivist package detects different environments as different;
it also does not detect S4 methods correctly due to method inheritance;
it does not detect objects that have file-base storage of information
(specifically RasterLayer-class objects);
the default hashing algorithm is relatively slow.
heavily nested function calls may want Cache arguments to propagate through
This version of the Cache function accommodates those four special,
though quite common, cases by:
converting any environments into list equivalents;
identifying the dispatched S4 method (including those made through inheritance) before hashing so the correct method is being cached;
by hashing the linked file, rather than the Raster object.
Currently, only file-backed Raster* objects are digested
(e.g., not ff objects, or any other R object where the data
are on disk instead of in RAM);
using fastdigest internally when the object
is in RAM, which can be up to ten times faster than
digest. Note that file-backed objects are still
hashed using digest.
Cache will save arguments passed by user in a hidden environment. Any nested Cache functions will use arguments in this order 1) actual arguments passed at each Cache call, 2) any inherited arguments from an outer Cache call, 3) the default values of the Cache function. See section on Nested Caching.
If Cache is called within a SpaDES module, then the cached entry will automatically
get 3 extra userTags: eventTime, eventType, and moduleName.
These can then be used in clearCache to selectively remove cached objects
by eventTime, eventType or moduleName.
Cache will add a tag to the artifact in the database called accessed,
which will assign the time that it was accessed, either read or write.
That way, artifacts can be shown (using showCache) or removed (using
clearCache) selectively, based on their access dates, rather than only
by their creation dates. See example in clearCache.
Cache (uppercase C) is used here so that it is not confused with, and does
not mask, the archivist::cache function.
# NOT RUN {
tmpDir <- file.path(tempdir())
# Basic use
ranNumsA <- Cache(rnorm, 10, 16, cacheRepo = tmpDir)
# All same
ranNumsB <- Cache(rnorm, 10, 16, cacheRepo = tmpDir) # recovers cached copy
ranNumsC <- rnorm(10, 16) %>% Cache(cacheRepo = tmpDir) # recovers cached copy
ranNumsD <- Cache(quote(rnorm(n = 10, 16)), cacheRepo = tmpDir) # recovers cached copy
# For more in depth uses, see vignette
# }
# NOT RUN {
browseVignettes(package = "reproducible")
# }
# NOT RUN {
# Equivalent
a <- Cache(rnorm, 1)
b %<% rnorm(1)
# }
Run the code above in your browser using DataLab