Empirical results are often contingent on analytical decisions that
are equally defensible, often arbitrary, and motivated by different reasons.
This decisions may introduce bias or at least variability. To this end,
specification curve analyses (Simonsohn et al., 2020) or multiverse
analyses (Steegen et al., 2016) refer to identifying the set of
theoretically justified, statistically valid (and potentially also non-redundant
specifications, fitting the "multiverse" of models represented by these
specifications and extract relevant parameters often to display the results
graphically as a so-called specification curve. This allows readers to
identify consequential specifications decisions and how they affect the results
or parameter of interest.
Use of this function
A general overview is provided in the vignettes vignette("specr").
Generally, you create relevant specification using the function setup().
You then pass the resulting object of a class specr.setup to the
present function specr() to run the specification curve analysis.
Further note that the resulting object of class specr.object allows
to use several generic function such as summary() or plot().
Use methods(class = "specr.object") for an overview on available
methods and e.g., ?plot.specr.object to view the dedicated help page.
Parallelization
By default, the function fits models across all specifications sequentially
(one after the other). If the data set is large, the models complex (e.g.,
large structural equation models, negative binomial models, or Bayesian models),
and the number of specifications is large, it can make sense to parallelize
these operations. One simply has to load the package furrr (which
in turn, builds on future) up front. Then parallelizing the fitting process
works as specified in the package description of furr/future by setting a
"plan" before running specr such as:
plan(multisession, workers = 4)
However, there are many more ways to specifically set up the plan, including
different strategy than multisession. For more information, see
vignette("parallelization") and the
reference page
for plan().
Disclaimer
We do see a lot of value in investigating how analytical choices
affect a statistical outcome of interest. However, we strongly caution
against using specr as a tool to somehow arrive at a better estimate
compared to a single model. Running a specification curve analysis
does not make your findings any more reliable, valid or generalizable
than a single analysis. The method is meant to inform about the effects
of analytical choices on results, and not a better way to estimate a
correlation or effect.