Function to test differences of adjusted predictions for
statistical significance. This is usually called contrasts or (pairwise)
comparisons. test_predictions() is an alias.
hypothesis_test(model, ...)test_predictions(model, ...)
# S3 method for default
hypothesis_test(
model,
terms = NULL,
by = NULL,
test = "pairwise",
equivalence = NULL,
scale = "response",
p_adjust = NULL,
df = NULL,
ci_level = 0.95,
collapse_levels = FALSE,
verbose = TRUE,
ci.lvl = ci_level,
...
)
# S3 method for ggeffects
hypothesis_test(
model,
by = NULL,
test = "pairwise",
equivalence = NULL,
scale = "response",
p_adjust = NULL,
df = NULL,
collapse_levels = FALSE,
verbose = TRUE,
...
)
A data frame containing predictions (e.g. for test = NULL),
contrasts or pairwise comparisons of adjusted predictions or estimated
marginal means.
A fitted model object, or an object of class ggeffects.
Arguments passed down to data_grid() when creating the reference
grid and to marginaleffects::predictions() resp. marginaleffects::slopes().
For instance, arguments type or transform can be used to back-transform
comparisons and contrasts to different scales. vcov can be used to
calculate heteroscedasticity-consistent standard errors for contrasts.
See examples at the bottom of
this vignette
for further details. Note the different ways to define the heteroscedasticity-consistent
variance-covariance matrix for ggpredict() and hypothesis_test() resp.
johnson_neyman(). For ggpredict(), the arguments are named vcov_fun
and vcov_args, whereas for hypothesis_test() and johnson_neyman(),
there is only the argument vcov. See ?marginaleffects::slopes for
further details.
Character vector with the names of the focal terms from model,
for which contrasts or comparisons should be displayed. At least one term
is required, maximum length is three terms. If the first focal term is numeric,
contrasts or comparisons for the slopes of this numeric predictor are
computed (possibly grouped by the levels of further categorical focal
predictors).
Character vector specifying the names of predictors to condition on.
Hypothesis test is then carried out for focal terms by each level of by
variables. This is useful especially for interaction terms, where we want
to test the interaction within "groups". by is only relevant for
categorical predictors.
Hypothesis to test. By default, pairwise-comparisons are conducted. See section Introduction into contrasts and pairwise comparisons.
ROPE's lower and higher bounds. Should be "default" or
a vector of length two (e.g., c(-0.1, 0.1)). If "default",
bayestestR::rope_range() is used. Instead of using the equivalence
argument, it is also possible to call the equivalence_test() method
directly. This requires the parameters package to be loaded. When
using equivalence_test(), two more columns with information about the
ROPE coverage and decision on H0 are added. Furthermore, it is possible
to plot() the results from equivalence_test(). See
bayestestR::equivalence_test() resp. parameters::equivalence_test.lm()
for details.
Character string, indicating the scale on which the contrasts or comparisons are represented. Can be one of:
"response" (default), which would return contrasts on the response
scale (e.g. for logistic regression, as probabilities);
"link" to return contrasts on scale of the linear predictors
(e.g. for logistic regression, as log-odds);
"probability" (or "probs") returns contrasts on the probability scale,
which is required for some model classes, like MASS::polr();
"oddsratios" to return contrasts on the odds ratio scale (only applies
to logistic regression models);
"irr" to return contrasts on the odds ratio scale (only applies to
count models);
or a transformation function like "exp" or "log", to return transformed
(exponentiated respectively logarithmic) contrasts; note that these
transformations are applied to the response scale.
Note: If the scale argument is not supported by the provided model,
it is automaticaly changed to a supported scale-type (a message is printed
when verbose = TRUE).
Character vector, if not NULL, indicates the method to
adjust p-values. See stats::p.adjust() or stats::p.adjust.methods
for details. Further possible adjustment methods are "tukey" or "sidak",
and for johnson_neyman(), "fdr" (or "bh") and "esarey" (or its
short-cut "es") are available options. Some caution is necessary when
adjusting p-value for multiple comparisons. See also section P-value adjustment
below.
Degrees of freedom that will be used to compute the p-values and
confidence intervals. If NULL, degrees of freedom will be extracted from
the model using insight::get_df() with type = "wald".
Numeric, the level of the confidence intervals.
Logical, if TRUE, term labels that refer to identical
levels are no longer separated by "-", but instead collapsed into a unique
term label (e.g., "level a-level a" becomes "level a"). See 'Examples'.
Toggle messages and warnings.
Deprecated, please use ci_level.
There are many ways to test contrasts or pairwise comparisons. A detailed introduction with many (visual) examples is shown in this vignette.
Note that p-value adjustment for methods supported by p.adjust() (see also
p.adjust.methods), each row is considered as one set of comparisons, no
matter which test was specified. That is, for instance, when hypothesis_test()
returns eight rows of predictions (when test = NULL), and p_adjust = "bonferroni",
the p-values are adjusted in the same way as if we had a test of pairwise
comparisons (test = "pairwise") where eight rows of comparisons are
returned. For methods "tukey" or "sidak", a rank adjustment is done
based on the number of combinations of levels from the focal predictors
in terms. Thus, the latter two methods may be useful for certain tests
only, in particular pairwise comparisons.
For johnson_neyman(), the only available adjustment methods are "fdr"
(or "bh") (Benjamini & Hochberg (1995)) and "esarey" (or "es")
(Esarey and Sumner 2017). These usually return similar results. The major
difference is that "fdr" can be slightly faster and more stable in edge
cases, however, confidence intervals are not updated. Only the p-values are
adjusted. "esarey" is slower, but confidence intervals are updated as well.
The verbose argument can be used to display or silence messages and
warnings. Furthermore, options() can be used to set defaults for the
print() and print_html() method. The following options are available,
which can simply be run in the console:
ggeffects_ci_brackets: Define a character vector of length two, indicating
the opening and closing parentheses that encompass the confidence intervals
values, e.g. options(ggeffects_ci_brackets = c("[", "]")).
ggeffects_collapse_ci: Logical, if TRUE, the columns with predicted
values (or contrasts) and confidence intervals are collapsed into one
column, e.g. options(ggeffects_collapse_ci = TRUE).
ggeffects_collapse_p: Logical, if TRUE, the columns with predicted
values (or contrasts) and p-values are collapsed into one column, e.g.
options(ggeffects_collapse_p = TRUE). Note that p-values are replaced
by asterisk-symbols (stars) or empty strings when ggeffects_collapse_p = TRUE,
depending on the significance level.
ggeffects_collapse_tables: Logical, if TRUE, multiple tables for
subgroups are combined into one table. Only works when there is more than
one focal term, e.g. options(ggeffects_collapse_tables = TRUE).
ggeffects_output_format: String, either "text" or "html". Defines the
default output format from ggpredict(). If "html", a formatted HTML
table is created and printed to the view pane. If "text" or NULL, a
formatted table is printed to the console, e.g. options(ggeffects_output_format = "html").
ggeffects_html_engine: String, either "tt" or "gt". Defines the default
engine to use for printing HTML tables. If "tt", the tinytable package
is used, if "gt", the gt package is used, e.g.
options(ggeffects_html_engine = "gt").
Use options(<option_name> = NULL) to remove the option.
Esarey, J., & Sumner, J. L. (2017). Marginal effects in interaction models: Determining and controlling the false positive rate. Comparative Political Studies, 1–33. Advance online publication. doi: 10.1177/0010414017730080
There is also an equivalence_test() method in the parameters
package (parameters::equivalence_test.lm()), which can be used to
test contrasts or comparisons for practical equivalence. This method also
has a plot() method, hence it is possible to do something like:
library(parameters)
ggpredict(model, focal_terms) |>
equivalence_test() |>
plot()