marginaleffects (version 0.29.0)

inferences: (EXPERIMENTAL) Bootstrap, Conformal, and Simulation-Based Inference

Description

Warning: This function is experimental. It may be renamed, the user interface may change, or the functionality may migrate to arguments in other marginaleffects functions.

Apply this function to a marginaleffects object to change the inferential method used to compute uncertainty estimates.

Usage

inferences(
  x,
  method,
  R = 1000,
  conf_type = "perc",
  conformal_test = NULL,
  conformal_calibration = NULL,
  conformal_score = "residual_abs",
  estimator = NULL,
  ...
)

Value

A marginaleffects object with simulation or bootstrap resamples and objects attached.

Arguments

x

Object produced by one of the core marginaleffects functions, or a data frame suitable for the function supplied to the estimator argument.

method

String

• "delta": delta method standard errors

• "boot" package

• "fwb": fractional weighted bootstrap

• "rsample" package

• "simulation" from a multivariate normal distribution (Krinsky & Robb, 1986)

• "mi" multiple imputation for missing data

• "conformal_split": prediction intervals using split conformal prediction (see Angelopoulos & Bates, 2022)

• "conformal_cv+": prediction intervals using cross-validation+ conformal prediction (see Barber et al., 2020)

R

Number of resamples, simulations, or cross-validation folds.

conf_type

String: type of bootstrap interval to construct.

• boot: "perc", "norm", "basic", or "bca"

• fwb: "perc", "norm", "wald", "basic", "bc", or "bca"

• rsample: "perc" or "bca"

• simulation: "perc" or "wald"

conformal_test

Data frame of test data for conformal prediction.

conformal_calibration

Data frame of calibration data for split conformal prediction (method="conformal_split).

conformal_score

String. Warning: The type argument in predictions() must generate predictions which are on the same scale as the outcome variable. Typically, this means that type must be "response" or "probs".

• "residual_abs" or "residual_sq" for regression tasks (numeric outcome)

• "softmax" for classification tasks (when predictions() returns a group columns, such as multinomial or ordinal logit models.

estimator

Function that accepts a data frame, fits a model, applies a marginaleffects function, and returns the object. Only supported with method = "rsample" or method = "boot". When method = "rsample", the output must include a "term" column. This is not always the case for predictions(), in which case users may have to create the column manually.

...

• If method = "boot", additional arguments are passed to boot::boot().

• If method = "fwb", additional arguments are passed to fwb::fwb().

• If method = "rsample", additional arguments are passed to rsample::bootstraps(), unless the user supplies a group argument, in which case all arguments are passed to rsample::group_bootstraps().

• Additional arguments are ignored for all other methods.

Details

When method = "simulation", we conduct simulation-based inference following the method discussed in Krinsky & Robb (1986):

1. Draw R sets of simulated coefficients from a multivariate normal distribution with mean equal to the original model's estimated coefficients and variance equal to the model's variance-covariance matrix (classical, "HC3", or other).

2. Use the R sets of coefficients to compute R sets of estimands: predictions, comparisons, slopes, or hypotheses.

3. Take quantiles of the resulting distribution of estimands to obtain a confidence interval (when conf_type = "perc") and the standard deviation of simulated estimates to estimate the standard error (which is used for a Z-test and Wald confidence intervals when conf_type = "wald").

When method = "fwb", drawn weights are supplied to the model fitting function's weights argument; if the model doesn't accept non-integer weights, this method should not be used. If weights were included in the original model fit, they are extracted by weights() and multiplied by the drawn weights. These weights are supplied to the wts argument of the estimation function (e.g., comparisons()).

Warning: custom model classes are not supported by inferences() because they are not guaranteed to come with an appropriate update() method.

References

Krinsky, I., and A. L. Robb. 1986. "On Approximating the Statistical Properties of Elasticities." Review of Economics and Statistics 68 (4): 715–9.

King, Gary, Michael Tomz, and Jason Wittenberg. "Making the most of statistical analyses: Improving interpretation and presentation." American journal of political science (2000): 347-361

Dowd, Bryan E., William H. Greene, and Edward C. Norton. "Computation of standard errors." Health services research 49.2 (2014): 731-750.

Angelopoulos, Anastasios N., and Stephen Bates. 2022. "A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification." arXiv. https://doi.org/10.48550/arXiv.2107.07511.

Barber, Rina Foygel, Emmanuel J. Candes, Aaditya Ramdas, and Ryan J. Tibshirani. 2020. "Predictive Inference with the Jackknife+." arXiv. http://arxiv.org/abs/1905.02928.

Examples

if (FALSE) {
library(magrittr)
set.seed(1024)
mod <- lm(Sepal.Length ~ Sepal.Width * Species, data = iris)

# bootstrap
avg_predictions(mod, by = "Species") %>%
    inferences(method = "boot")

avg_predictions(mod, by = "Species") %>%
    inferences(method = "rsample")

# Fractional (bayesian) bootstrap
avg_slopes(mod, by = "Species") %>%
    inferences(method = "fwb") %>%
    get_draws("rvar") %>%
    data.frame()

# Simulation-based inference
slopes(mod) %>%
    inferences(method = "simulation") %>%
    head()

# Two-step estimation procedure: Propensity score + G-Computation
lalonde <- get_dataset("lalonde")
estimator <- function(data) {
    # Step 1: Estimate propensity scores
    fit1 <- glm(treat ~ age + educ + race, family = binomial, data = data)
    ps <- predict(fit1, type = "response")
    # Step 2: Fit weighted outcome model
    m <- lm(re78 ~ treat * (re75 + age + educ + race),
        data = data, weight = ps
    )
    # Step 3: Compute average treatment effect by G-computation
    avg_comparisons(m, variables = "treat", wts = ps, vcov = FALSE)
}
inferences(lalonde, method = "rsample", estimator = estimator)
}