DeclareDesign (version 1.1.0)

select_diagnosands: Select diagnosands

Description

Helper to select (and adjust) a set of named diagnosands

Usage

select_diagnosands(..., alpha = 0.05, subset = NULL, na.rm = FALSE)

Arguments

...

Requested diagnosands from the extended diagnosand list

alpha

Significance level, passed to declare_diagnosands

subset

Subset indicator, passed to declare_diagnosands

na.rm

Logical for handling NAs when calculating diagnosands

Details

The diagnosands available from the extended diagnosand list are:

| Diagnosand        | Meaning                                          | Formula                                                     |
|-------------------|--------------------------------------------------|-------------------------------------------------------------|
| mean_estimand     | Expected value of the estimand                   | `mean(estimand)`                                            |
| mean_estimate     | Expected value of the estimate                   | `mean(estimate)`                                            |
| bias              | (Expected) bias                                  | `mean(estimate - estimand)`                                 |
| sd_estimate       | The s.d. of the estimate ('true' standard error) | `sd(estimate)`                                              |
| rmse              | The root mean squared error                      | `sqrt(mean((estimate - estimand) ^ 2))`                     |
| power             | Power: probability of rejecting a null of zero   | `mean(p.value <= alpha)`                                    |
| coverage          | Coverage: prob the CI will cover the truth       | `mean(estimand <= conf.high & estimand >= conf.low)`        |
| mean_se           | The expected standard error                      | `mean(std.error)`                                           |
| type_s_rate       | Prob a significant estimate has wrong sign       | `mean((sign(estimate) != sign(estimand))[p.value <= alpha])`|
| exaggeration_ratio| The expected exaggeration (given significance)   | `mean((estimate/estimand)[p.value <= alpha])`               |
| var_estimate      | The variance of the estimate                     | `pop.var(estimate)`                                         |
| mean_var_hat      | Expected estimate of the 'true' var of estimate  | `mean(std.error^2)`                                         |
| prop_pos_sig      | Probability of a positive estimate given sig.    | `mean(estimate > 0 & p.value <= alpha)`                     |
| mean_ci_length    | The expected length of the confidence interval   | `mean(conf.high - conf.low)`                                |

The alpha argument can be used to adjust significance levels, and the subset argument can be used for further conditioning. The output is a function that can be applied directly to a simulations dataframe.

Examples

select_diagnosands("bias")
select_diagnosands("prop_pos_sig", "mean_se", alpha = .001)
select_diagnosands("bias")(data.frame(estimate = 2, estimand = 3))
select_diagnosands("bias")(data.frame(estimate = 1:2, estimand = c(NA, 2)))
select_diagnosands("bias", na.rm = TRUE)(data.frame(estimate = 1:2, estimand = c(NA, 2)))

# Example of use with diagnose_design
design <- 
  declare_model(N = 100, u = rnorm(N),
                Y_Z_0 = 0, 
                Y_Z_1 = ifelse(rbinom(N, 1, prob = 0.5), 0.1, -0.1) + u
  ) +
  declare_assignment(Z = complete_ra(N)) + 
  declare_inquiry(ATE = mean(Y_Z_1 - Y_Z_0)) + 
  declare_measurement(Y = reveal_outcomes(Y ~ Z)) +
  declare_estimator(Y ~ Z, inquiry = "ATE")

# Compare the average se and the sd of the sampling distribution of the estimate
diagnose_design(design,
                diagnosands = select_diagnosands("sd_estimate", "mean_se"),
                sims = 100)