MetaUtility (version 1.2.0)

prop_stronger: Estimate proportion of true effect sizes above or below a threshold

Description

Estimates the proportion of true (i.e., population parameter) effect sizes in a meta-analysis that are above or below a specified threshold of scientific importance.

Usage

prop_stronger(q, M, t2, se.M = NA, se.t2 = NA, CI.level = 0.95,
  tail = NA, dat = NULL, R = 2000, bootstrap = "ifneeded",
  yi.name = "yi", vi.name = "vi")

Arguments

q

True effect size that is the threshold for "scientific importance"

M

Pooled point estimate from meta-analysis

t2

Estimated heterogeneity (tau^2) from meta-analysis

se.M

Estimated standard error of pooled point estimate from meta-analysis

se.t2

Estimated standard error of tau^2 from meta-analysis

CI.level

Confidence level as a proportion

tail

above for the proportion of effects above q; below for the proportion of effects below q.

dat

Dataset of point estimates (with names equal to the passed yi.name) and their variances (with names equal to the passed vi.name). Only required when bootstrapping.

R

Number of bootstrap iterates. Only required when bootstrapping.

bootstrap

If equal to ifneeded, bootstraps if estimated proportion is less than 0.15 or more than 0.85. If equal to never, instead does not return inference in the above edge cases. Only required when bootstrapping.

yi.name

Name of the variable in dat containing the study-level point estimates. Only required when bootstrapping.

vi.name

Name of the variable in dat containing the study-level variances. Only required when bootstrapping.

Value

Returns a dataframe containing the point estimate for the proportion, its estimated standard error, and confidence interval limits.

Details

When the estimated proportion is less than 0.15 or more than 0.85, it is best to bootstrap the confidence interval using the bias-corrected and accelerated (BCa) method (Mathur & VanderWeele, 2018); this is the default behavior of prop_stronger. Sometimes BCa confidence interval estimation fails, in which case prop_stronger instead uses the percentile method, issuing a warning if this is the case. We use a modified "safe" version of the boot package code for bootstrapping such that if any bootstrap iterates fail (usually because of model estimation problems), the error message is printed but the bootstrap iterate is simply discarded so that confidence interval estimation can proceed.

References

1. Mathur MB & VanderWeele TJ. New metrics for meta-analyses of heterogeneous effects. Statistics in Medicine (2018).

2. Mathur MB & VanderWeele TJ. New metrics for multisite replication projects. Under review.

Examples

# NOT RUN {
##### Example 1: BCG Vaccine and Tuberculosis Meta-Analysis #####

# calculate effect sizes for example dataset
d = metafor::escalc(measure="RR", ai=tpos, bi=tneg,
                   ci=cpos, di=cneg, data=metafor::dat.bcg)

# fit random-effects model
# note that metafor package returns on the log scale
m = metafor::rma.uni(yi= d$yi, vi=d$vi, knha=TRUE,
measure="RR", method="REML" )

# pooled point estimate (RR scale)
exp(m$b)

# estimate the proportion of effects stronger than RR = 0.80
# no bootstrapping will be needed
prop_stronger( q = log(0.8),
               M = as.numeric(m$b),
               t2 = m$tau2,
               se.M = as.numeric(m$vb),
               se.t2 = m$se.tau2,
               CI.level = 0.95,
              tail = "below",
              bootstrap = "ifneeded")

# }
# NOT RUN {
# now try a more extreme threshold, q, such that the function will use bootstrapping
# now we will need to pass the final 4 arguments as well
prop_stronger( q = log(0.9),
               M = as.numeric(m$b),
              t2 = m$tau2,
              se.M = as.numeric(m$vb),
              se.t2 = m$se.tau2,
              CI.level = 0.95,
              tail = "below",

              # below arguments control bootstrapping
              # only 100 iterates for demo purposes (should be higher in practice)
              dat = d,
              R = 100,
              bootstrap = "ifneeded",
              yi.name = "yi",
              vi.name = "vi" )
# }
# NOT RUN {
##### Example 2: Meta-Analysis of Multisite Replication Studies #####

 # replication estimates (Fisher's z scale) and SEs
 # from moral credential example in reference #2
 r.fis = c(0.303, 0.078, 0.113, -0.055, 0.056, 0.073,
 0.263, 0.056, 0.002, -0.106, 0.09, 0.024, 0.069, 0.074,
 0.107, 0.01, -0.089, -0.187, 0.265, 0.076, 0.082)

 r.SE = c(0.111, 0.092, 0.156, 0.106, 0.105, 0.057,
 0.091, 0.089, 0.081, 0.1, 0.093, 0.086, 0.076,
 0.094, 0.065, 0.087, 0.108, 0.114, 0.073, 0.105, 0.04)

 d = data.frame( yi = r.fis,
                 vi = r.SE^2 )

 # meta-analyze the replications
 m = metafor::rma.uni( yi = r.fis, vi = r.SE^2, measure = "ZCOR" )

 # probability of true effect above r = 0.10 = 28%
 # convert threshold on r scale to Fisher's z
 q = r_to_z(0.10)

 # bootstrap reps should be higher in practice (e.g., 1000)
 # here using only 100 for speed
 prop_stronger( q = q,
                M = m$b,
                se.M = m$se,
                t2 = m$tau2,
                se.t2 = m$se.tau2,
                tail = "above",
                dat = d,
                R = 250 )


 # probability of true effect equally strong in opposite direction
 q.star = r_to_z(-0.10)
 prop_stronger( q = q.star,
                M = m$b,
                se.M = m$se,
                t2 = m$tau2,
                se.t2 = m$se.tau2,
                tail = "below",
                dat = d,
                R = 250 )
# }