frag.ma: Assessing the Fragility of a Meta-Analysis With a Binary Outcome

Description

Produces results, such as fragility index, fragility quotient, and event status modifications, for altering the statistical significance of a meta-analysis with a binary outcome.

Usage

frag.ma(e0, n0, e1, n1, data, measure = "OR", alpha = 0.05,
        mod.dir = "both", OR = 1, RR = 1, RD = 0,
        method = "DL", test = "z", drop00 = FALSE, ...)

Value

An object of class "frag.ma". The object is a list containing the following components:

data: original data in the form of a data frame with six columns: event counts, non-event counts, and sample sizes in group 0 and those in group 1 among all studies in the meta-analysis.
measure: measure of treatment effect (i.e., effect size).
alpha: pre-specified statistical significance level.
null: value of odds ratio, relative risk, or risk difference (specified by measure) under the null hypothesis. Note that odds ratio and relative risk are presented on a natural logarithmic scale in all output values.
est.ori: the point estimate of the overall effect size based on the original meta-analysis.
ci.ori: the confidence interval of the overall effect size based on the original meta-analysis.
pval.ori: the p-value of the overall effect size based on the original meta-analysis.
mod.dir: the direction of the confidence interval change due to event status modifications when the original confidence interval covers the null value (i.e., non-significance altered to significance).
FI: fragility index.
FQ: fragility quotient.
dir: direction of the change of statistical significance.
sid.iter: study index(es) with event status modifications during the iterative algorithm.
g0.mod.iter: event status modification(s) in group 0 during the iterative algorithm.
g1.mod.iter: event status modification(s) in group 1 during the iterative algorithm.
est.iter: point estimate(s) of the overall effect size during the iterative algorithm.
ci.iter: confidence interval(s) of the overall effect size during the iterative algorithm.
data.mod: data with event status modification(s) that alter the significance or non-significance.

Arguments

e0: a numeric vector or the corresponding column name in the argument data, indicating event counts in group 0 among the studies in the meta-analysis.
n0: a numeric vector or the corresponding column name in the argument data, indicating sample sizes in group 0 among the studies in the meta-analysis.
e1: a numeric vector or the corresponding column name in the argument data, indicating event counts in group 1 among the studies in the meta-analysis.
n1: a numeric vector or the corresponding column name in the argument data, indicating sample sizes in group 1 among the studies in the meta-analysis.
data: an optional data frame containing the dataset of the collected studies in the meta-analysis with a binary outcome. If data is specified, the previous arguments, e0, n0, e1, and n1, should be specified as their corresponding column names in data.
measure: a character string indicating the measure of treatment effect (i.e., effect size) for the binary outcome. It should be one of "OR" (odds ratio, the default), "RR" (relative risk), and "RD" (risk difference).
alpha: a numeric value between 0 and 1, indicating the statistical significance level (the default is 0.05). It determines the confidence level \((1-\alpha)\times100\%\) of the confidence interval when deriving the fragility index/quotient.
mod.dir: a character string indicating the direction of the confidence interval change due to event status modifications when the original confidence interval covers the null value (i.e., non-significance altered to significance). It is not used when significance is altered to non-significance. It should be one of "left" (moving to the left side of the null value), "right" (moving to the right side of the null value), "one" (based on the direction of the point estimate of the original overall effect size), and "both" (both directions, the default).
OR: a numeric positive value indicating the value of odds ratio under the null hypothesis (the default is 1). It is used only if the argument measure is "OR".
RR: a numeric positive value indicating the value of relative risk under the null hypothesis (the default is 1). It is used only if the argument measure is "RR".
RD: a numeric value between \(-\)1 indicating the value of risk difference under the null hypothesis (the default is 0). It is used only if the argument measure is "RD".
method: a character string specifying the method for performing meta-analysis. It is passed to rma.uni. Options include the fixed-effect model by specifying "FE" and the random-effects model by specifying "DL" (DerSimonian--Laird estimator), "HE" (Hedges estimator), "HS" (Hunter--Schmidt estimator), "SJ" (Sidik--Jonkman estimator), "ML" (maximum likelihood estimator), "REML" (restricted maximum likelihood estimator), "EB" (empirical Bayes estimator), "PM" (Paule--Mandel estimator), or "GENQ" (generalized \(W\)-statistic estimator) for different methods to estimate the heterogeneity variance. See details in the manual of the package metafor.
test: a character string specifying how confidence intervals are derived. It is passed to rma.uni. Options include "z", "t", "knha" (Hartung--Knapp--Sidik--Jonkman method), and "adhoc"; the first indicates Wald-type confidence intervals based on the standard normal distribution (the default), while the latter three yield confidence intervals based on the t-distribution. See details in the manual of the package metafor.
drop00: a logical value indicating whether studies with no events (or only events) in both groups should be dropped when calculating the observed effect sizes or outcomes. It is passed to rma.uni.
...: other arguments that can be passed to rma.uni. See details in the manual of the package metafor.

References

Atal I, Porcher R, Boutron I, Ravaud P (2019). "The statistical significance of meta-analyses is frequently fragile: definition of a fragility index for meta-analyses." Journal of Clinical Epidemiology, 111, 32--40. <tools:::Rd_expr_doi("10.1016/j.jclinepi.2019.03.012")>

Lin L, Chu H (2022). "Assessing and visualizing fragility of clinical results with binary outcomes in R using the fragility package." PLOS ONE, 17(6), e0268754. <tools:::Rd_expr_doi("10.1371/journal.pone.0268754")>

Lin L, Xing A, Chu H, Murad MH, Xu C, Baer BR, Wells MT, Sanchez-Ramos L (2023). "Assessing the robustness of results from clinical trials and meta-analyses with the fragility index." American Journal of Obstetrics and Gynecology, 228(3), 276--282. <tools:::Rd_expr_doi("10.1016/j.ajog.2022.08.053")>

Wang Z, Xing X, Mun EY, Wu C, Lin L (2025). "The role of double-zero-event studies in evidence synthesis: Evaluating robustness using the fragility index." Journal of Evaluation in Clinical Practice, 31(1), e14301. <tools:::Rd_expr_doi("10.1111/jep.14301")>

Examples

Run this code

## Load datasets of meta-analyses on nutrition support
data(dat.ns)

## Assess fragility of the 11th meta-analysis
out1 <- frag.ma(e0, n0, e1, n1, data = dat.ns[dat.ns$ma.id == 11,])
out1

## Visualize the process of event status modifications
##  for altering the non-significance to significance
plot(out1)

## Assess fragility based on relative risk
out2 <- frag.ma(e0, n0, e1, n1, data = dat.ns[dat.ns$ma.id == 11,],
  measure = "RR")
out2

## Assess fragility based on risk difference
out3 <- frag.ma(e0, n0, e1, n1, data = dat.ns[dat.ns$ma.id == 11,],
  measure = "RD")
out3

## Use the REML method to estimate the heterogeneity variance
out4 <- frag.ma(e0, n0, e1, n1, data = dat.ns[dat.ns$ma.id == 11,],
  method = "REML")
out4

## Use the Hartung-Knapp-Sidik-Jonkman to derive
##  the confidence interval in meta-analysis
out5 <- frag.ma(e0, n0, e1, n1, data = dat.ns[dat.ns$ma.id == 11,],
  method = "REML", test = "knha")
out5

## Change the significance level to 0.005
out6 <- frag.ma(e0, n0, e1, n1, data = dat.ns[dat.ns$ma.id == 11,],
  alpha = 0.005)
out6
plot(out6)

## Reduce the distance between the two lines by breaking the y-axis
plot(out6, ybreaks = c(5, 9))

## Suppress the markers of study IDs around the lines
plot(out6, ybreaks = c(5, 9), study.marker = FALSE)

## Change the size of markers of study IDs
plot(out6, ybreaks = c(5, 9), cex.marker = 0.5)

## Change the distance between the markers of study IDs and the lines
plot(out6, ybreaks = c(5, 9), offset.marker = 0)

## Change the colors and width of the lines
plot(out6, ybreaks = c(5, 9), col.line = c("gray", "orange"), lwd = 2)

## Change the location of the legend
plot(out6, ybreaks = c(5, 9), x.legend = "topleft")

# \donttest{
## Assess fragility of the first meta-analysis;
##  it takes longer time
out7 <- frag.ma(e0, n0, e1, n1, data = dat.ns[dat.ns$ma.id == 1,])
out7
plot(out7)
plot(out7, ybreaks = c(840, 880))

## Achieve significance by moving the CI based on the direction of the 
##  original overall effect size estimate
out8 <- frag.ma(e0, n0, e1, n1, data = dat.ns[dat.ns$ma.id == 1,],
  mod.dir = "one")
out8

## Achieve significance by moving the CI to the left side
out9 <- frag.ma(e0, n0, e1, n1, data = dat.ns[dat.ns$ma.id == 1,],
  mod.dir = "left")
out9

## Achieve significance by moving the CI to the right side;
##  it takes longer time than that moving the CI to the left side
##  as the original overall effect size estimate was less than the null
out10 <- frag.ma(e0, n0, e1, n1, data = dat.ns[dat.ns$ma.id == 1,],
  mod.dir = "right")
out10
dev.new(width = 12, height = 6)
plot(out10, cex.marker = 0.5)
# }