frag.study: Assessing the Fragility of an Individual Study With a Binary Outcome

Description

Produces results, such as fragility index, fragility quotient, and event status modifications, for altering the statistical significance of an individual study with a binary outcome.

Usage

frag.study(e0, n0, e1, n1, data, all = FALSE, methods,
           modify0 = "both", modify1 = "both", alpha = 0.05,
           alternative = "two.sided", OR = 1, RR = 1, RD = 0,
           allcase = TRUE)

Value

An object of class "frag.study" (if the argument all = FALSE) or of both classes "frag.study" and "frag.study.all" (all = TRUE). The object is a list containing the following components:

data: original data in the form of 2\(\times\)2 table.
methods: methods used to calculate the p-value(s) of the association between treatment and outcome.
alpha: pre-specified statistical significance level.
alternative: alternative hypothesis.
null: values of odds ratio, relative risk, and risk difference under the null hypothesis; they are only used if "OR", "RR", and "RD" are included in the argument methods, respectively.
modify0: type of event status modifications in group 0.
modify1: type of event status modifications in group 1.
f0.range: range of event status modifications in group 0.
f1.range: range of event status modifications in group 1.
allcase: a logical value indicating whether all cases of minimal event status modifications for altering statistical significance are obtained; it is only included in the object of class "frag.study".
tot.mods: a matrix containing all possible event status modifications with rows representing modifications in group 0 and columns representing those in group 1; it is determined by f0.range and f1.range, and is only included in the object of class "frag.study.all".
pval: p-value(s) produced by the method(s) included in the argument methods.
FI: fragility index(es) based on the method(s).
FQ: fragility quotient(s) based on the method(s).
dir: direction of the change of statistical significance.
mods: minimal event status modification(s) that alter statistical significance or non-significance.
pvals: p-values based on all possible event status modifications (as in tot.mods); it is only included in the object of class "frag.study.all".
FI0: fragility index(es) based on the method(s) when event status modifications are restricted to group 0; it is only included in the object of class "frag.study.all" if both arguments modify0 and modify1 are not "none".
FQ0: fragility quotient(s) based on the method(s) when event status modifications are restricted to group 0; it is only included in the object of class "frag.study.all" if both arguments modify0 and modify1 are not "none".
dir0: direction of the change of statistical significance when event status modifications are restricted to group 0; it is only included in the object of class "frag.study.all" if both arguments modify0 and modify1 are not "none".
mods0: minimal event status modification(s) that alter statistical significance or non-significance when event status modifications are restricted to group 0; it is only included in the object of class "frag.study.all" if both arguments modify0 and modify1 are not "none".
FI1: fragility index(es) based on the method(s) when event status modifications are restricted to group 1; it is only included in the object of class "frag.study.all" if both arguments modify0 and modify1 are not "none".
FQ1: fragility quotient(s) based on the method(s) when event status modifications are restricted to group 1; it is only included in the object of class "frag.study.all" if both arguments modify0 and modify1 are not "none".
dir1: direction of the change of statistical significance when event status modifications are restricted to group 1; it is only included in the object of class "frag.study.all" if both arguments modify0 and modify1 are not "none".
mods1: minimal event status modification(s) that alter statistical significance or non-significance when event status modifications are restricted to group 1; it is only included in the object of class "frag.study.all" if both arguments modify0 and modify1 are not "none".

Of note, if both arguments modify0 and modify0 are "none" (i.e., no modification), the items following pval in the above list are not produced, because fragility cannot be assessed without event status modifications.

Arguments

e0: a numeric value or the corresponding column name in the argument data, indicating event count in group 0.
n0: a numeric value or the corresponding column name in the argument data, indicating sample size in group 0.
e1: a numeric value or the corresponding column name in the argument data, indicating event count in group 1.
n1: a numeric value or the corresponding column name in the argument data, indicating sample size in group 1.
data: an optional data frame containing the dataset of the individual study with a binary outcome. If data is specified, the previous arguments, e0, n0, e1, and n1, should be specified as their corresponding column names in data.
all: a logical value indicating whether all eligible event status modifications will be considered (the default is FALSE).
methods: a character string or a vector of character strings indicating the method(s) used to calculate the p-value(s) of the association between treatment and outcome. It must consist of "Fisher" (Fisher's exact test), "chisq" (chi-squared test), "OR" (odds ratio), "RR" (relative risk), and/or "RD" (risk difference). The default is the vector consisting of all the aforementioned five methods.
modify0: a character string indicating the event status modifications in group 0. It should be one of "increase" (increasing event numbers), "decrease" (decreasing event numbers), "both" (the default, modifying event numbers in both directions), and "none" (no modification).
modify1: a character string indicating the event status modifications in group 1. It has the same usage with modify0, with the default being "both".
alpha: a numeric value between 0 and 1, indicating the statistical significance level (the default is 0.05).
alternative: a character string of either "two.sided" (the default) or "one.sided", indicating the alternative hypothesis. It is only used for "OR", "RR", and "RD" in the argument methods.
OR: a numeric positive value indicating the value of odds ratio under the null hypothesis (the default is 1). It is used only if the argument methods includes "OR".
RR: a numeric positive value indicating the value of relative risk under the null hypothesis (the default is 1). It is used only if the argument methods includes "RR".
RD: a numeric value between \(-\)1 indicating the value of risk difference under the null hypothesis (the default is 0). It is used only if the argument methods includes "RD".
allcase: a logical value indicating whether obtaining all cases of minimal event status modifications for altering statistical significance or non-significance (the default is TRUE).

References

Lin L, Chu H (2022). "Assessing and visualizing fragility of clinical results with binary outcomes in R using the fragility package." PLOS ONE, 17(6), e0268754. <tools:::Rd_expr_doi("10.1371/journal.pone.0268754")>

Lin L, Xing A, Chu H, Murad MH, Xu C, Baer BR, Wells MT, Sanchez-Ramos L (2023). "Assessing the robustness of results from clinical trials and meta-analyses with the fragility index." American Journal of Obstetrics and Gynecology, 228(3), 276--282. <tools:::Rd_expr_doi("10.1016/j.ajog.2022.08.053")>

Walsh M, Srinathan SK, McAuley DF, Mrkobrada M, Levine O, Ribic C, Molnar AO, Dattani ND, Burke A, Guyatt G, Thabane L, Walter SD, Pogue J, Devereaux PJ (2014). "The statistical significance of randomized controlled trial results is frequently fragile: a case for a Fragility Index." Journal of Clinical Epidemiology, 67(6), 622--8. <tools:::Rd_expr_doi("10.1016/j.jclinepi.2013.10.019")>

Examples

Run this code

## Load datasets of trials on antidepressant drugs
data(dat.ad)

## Assess fragility of trial 13
out1 <- frag.study(e0 = e0, n0 = n0, e1 = e1, n1 = n1, data = dat.ad[13,])
out1

## Consider all possible event status modifications
##  and focus on odds ratio
out2 <- frag.study(e0 = e0, n0 = n0, e1 = e1, n1 = n1, data = dat.ad[13,],
  all = TRUE, methods = "OR")
out2

## Visualize the fragility
# \donttest{
plot(out2)
# }

## Restrict the modifications to treatment group 0
plot(out2, modify1 = FALSE)

## Change the truncation threshold of p-value to 10^-30
plot(out2, modify1 = FALSE, trun = 30)

## Restrict the modifications to treatment group 1
plot(out2, modify0 = FALSE)

## Assess fragility of trial 13
##  by only increasing event counts in treatment group 0
out3 <- frag.study(e0 = e0, n0 = n0, e1 = e1, n1 = n1, data = dat.ad[13,],
  all = TRUE, modify0 = "increase", modify1 = "none")
out3

## The default plot is for the first method (i.e., Fisher's exact test)
##  if the argument method is not specified
plot(out3)

## Generate plots based on other methods
plot(out3, method = "chisq")
plot(out3, method = "OR")
plot(out3, method = "RR")
plot(out3, method = "RD")

## Assess fragility of trial 13
##  by only modifying event status in treatment group 1
out4 <- frag.study(e0 = e0, n0 = n0, e1 = e1, n1 = n1, data = dat.ad[13,],
  all = TRUE, modify0 = "none", modify1 = "both")
out4
plot(out4)

## Change background colors in the plot
plot(out4, col.sig = c(adjustcolor("blue", alpha.f = 0.1),
  adjustcolor("green", alpha.f = 0.1)))
## Adjust the plot by specifying additional arguments of plot.default()
plot(out4, mgp = c(2, 1, 0), main = "Trial 13", font.main = 1)

## An example of non-significance altered to significance
out5 <- frag.study(e0 = e0, n0 = n0, e1 = e1, n1 = n1, data = dat.ad[21,],
  all = TRUE)
out5
plot(out5, method = "Fisher", legend.pvals = 0.001,
  main = paste("Fragility index =", out5$FI["Fisher"], "in trial 21"))

## Change the statistical significance level to 0.005
out6 <- frag.study(e0 = e0, n0 = n0, e1 = e1, n1 = n1, data = dat.ad[21,],
  all = TRUE, alpha = 0.005)
out6
plot(out6)

## Change the null value of odds ratio to 2
out7 <- frag.study(e0 = e0, n0 = n0, e1 = e1, n1 = n1, data = dat.ad[21,],
  all = TRUE, methods = c("OR"), alpha = 0.05, OR = 2)
out7
plot(out7)