R package **meta** is a user-friendly general package providing
standard methods for meta-analysis and supporting Schwarzer et
al. (2015),
https://link.springer.com/book/10.1007/978-3-319-21416-0.

Guido Schwarzer guido.schwarzer@uniklinik-freiburg.de

R package **meta** (Schwarzer, 2007; Balduzzi et al., 2019)
provides the following statistical methods for meta-analysis.

Common effect (also called fixed effect) and random effects model:

Meta-analysis of continuous outcome data (

`metacont`

)Meta-analysis of binary outcome data (

`metabin`

)Meta-analysis of incidence rates (

`metainc`

)Generic inverse variance meta-analysis (

`metagen`

)Meta-analysis of single correlations (

`metacor`

)Meta-analysis of single means (

`metamean`

)Meta-analysis of single proportions (

`metaprop`

)Meta-analysis of single incidence rates (

`metarate`

)

Several plots for meta-analysis:

Forest plot (

`forest.meta`

,`forest.metabind`

)Funnel plot (

`funnel.meta`

)Galbraith plot / radial plot (

`radial.meta`

)L'Abbe plot for meta-analysis with binary outcome data (

`labbe.metabin`

,`labbe.default`

)Baujat plot to explore heterogeneity in meta-analysis (

`baujat.meta`

)Bubble plot to display the result of a meta-regression (

`bubble.metareg`

)

Three-level meta-analysis model (Van den Noortgate et al., 2013)

Generalised linear mixed models (GLMMs) for binary and count data (Stijnen et al., 2010) (

`metabin`

,`metainc`

,`metaprop`

, and`metarate`

)Various estimators for the between-study variance \(\tau^2\) in a random effects model (Veroniki et al., 2016); see description of argument

`method.tau`

belowHartung-Knapp method for random effects meta-analysis (Hartung & Knapp, 2001a,b), see description of arguments

`method.random.ci`

and`adhoc.hakn.ci`

belowKenward-Roger method for random effects meta-analysis (Partlett and Riley, 2017), see description of arguments

`method.random.ci`

and`method.predict`

belowPrediction interval for the treatment effect of a new study (Higgins et al., 2009; Partlett and Riley, 2017; Nagashima et al., 2019), see description of argument

`method.predict`

belowStatistical tests for funnel plot asymmetry (

`metabias.meta`

,`metabias.rm5`

) and trim-and-fill method (`trimfill.meta`

,`trimfill.default`

) to evaluate bias in meta-analysisMeta-regression (

`metareg`

)Cumulative meta-analysis (

`metacum`

) and leave-one-out meta-analysis (`metainf`

)Import data from Review Manager 5 (

`read.rm5`

), see also`metacr`

to conduct meta-analysis for a single comparison and outcome from a Cochrane review

Additional statistical meta-analysis methods are provided by add-on R packages:

Frequentist methods for network meta-analysis (R package

**netmeta**)Statistical methods for sensitivity analysis in meta-analysis (R package

**metasens**)Statistical methods for meta-analysis of diagnostic accuracy studies with several cutpoints (R package

**diagmeta**)

In the following, more details on available and default statistical
meta-analysis methods are provided and R function
`settings.meta`

is briefly described which can be used
to change the default settings. Additional information on
meta-analysis objects and available summary measures can be found
on the help pages `meta-object`

and
`meta-sm`

.

The following methods are available in all meta-analysis functions to estimate the between-study variance \(\tau^2\).

Argument | Method |

`method.tau = "REML"` | Restricted maximum-likelihood estimator (Viechtbauer, 2005) |

(default) | |

`method.tau = "PM"` | Paule-Mandel estimator (Paule and Mandel, 1982) |

`method.tau = "DL"` | DerSimonian-Laird estimator (DerSimonian and Laird, 1986) |

`method.tau = "ML"` | Maximum-likelihood estimator (Viechtbauer, 2005) |

`method.tau = "HS"` | Hunter-Schmidt estimator (Hunter and Schmidt, 2015) |

`method.tau = "SJ"` | Sidik-Jonkman estimator (Sidik and Jonkman, 2005) |

`method.tau = "HE"` | Hedges estimator (Hedges and Olkin, 1985) |

`method.tau = "EB"` | Empirical Bayes estimator (Morris, 1983) |

For GLMMs, only the maximum-likelihood method is available.

Historically, the DerSimonian-Laird method was the de facto
standard to estimate the between-study variance \(\tau^2\) and is
the default in some software packages including Review Manager 5
(RevMan 5) and R package **meta**, version 4 and below. However,
its role has been challenged and especially the REML and
Paule-Mandel estimators have been recommended (Veroniki et al.,
2016; Langan et al., 2019). Accordingly, the currenct default in R
package **meta** is the REML estimator.

The following R command could be used to employ the Paule-Mandel instead of the REML estimator in all meta-analyses of the current R session:

`settings.meta(method.tau = "PM")`

Other estimators for \(\tau^2\) could be selected in a similar way.

Note, for binary outcomes, two variants of the DerSimonian-Laird
estimator are available if the Mantel-Haenszel method is used for
pooling. If argument `Q.Cochrane = TRUE`

(default), the
heterogeneity statistic Q is based on the Mantel-Haenszel instead
of the inverse variance estimator under the common effect
model. This is the estimator for \(\tau^2\) implemented in RevMan
5.

The following methods are available in all meta-analysis functions to calculate a confidence interval for the random effects estimate.

Argument | Method |

`method.random.ci = "classic"` | Based on standard normal quantile |

(DerSimonian and Laird, 1986) (default) | |

`method.random.ci = "HK"` | Method by Hartung and Knapp (2001a/b) |

`method.random.ci = "KR"` | Kenward-Roger method (Partlett and Riley, 2017) |

DerSimonian and Laird (1986) introduced the classic random effects model using a quantile of the standard normal distribution to calculate a confidence interval for the random effects estimate. This method implicitly assumes that the weights in the random effects meta-analysis are not estimated but given. Particularly, the uncertainty in the estimation of the between-study variance \(\tau^2\) is ignored.

Hartung and Knapp (2001a,b) proposed an alternative method for
random effects meta-analysis based on a refined variance estimator
for the treatment estimate and a quantile of a
*t*-distribution with *k-1* degrees of freedom where
*k* corresponds to the number of studies in the
meta-analysis.

The Kenward-Roger method is only available for the REML estimator
(`method.tau = "REML"`

) of the between-study variance
\(\tau^2\) (Partlett and Riley, 2017). This method is based on an
adjusted variance estimate for the random effects
estimate. Furthermore, a quantile of a *t*-distribution with
adequately modified degrees of freedom is used to calculate the
confidence interval.

For GLMMs and three-level models, the Kenward-Roger method is not
available, but a method similar to Knapp and Hartung (2003) is used
if `method.random.ci = "HK"`

. For this method, the variance
estimator is not modified, however, a quantile of a
*t*-distribution with *k-1* degrees of freedom is used;
see description of argument `test`

in
`rma.glmm`

and `rma.mv`

.

Simulation studies (Hartung and Knapp, 2001a,b; IntHout et al.,
2014; Langan et al., 2019) show improved coverage probabilities of
the Hartung-Knapp method compared to the classic random effects
method. However, in rare settings with very homogeneous treatment
estimates, the Hartung-Knapp variance estimate can be arbitrarily
small resulting in a very narrow confidence interval (Knapp and
Hartung, 2003; Wiksten et al., 2016). In such cases, an *ad
hoc* variance correction has been proposed by utilising the
variance estimate from the classic random effects model with the
Hartung-Knapp method (Knapp and Hartung, 2003; IQWiQ, 2022). An
alternative *ad hoc* approach is to use the confidence
interval of the classic common or random effects meta-analysis if
it is wider than the interval from the Hartung-Knapp method
(Wiksten et al., 2016; Jackson et al., 2017).

Argument `adhoc.hakn.ci`

can be used to choose the *ad
hoc* correction for the Hartung-Knapp (HK) method:

Argument | Ad hoc method |

`adhoc.hakn.ci = ""` | no ad hoc correction (default) |

`adhoc.hakn.ci = "se"` | use variance correction if HK standard error is smaller |

than standard error from classic random effects | |

meta-analysis (Knapp and Hartung, 2003) | |

`adhoc.hakn.ci = "IQWiG6"` | use variance correction if HK confidence interval |

is narrower than CI from classic random effects model | |

with DerSimonian-Laird estimator (IQWiG, 2022) | |

`adhoc.hakn.ci = "ci"` | use wider confidence interval of classic random effects |

and HK meta-analysis | |

(Hybrid method 2 in Jackson et al., 2017) |

For GLMMs and three-level models, the *ad hoc* variance
corrections are not available.

The following methods are available in all meta-analysis functions to calculate a prediction interval for the treatment effect in a single new study.

Argument | Method |

`method.predict = "HTS"` | Based on t-distribution
(Higgins et al., 2009) (default) |

`method.predict = "HK"` | Hartung-Knapp method (Partlett and Riley, 2017) |

`method.predict = "KR"` | Kenward-Roger method (Partlett and Riley, 2017) |

`method.predict = "NNF"` | Bootstrap approach (Nagashima et al., 2019) |

`method.predict = "S"` | Based on standard normal quantile (Skipka, 2006) |

By default (`method.predict = "HTS"`

), the prediction interval
is based on a *t*-distribution with *k-2* degrees of
freedom where *k* corresponds to the number of studies in the
meta-analysis, see equation (12) in Higgins et al. (2009).

The Kenward-Roger method is only available for the REML estimator
(`method.tau = "REML"`

) of the between-study variance
\(\tau^2\) (Partlett and Riley, 2017). This method is based on an
adjusted variance estimate for the random effects
estimate. Furthermore, a quantile of a *t*-distribution with
adequately modified degrees of freedom minus 1 is used to calculate
the prediction interval.

The bootstrap approach is only available if R package **pimeta**
is installed (Nagashima et al., 2019). Internally, the
`pima`

function is called with argument
`method = "boot"`

. Argument `seed.predict`

can be used to
get a reproducible bootstrap prediction interval and argument
`seed.predict.subgroup`

for reproducible bootstrap prediction
intervals in subgroups.

The method of Skipka (2006) ignores the uncertainty in the estimation of the between-study variance \(\tau^2\) and thus has too narrow limits for meta-analyses with a small number of studies.

For GLMMs and three-level models, the Kenward-Roger method and the
bootstrap approach are not available, but a method similar to Knapp
and Hartung (2003) is used if `method.random.ci = "HK"`

. For
this method, the variance estimator is not modified, however, a
quantile of a *t*-distribution with *k-2* degrees of
freedom is used; see description of argument `test`

in
`rma.glmm`

and `rma.mv`

.

Argument `adhoc.hakn.pi`

can be used to choose the *ad
hoc* correction for the Hartung-Knapp method:

Argument | Ad hoc method |

`adhoc.hakn.pi = ""` | no ad hoc correction (default) |

`adhoc.hakn.pi = "se"` | use variance correction if HK standard error is smaller |

For GLMMs and three-level models, the *ad hoc* variance
corrections are not available.

The following methods are available in all meta-analysis functions to calculate a confidence interval for \(\tau^2\) and \(\tau\).

Argument | Method |

`method.tau.ci = "J"` | Method by Jackson (2013) |

`method.tau.ci = "BJ"` | Method by Biggerstaff and Jackson (2008) |

`method.tau.ci = "QP"` | Q-Profile method (Viechtbauer, 2007) |

`method.tau.ci = "PL"` | Profile-Likelihood method for three-level |

meta-analysis model (Van den Noortgate et al., 2013) | |

`method.tau.ci = ""` | No confidence interval |

The first three methods have been recommended by Veroniki et al. (2016). By default, the Jackson method is used for the DerSimonian-Laird estimator of \(\tau^2\) and the Q-profile method for all other estimators of \(\tau^2\).

The Profile-Likelihood method is the only method available for the three-level meta-analysis model.

For GLMMs, no confidence intervals for \(\tau^2\) and \(\tau\) are calculated.

R function `settings.meta`

can be used to change the
previously described and several other default settings for the
current R session.

Some pre-defined general settings are available:

`settings.meta("RevMan5")`

`settings.meta("JAMA")`

`settings.meta("IQWiG5")`

`settings.meta("IQWiG6")`

`settings.meta("geneexpr")`

The first command can be used to reproduce meta-analyses from
Cochrane reviews conducted with *Review Manager 5* (RevMan 5,
https://training.cochrane.org/online-learning/core-software/revman)
and specifies to use a RevMan 5 layout in forest plots.

The second command can be used to generate forest plots following
instructions for authors of the *Journal of the American
Medical Association*
(https://jamanetwork.com/journals/jama/pages/instructions-for-authors/). Study
labels according to JAMA guidelines can be generated using
`labels.meta`

.

The next two commands implement the recommendations of the Institute for Quality and Efficiency in Health Care (IQWiG), Germany accordinging to General Methods 5 and 6, respectively (https://www.iqwig.de/en/about-us/methods/methods-paper/).

The last setting can be used to print p-values in scientific notation and to suppress the calculation of confidence intervals for the between-study variance.

See `settings.meta`

for more details on these
pre-defined general settings.

In addition, `settings.meta`

can be used to define
individual settings for the current R session. For example, the
following R command specifies the use of Hartung-Knapp and
Paule-Mandel method, and the printing of prediction intervals for
any meta-analysis generated after execution of this command:

`settings.meta(method.random.ci = "HK", method.tau = "PM", prediction = TRUE)`

Balduzzi S, Rücker G, Schwarzer G (2019):
How to perform a meta-analysis with R: a practical tutorial.
*Evidence-Based Mental Health*,
**22**, 153--160

Biggerstaff BJ, Jackson D (2008):
The exact distribution of Cochran’s heterogeneity statistic in
one-way random effects meta-analysis.
*Statistics in Medicine*,
**27**, 6093--110

DerSimonian R & Laird N (1986):
Meta-analysis in clinical trials.
*Controlled Clinical Trials*,
**7**, 177--88

Hartung J, Knapp G (2001a):
On tests of the overall treatment effect in meta-analysis with
normally distributed responses.
*Statistics in Medicine*,
**20**, 1771--82

Hartung J, Knapp G (2001b):
A refined method for the meta-analysis of controlled clinical
trials with binary outcome.
*Statistics in Medicine*,
**20**, 3875--89

Hedges LV & Olkin I (1985):
*Statistical methods for meta-analysis*.
San Diego, CA: Academic Press

Higgins JPT, Thompson SG, Spiegelhalter DJ (2009):
A re-evaluation of random-effects meta-analysis.
*Journal of the Royal Statistical Society: Series A*,
**172**, 137--59

Hunter JE & Schmidt FL (2015):
*Methods of Meta-Analysis: Correcting Error and Bias in
Research Findings* (Third edition).
Thousand Oaks, CA: Sage

IntHout J, Ioannidis JPA, Borm GF (2014):
The Hartung-Knapp-Sidik-Jonkman method for random effects
meta-analysis is straightforward and considerably outperforms the
standard DerSimonian-Laird method.
*BMC Medical Research Methodology*,
**14**, 25

IQWiG (2022): General Methods: Version 6.1. https://www.iqwig.de/en/about-us/methods/methods-paper/

Jackson D (2013):
Confidence intervals for the between-study variance in random
effects meta-analysis using generalised Cochran heterogeneity
statistics.
*Research Synthesis Methods*,
**4**, 220--229

Jackson D, Law M, Rücker G, Schwarzer G (2017):
The Hartung-Knapp modification for random-effects meta-analysis: A
useful refinement but are there any residual concerns?
*Statistics in Medicine*,
**36**, 3923--34

Knapp G & Hartung J (2003):
Improved tests for a random effects meta-regression with a single
covariate.
*Statistics in Medicine*,
**22**, 2693--710

Langan D, Higgins JPT, Jackson D, Bowden J, Veroniki AA,
Kontopantelis E, et al. (2019):
A comparison of heterogeneity variance estimators in simulated
random-effects meta-analyses.
*Research Synthesis Methods*,
**10**, 83--98

Schwarzer G (2007):
meta: An R package for meta-analysis.
*R News*,
**7**, 40--5

Schwarzer G, Carpenter JR and Rücker G (2015):
*Meta-Analysis with R (Use-R!)*.
Springer International Publishing, Switzerland

Skipka G (2006):
The inclusion of the estimated inter-study variation into forest
plots for random effects meta-analysis - a suggestion for a
graphical representation [abstract].
*XIV Cochrane Colloquium, Dublin*, 23-26.

Stijnen T, Hamza TH, Ozdemir P (2010):
Random effects meta-analysis of event outcome in the framework of
the generalized linear mixed model with applications in sparse
data.
*Statistics in Medicine*,
**29**, 3046--67

Veroniki AA, Jackson D, Viechtbauer W, Bender R, Bowden J, Knapp G,
et al. (2016):
Methods to estimate the between-study variance and its uncertainty
in meta-analysis.
*Research Synthesis Methods*,
**7**, 55--79

Van den Noortgate W, López-López JA, Marín-Martínez F, Sánchez-Meca J (2013):
Three-level meta-analysis of dependent effect sizes.
*Behavior Research Methods*,
**45**, 576--94

Viechtbauer W (2005):
Bias and efficiency of meta-analytic variance estimators in the
random-effects model.
*Journal of Educational and Behavioral Statistics*,
**30**, 261--93

Viechtbauer W (2007):
Confidence intervals for the amount of heterogeneity in
meta-analysis.
*Statistics in Medicine*,
**26**, 37--52

Viechtbauer W (2010):
Conducting Meta-Analyses in R with the metafor Package.
*Journal of Statistical Software*,
**36**, 1--48

Wiksten A, Rücker G, Schwarzer G (2016):
Hartung-Knapp method is not always conservative compared with
fixed-effect meta-analysis.
*Statistics in Medicine*,
**35**, 2503--15

`meta-object`

, `meta-sm`