Learn R Programming

metaSEM (version 2.0)

summary: Summary Method for tssem1, wls, meta, and meta3LFIML Objects

Description

For fixed-effects TSSEM and WLS summaries with positive degrees of freedom, the RMSEA estimate is $$\sqrt{G\max(\chi^2-df,0)/(N\,df)}$$ where \(\chi^2\) is the target model statistic. For fixed-effects TSSEM, \(N\) is the sum of the study sample sizes and \(G\) is the number of studies. For WLS, \(N\) is the supplied n and \(G=1\); tssem2() supplies the total sample size from the first stage. Confidence limits transform the corresponding noncentrality limits using the same \(N\) and \(G\). The denominator is \(N\), not \(N-G\) or \(N-1\); this scaling matches the normal-ML convention used by lavaan. For WLS, the scaling of the supplied sampling covariance matrix should also be considered when comparing fit statistics. When \(df=0\), RMSEA and its confidence limits are reported as zero; when \(df<0\), they are NA.

It summaries results for various class.

Usage

# S3 method for tssem1FEM
summary(object, ...)
# S3 method for tssem1FEM.cluster
summary(object, ...)
# S3 method for tssem1REM
summary(object, robust = FALSE, ...)
# S3 method for wls
summary(object, df.adjustment = 0, ...)
# S3 method for wls.cluster
summary(object, df.adjustment = 0, ...)
# S3 method for meta
summary(object, homoStat = TRUE, robust = FALSE, ...)
# S3 method for meta3LFIML
summary(object, allX = FALSE, robust = FALSE, ...)
# S3 method for reml
summary(object, ...)
# S3 method for CorPop
summary(object, ...)
# S3 method for Cor3L
summary(object, ...)
# S3 method for bootuniR2
summary(object, probs = c(0, 0.1, 0.5, 0.9, 1), cutoff.chisq.pvalue = 0.05,
    cutoff.CFI = 0.9, cutoff.SRMR = 0.1, cutoff.RMSEA = 0.05, ...)
# S3 method for osmasem
summary(object, fitIndices = FALSE, numObs, robust = FALSE, ...)
# S3 method for osmasem2
summary(object, fitIndices = FALSE, numObs, robust = FALSE, mxTryHard = FALSE,
    ...)
# S3 method for mxsem
summary(object, robust = FALSE, smallSample = FALSE, t.df = NULL, ...)
# S3 method for summary.tssem1FEM
print(x, ...)
# S3 method for summary.wls
print(x, ...)
# S3 method for summary.meta
print(x, ...)
# S3 method for summary.meta3LFIML
print(x, ...)
# S3 method for summary.reml
print(x, ...)
# S3 method for summary.CorPop
print(x, ...)
# S3 method for summary.Cor3L
print(x, ...)
# S3 method for summary.bootuniR2
print(x, ...)
# S3 method for summary.mxsem
print(x, ...)

Arguments

mxTryHard

Logical. For osmasem2 fit indices, whether to use mxTryHard when fitting the comparison models.

object

An object returned from either class tssem1FEM, class tssem1FEM.cluster, class tssem1REM, class wls, class wls.cluster, class meta, class meta3LFIML, class reml, class mxsem or class CorPop.

x

An object returned from either class summary.tssem1FEM, class tssem1FEM.cluster, class summary.wls, class summary.meta, class summary.meta3LFIML, class summary.reml or class summary.CorPop.

homoStat

Logical. Whether to conduct a homogeneity test on the effect sizes.

allX

Logical. Whether to report the predictors and the auxiliary variables.

robust

Logical. Whether to use robust (sandwich) standard errors. For meta, meta3LFIML, and tssem1REM objects they are from imxRobustSE. For mxsem objects, see the section ‘Robust and small-sample inference for mxsem objects’.

smallSample

Logical, for mxsem objects only. If TRUE (with robust=TRUE), the cluster-robust covariance of a single-level clustered fit is also multiplied by \((N-1)/(N-p)\), where \(N\) is the number of rows and \(p\) the number of free parameters, including those determined by constraints or at a bound. Not available for two-level fits. See clusterRobustSE.

t.df

NULL or a single positive number, for mxsem objects only. If NULL (the default), Wald tests and 95% Wald confidence intervals use the standard normal distribution (\(z\)). If a number, they use a \(t\) distribution with t.df degrees of freedom: the \(p\) values and intervals of all parameters and the delta-method intervals of defined (:=) parameters. The standard errors themselves are unchanged. Requires intervals.type="z". It can be combined with model-based or robust standard errors, but it is intended for cluster-robust inference with few independent clusters; see the section ‘Robust and small-sample inference for mxsem objects’ for how to choose it.

df.adjustment

Numeric. Adjust the degrees of freedom manually. It may be necessary if the df calculated is incorrect when diag.constraints=TRUE. For wls.cluster objects, the adjustment is applied to each fitted model.

probs

Quantiles for the parameter estimates.

cutoff.chisq.pvalue

Cutoff of the p-value for the chi-square statistic.

cutoff.CFI

The cutoff of the CFI.

cutoff.SRMR

The cutoff of the SRMR.

cutoff.RMSEA

The cutoff of the RMSEA.

fitIndices

Whether to calculate the chi-square statistic and various goodness-of-fit indices in osmasem. Note. It may take a while since statistics of the saturated and independence models are required.

numObs

The number of observations in calculating the fit statistics in osmasem. If it is missing, the total number of observations is used.

...

Further arguments to be passed to printCoefmat

Robust and small-sample inference for mxsem objects

summary(fit, robust=TRUE) replaces the model-based standard errors of an mxsem fit by sandwich standard errors: an observation-level sandwich for a single-level fit without cluster, a cluster sandwich for a single-level fit with cluster (see clusterRobustSE), and a cluster-score sandwich for a two-level fit (see sem). A cluster sandwich is justified as the number of independent clusters grows (studies in a meta-analysis, not effect sizes). With few clusters its standard errors tend to be too small, and \(z\) tests and intervals are then liberal.

t.df replaces the \(z\) reference distribution by a \(t\) distribution with the given degrees of freedom, which widens the intervals and raises the \(p\) values when t.df is small. It does not change the estimates or the standard errors, and it is not computed automatically, because the appropriate value depends on the design and the parameter:

  • Fixed effects of a meta-analysis (the intercept and the regression coefficients of moderators) with cluster-robust standard errors: t.df = K - p, where \(K\) is the number of independent studies (clusters; reported by summary() as “Number of clusters”, not the number of effect sizes) and \(p\) the number of fixed-effect coefficients in the mean model, including the intercept. For example, an intercept and two moderators give \(p=3\). This is the reference distribution of robust variance estimation proposed by Hedges, Tipton, and Johnson (2010), and of metafor::robust(). Note that \(p\) counts the coefficients of the mean model only, not all free parameters as in smallSample.

  • Other clustered models: t.df = G - 1, where \(G\) is the number of independent clusters, is a common convention for cluster-robust tests (Cameron & Miller, 2015; McNeish, 2026). In multiple-group models, \(G\) is the total number of clusters.

Limitations:

  • The same t.df is applied to every parameter. The literature that justifies it concerns regression coefficients and intercepts of the mean model. It does not justify tests of variance components, factor loadings, or other covariance parameters, and it does not address estimates at or near a bound, whose robust standard errors are NA.

  • \(K - p\) can still be too large, and the test liberal, when a moderator is unbalanced or varies mainly between studies (Tipton, 2015). Tests with fewer than about 4 degrees of freedom are unreliable. In those cases, the CR2 correction with Satterthwaite degrees of freedom (Pustejovsky & Tipton, 2018), available in the clubSandwich package for meta-regression models, is preferable.

  • It does not affect likelihood-ratio tests or model-fit statistics.

References:

  • Cameron, A. C., & Miller, D. L. (2015). A practitioner's guide to cluster-robust inference. Journal of Human Resources, 50(2), 317--372.

  • Hedges, L. V., Tipton, E., & Johnson, M. C. (2010). Robust variance estimation in meta-regression with dependent effect size estimates. Research Synthesis Methods, 1(1), 39--65. tools:::Rd_expr_doi("10.1002/jrsm.5")

  • McNeish, D. (2026). A practical guide to selecting and blending approaches for clustered data: Clustered errors, multilevel models, and fixed-effect models. Psychological Methods, 31(2), 225--251. tools:::Rd_expr_doi("10.1037/met0000620")

  • Pustejovsky, J. E., & Tipton, E. (2018). Small-sample methods for cluster-robust variance estimation and hypothesis testing in fixed effects models. Journal of Business & Economic Statistics, 36(4), 672--683. tools:::Rd_expr_doi("10.1080/07350015.2016.1247004")

  • Tipton, E. (2015). Small sample adjustments for robust variance estimation with meta-regression. Psychological Methods, 20(3), 375--393. tools:::Rd_expr_doi("10.1037/met0000011")

Author

Mike W.-L. Cheung <[email protected]>

See Also

tssem1, wls, meta, reml, rCor, bootuniR2, osmasem

Examples

Run this code
# \donttest{
## Flat meta-regression of 66 effect sizes nested in 21 clusters, with
## study-clustered robust standard errors.
data(Bornmann07)
Bornmann07$Fellowship <- ifelse(Bornmann07$Type == "Fellowship", 1, 0)
model <- "f =~ 1*logOR
          f ~~ tau2*f
          logOR ~~ data.v*logOR
          f ~ b0*1
          logOR ~ b1*Fellowship"
RAM <- lavaan2RAM(model, obs.variables=c("logOR", "Fellowship"),
                  std.lv=FALSE)
fit <- sem(RAM=RAM, data=Bornmann07, cluster="Cluster")

## z tests and intervals
summary(fit, robust=TRUE)

## t tests and intervals with K - p degrees of freedom: K = 21 clusters
## and p = 2 fixed effects (b0 and b1). Only the rows of b0 and b1 are
## justified by the literature.
K <- length(unique(Bornmann07$Cluster))
summary(fit, robust=TRUE, t.df=K - 2)
# }

Run the code above in your browser using DataLab