hybrid: hybrid

Description

Function to statistically combine conventional and preregistered/replications studies by means of the hybrid methods as described in van Aert and van Assen (2018) and van Aert (2023).

Usage

hybrid(
  m1i,
  m2i,
  mi,
  ri,
  sd1i,
  sd2i,
  sdi,
  n1i,
  n2i,
  ni,
  tobs,
  yi,
  vi,
  conventional,
  mods = NULL,
  alpha = 0.05,
  side,
  control
)

Value

k: total number of effect sizes
k.conventional: number of effect sizes of conventional studies
est: parameter estimates of the fixed effects of the hybrid method
tau2: estimate of the between-study variance in true effect size of hybrid method
se: standard error of the fixed effects and of the estimated between-study variance
ci.lb: lower bound of hybrid method's confidence interval of the average effect size
ci.ub: upper bound of hybrid method's confidence interval of the average effect size
L.0: test statistic of hybrid method's test of null-hypothesis of no effect. This is either a z-value (zval in the output) or a chi-square value (LR in the output) in case of a likelihood-ratio test
pval.0: p-value of hybrid method's test of null-hypothesis of no effect
tau2: estimate of the between-study variance in true effect size of hybrid method
tau2.lb: lower bound of hybrid method's confidence interval of the between-study variance
tau2.ub: upper bound of hybrid method's confidence interval of the between-study variance
L.het: test statistic of hybrid method's test of null-hypothesis of no heterogeneity This is either a z-value (zval in the output) or a chi-square value (LR in the output) in case of a likelihood-ratio test
pval.het: p-value of hybrid method's test of null-hypothesis of no heterogeneity
optim.info: model fitting results if the implementation of van Aert (2023) is used

The elements below are only returned if the deprecated implementation of van Aert and van Assen (2018) is used:

est.hyr: effect size estimate of hybridR method
ci.lb.hyr: lower bound of hybridR method's confidence interval of the effect size
ci.ub.hyr: upper bound of hybridR method's confidence interval of the effect size
L.0.hyr: test statistic of hybridR method's test of null-hypothesis of no effect
pval.0.hyr: p-value of hybridR method's test of null-hypothesis of no effect
pval.o: two-tailed p-value of conventional study
est.hy0: effect size estimate of hybrid0 method
ci.lb.hy0: lower bound of hybrid0 method's confidence interval of the effect size
ci.ub.hy0: upper bound of hybrid0 method's confidence interval of the effect size
L.0.hy0: test statistic of hybrid0 method's test of null-hypothesis of no effect
pval.0.hy0: two-tailed p-value of hybrid0 method's test of null-hypothesis of no effect
est.fe: effect size estimate based on fixed-effect meta-analysis
se.fe: standard error of effect size estimate based on traditional fixed-effect meta-analysis
zval.fe: test statistic of the null-hypothesis of no effect based on fixed-effect meta-analysis
pval.fe: two-tailed p-value of the null-hypothesis of no effect based on fixed-effect meta-analysis
ci.lb.fe: lower bound of confidence interval based on traditional fixed-effect meta-analysis
ci.ub.fe: upper bound of confidence interval based on fixed-effect meta-analysis
est.repl: effect size estimate of replication
se.repl: standard error of replication's effect size estimate
ci.lb.repl: lower bound of replication's confidence interval
ci.ub.repl: upper bound of replication's confidence interval
stat.repl: test statistic of replication for testing null-hypothesis of no effect
pval.repl: two-tailed p-value of replication for testing null-hypothesis of no effect

Arguments

m1i: A vector of means in group 1 for two-independent means
m2i: A vector of means in group 2 for two-independent means
mi: A vector of group means for one-sample means
ri: A vector of raw correlations
sd1i: A vector of standard deviations in group 1 for two-independent means
sd2i: A vector of standard deviations in group 2 for two-independent means
sdi: A vector of standard deviations for one-sample means
n1i: A vector of sample sizes in group 1 for two-independent means
n2i: A vector of sample sizes in group 2 for two-independent means
ni: A vector of sample sizes for one-sample means and correlations
tobs: A vector of t-values
yi: A vector of standardized effect sizes (see Details)
vi: A vector of sampling variances belonging to the standardized effect sizes (see Details)
conventional: A vector indicating whether a study is a conventional study (indicated with a 1) and therefore was susceptible to bias. Studies not susceptible to bias are indicated with a 0
mods: A one-sided formula to specify the moderators to include. For example x1 can be included as moderator by specifying mods = ~ x1
alpha: A numerical value specifying the alpha level as used in the conventional study (default is 0.05, see Details)
side: A character indicating whether the observed effect size of the conventional studies are in the right-tail of the distribution (i.e., positive) or in the left-tail of the distribution (i.e., negative) (either "right" or "left")
control: An optional list of elements that give the user more control over the estimation procedures (see Note)

Author

Robbie C.M. van Aert R.C.M.vanAert@tilburguniversity.edu

Details

Three different effect sizes can be used as input for the hybrid function: one-sample means, two-independent means, and raw correlation coefficients. Analyzing one-sample means and two-independent means can be done by either providing the function group means (mi or m1i and m2i), standard deviations (sdi or sd1i and sd2i), and sample sizes (ni or n1i and n2i) or t-values (tobs) and sample sizes (ni or n1i and n2i). Pearson correlation coefficients can be analyzed by supplying ri and ni to the hybrid function. These correlation coefficients are internally transformed to Fisher's z correlations before analyzing the data. The results in the output are of the Fisher's z transformed correlations. It is also possible to specify the standardized effect sizes and its sampling variances directly via the yi and vi arguments.

Two other arguments that need to be specified are side and conventional. side indicates whether the effect size in the conventional study was expected to be in the right-tail (side = "right" or in the left-tail (side = "left") of the distribution. The argument conventional has to be used to indicate which studies are conventional studies and expected to be susceptible to bias. A 1 indicates that a study was susceptible to bias and a 0 indicates that a study was not susceptible to bias.

It is assumed that two-tailed hypothesis tests were conducted in the conventional studies. In case one-tailed hypothesis tests were conducted in the conventional studies, the alpha level has to be multiplied by two. For example, if one-tailed hypothesis tests were conducted with an alpha level of .05, an alpha of 0.1 has to be supplied to the hybrid function.

Previous version

The usage of a previous version of the hybrid function was more restricted. Users could only apply the method to a single conventional study and replication. Before the addition of the extra functionality to also analyze multiple conventional studies and replications, data of the conventional study and replication were specified in vectors containing two elements with the first element being the data of the conventional study and the second one the data of the replication. In order to maintain backwards compatibility, it is still possible to analyze data like this by using the arguments m1i, m2i, mi, ri, sd1i, sd2i, sdi, n1i, n2i, ni, tobs. However, using the hybrid function in this way is now deprecated.

References

van Aert, R. C. M., & van Assen, M. A. L. M. (2018). Examining reproducibility in psychology: A hybrid method for statistically combining a biased original study and replication. Behavior Research Methods, 50(4): 1515-1539. doi:10.3758/s13428-017-0967-6

van Aert, R.C.M. (2023). Empowering meta-analysis by taking advantage of preregistered studies: The extended hybrid meta-analysis method. Manuscript in preparation.

Examples

Run this code


### Application using the implementation of van Aert (2023) with more than two studies.
# Note that these data come from the "metadat" R package that needs to be loaded.
# The data come from the meta-analyses by Lehmann et al. (2018). See the help 
# file of dat.lehmann2018 for more information about this meta-analysis.
library(metadat)
dat <- dat.lehmann2018

### Create a new object to indicate which studies were conventional studies and 
# which ones were preregistered
dat$conventional <- ifelse(dat$Preregistered == "Not Pre-Registered", 1, 0)

### Lehmann et al. split the analyses for males and females. We only use the data
# of females for this example.
red_romance_femalep <- dat[dat$Gender == "Females", ]

### Prepare data for the analysis
yi <- red_romance_femalep$yi
vi <- red_romance_femalep$vi
conventional <- red_romance_femalep$conventional

### Apply the hybrid method with Wald-type hypothesis tests and confidence intervals
hybrid(yi = yi, vi = vi, conventional = conventional, side = "right")

### Apply the hybrid method with likelihood ratio hypothesis tests and profile 
# likelihood confidence intervals
hybrid(yi = yi, vi = vi, conventional = conventional, side = "right", 
control = list(type = "profile"))

### Include Color_Match as moderator
Color_Match <- red_romance_femalep$Color_Match

### Apply the hybrid method with Wald-type hypothesis tests and confidence intervals
hybrid(yi = yi, vi = vi, conventional = conventional, side = "right", mods = ~ Color_Match)

### Apply the hybrid method with likelihood ratio hypothesis tests and profile 
# likelihood confidence intervals
hybrid(yi = yi, vi = vi, conventional = conventional, side = "right", mods = ~ Color_Match, 
control = list(type = "profile"))

### Application using the implementation of van Aert and van Assen (2018). 
# The hybrid method is applied to example on page 5 of van Aert and van Assen 
# (2018).

pval <- c(0.03, 0.3) # p-value conventional study and replication
n1i <- n2i <- c(40, 80) # Sample sizes per group 
tobs <- qt(pval/2, df = n1i+n2i-2, lower.tail = FALSE) # Observed t-values 

### Apply hybrid method using the implementation of van Aert and van Assen (2018)
hybrid(tobs = tobs, n1i = n1i, n2i = n2i, side = "right", conventional = c(1,0), 
control = list(implementation = "two"))

Run the code above in your browser using DataLab