Function to apply the p-uniform* method for one-sample mean, two-independent means, and one raw correlation coefficient as described in van Aert and van Assen (2023).
puni_star(
mi,
ri,
ni,
sdi,
m1i,
m2i,
n1i,
n2i,
sd1i,
sd2i,
tobs,
yi,
vi,
alpha = 0.05,
side,
method = "ML",
boot = FALSE,
control
)p-uniform*'s effect size estimate
lower bound of p-uniform*'s 95% confidence interval of the effect size
upper bound of p-uniform*'s 95% confidence interval of the effect size
test statistic of p-uniform*'s test of the null hypothesis of no effect
one-tailed p-value of p-uniform*'s test of null hypothesis of no effect
p-uniform*'s estimate of the between-study variance
lower bound of p-uniform*'s 95% confidence interval of the between-study variance
upper bound of p-uniform*'s 95% confidence interval of the between-study variance
test statistic of p-uniform*'s test of the null hypothesis of no between-study variance
one-tailed p-value of p-uniform*'s test of null hypothesis of no between-study variance
one-tailed p-value of p-uniform*'s test of null hypothesis of no between-study variance obtained with a parametric bootstrap
a number of additional elements
A vector of group means for one-sample means
A vector of raw correlations
A vector of sample sizes for one-sample means and correlations
A vector of standard deviations for one-sample means
A vector of means in group 1 for two-independent means
A vector of means in group 2 for two-independent means
A vector of sample sizes in group 1 for two-independent means
A vector of sample sizes in group 2 for two-independent means
A vector of standard deviations in group 1 for two-independent means
A vector of standard deviations in group 2 for two-independent means
A vector of t-values
A vector of standardized effect sizes (see Details)
A vector of sampling variances belonging to the standardized effect sizes (see Details)
A numerical value specifying the alpha level as used in primary studies (default is 0.05 but see Details).
A character indicating whether the effect sizes in the primary studies
are in the right-tail of the distribution (i.e., positive) or in the left-tail
of the distribution (i.e., negative) (either "right" or "left")
A character indicating the method to be used "ML" (default),
"P", or "LNP"
A logical indicating whether the p-value of testing whether the
between-study variance is zero for methods P and LNP should be
obtained by means of a parametric bootstrap. The default value is FALSE.
An optional list of elements that give the user more control over the optimization and root-finding algorithms (see Note)
Robbie C.M. van Aert R.C.M.vanAert@tilburguniversity.edu
Three different effect size measures can be used as input for the puni_star
function: one-sample means, two-independent means, and raw correlation coefficients.
Analyzing one-sample means and two-independent means can be done by either providing
the function group means (mi or m1i and m2i), standard deviations
(sdi or sd1i and sd2i), and sample sizes (ni or
n1i and n2i) or t-values (tobs) and sample sizes (ni
or n1i and n2i). Both options should be accompanied with input
for the arguments side, method, and alpha. See the Example section for
examples. Raw correlation coefficients can be analyzed by supplying the raw
correlation coefficients ri and sample sizes and ni to the
puni_star function next to input for the arguments side,
method, and alpha. Note that the method internally transforms the
raw correlation coefficients to Fisher's z correlation coefficients. The output
of the function also shows the results for the Fisher's z correlation coefficient.
Hence, the results need to be transformed to raw correlation coefficients if
this is preferred by the user.
It is also possible to specify the standardized effect sizes and its sampling
variances directly via the yi and vi arguments. However, extensive
knowledge about computing standardized effect sizes and its sampling variances
is required and specifying standardized effect sizes and sampling variances is
not recommended to be used if the p-values in the primary studies are not computed
with a z-test. In case the p-values in the primary studies were computed with,
for instance, a t-test, the p-values of a z-test and t-test do not exactly
coincide and studies may be incorrectly included as a statistically significant or
nonsignificant effect size. Furthermore, critical values in the primary studies
are not transformed to critical z-values if yi and vi are used
as input. This yields less accurate results.
The puni_star function assumes that two-tailed hypothesis tests were conducted
in the primary studies. In case one-tailed hypothesis tests were conducted in
the primary studies, the submitted alpha argument to the puni_star
function has to be multiplied by two. For example, if one-tailed hypothesis tests were
conducted with an alpha level of .05, an alpha of 0.1 has to be submitted to
the puni_star function.
Note that only one effect size measure can be specified at a time. A combination of effect size measures usually causes true heterogeneity among effect sizes and including different effect size measures is therefore not recommended.
Selecting a method
Three different methods are currently implemented in the puni_star function.
The ML method refers to maximum likelihood estimation of the effect size
and the between-study variance. Profile likelihood confidence intervals around
the estimates are computed by means of inverting the likelihood-ratio test.
Likelihood-ratio tests are used for testing the null hypotheses of no effect
and no between-study variance. The ML method is the recommended method
for applying p-uniform*.
The two other methods (P and LNP) are moment based estimators.
The method P is based on the distribution of the sum of independent
uniformly distributed random variables (Irwin-Hall distribution) and the
LNP method refers to Fisher's method (1950, Chapter 4). For these methods,
a p-value for testing the null hypothesis of no between-study variance can also be
obtained by means of a parametric bootstrap. This is necessary since the data
is otherwise first used for estimating the effect size in the procedure for testing
the null hypothesis of no between-study variance and then also used for computing
a p-value. The test of no effect is not available for the methods P and LNP
and the publication bias test for these methods is not yet implemented.
Fisher, R.A. (1950). Statistical methods for research workers (11th ed.). London: Oliver & Boyd.
van Aert, R.C.M., & van Assen, M.A.L.M. (2023). Correcting for publication bias in a meta-analysis with the p-uniform* method. Manuscript submitted for publication. Preprint: https://osf.io/preprints/bitss/zqjr9/
### Generate data for one-sample mean with mu = 0.2 and tau^2 = 0.01
set.seed(123)
ni <- rep(50, 25)
sdi <- rep(1, 25)
ui <- rnorm(25, mean = 0.2, sd = 0.1)
mi <- rnorm(25, mean = ui, sd = sdi/sqrt(ni))
tobs <- mi/(sdi/sqrt(ni))
### Apply p-uniform* method using sample means
puni_star(mi = mi, ni = ni, sdi = sdi, alpha = 0.05, side = "right", method = "ML")
### Apply p-uniform* method using t-values
puni_star(tobs = tobs, ni = ni, alpha = 0.05, side = "right", method = "ML")
Run the code above in your browser using DataLab