This is the master function for meta-analyses of d values - it facilitates the computation of bare-bones, artifact-distribution, and individual-correction meta-analyses of correlations for any number of group-wise contrasts and any number of dependent variables.
When artifact-distribution meta-analyses are performed, this function will automatically extract the artifact information from a database and organize it into the requested type of artifact distribution object (i.e., either Taylor series or interactive artifact distributions).
This function is also equipped with the capability to clean databases containing inconsistently recorded artifact data, impute missing artifacts (when individual-correction meta-analyses are requested), and remove dependency among samples by forming composites or averaging effect sizes and artifacts.
The automatic compositing features are employed when sample_ids and/or construct names are provided.
When multiple meta-analyses are computed within this program, the result of this function takes on the class ma_master, which means that it is a list of meta-analyses. Follow-up analyses (e.g., sensitivity, heterogeneity, meta-regression) performed on ma_master objects will analyze data from all meta-analyses recorded in the object.
ma_d(d, n1, n2 = NULL, n_adj = NULL, sample_id = NULL, citekey = NULL,
treat_as_d = TRUE, ma_method = "bb", ad_type = "tsa",
correction_method = "auto", group_id = NULL, group1 = NULL,
group2 = NULL, group_order = NULL, construct_y = NULL,
measure_y = NULL, construct_order = NULL, wt_type = "inv_var_mean",
error_type = "mean", correct_bias = TRUE, correct_rel = NULL,
correct_rGg = FALSE, correct_ryy = TRUE, correct_rr = NULL,
correct_rr_g = TRUE, correct_rr_y = TRUE, indirect_rr = NULL,
indirect_rr_g = TRUE, indirect_rr_y = TRUE, rGg = NULL, pi = NULL,
pa = NULL, ryy = NULL, ryy_restricted = TRUE, ryy_type = "alpha",
uy = NULL, uy_observed = TRUE, sign_rz = NULL, sign_rgz = 1,
sign_ryz = 1, conf_level = 0.95, cred_level = 0.8, conf_method = "t",
cred_method = "t", var_unbiased = TRUE, moderators = NULL,
cat_moderators = TRUE, moderator_type = "simple", pairwise_ads = FALSE,
residual_ads = TRUE, check_dependence = TRUE,
collapse_method = "composite", intercor = 0.5, partial_intercor = FALSE,
clean_artifacts = TRUE, impute_artifacts = ifelse(ma_method == "ad",
FALSE, TRUE), impute_method = "bootstrap_mod", seed = 42, decimals = 2,
hs_override = FALSE, use_all_arts = FALSE, estimate_pa = FALSE,
supplemental_ads = NULL, data = NULL, ...)Vector or column name of observed d values.
Vector or column name of sample sizes.
Vector or column name of sample sizes.
Optional: Vector or column name of sample sizes adjusted for sporadic artifact corrections.
Optional vector of identification labels for samples/studies in the meta-analysis.
Optional vector of bibliographic citation keys for samples/studies in the meta-analysis (if multiple citekeys pertain to a given effect size, combine them into a single string entry with comma delimiters (e.g., "citkey1,citekey2").
Logical scalar determining whether d values are to be meta-analyzed as d values (TRUE) or whether they should be meta-analyzed as correlations (FALSE).
Method to be used to compute the meta-analysis: "bb" (barebones), "ic" (individual correction), or "ad" (artifact distribution).
For when ma_method is "ad", specifies the type of artifact distribution to use: "int" or "tsa".
Character scalar or a matrix with group_id levels as row names and construct_y levels as column names.
When ma_method is "ad", select one of the following methods for correcting artifacts: "auto", "meas", "uvdrr", "uvirr", "bvdrr", "bvirr",
"rbOrig", "rb1Orig", "rb2Orig", "rbAdj", "rb1Adj", and "rb2Adj".
(note: "rb1Orig", "rb2Orig", "rb1Adj", and "rb2Adj" can only be used when Taylor series artifact distributions are provided and "rbOrig" and "rbAdj" can only
be used when interactive artifact distributions are provided). See "Details" of ma_d_ad for descriptions of the available methods.
Vector of group comparison IDs (e.g., Treatment1-Control, Treatment2-Control).
The group_id argument supersedes the group1 and group2 arguments.
If group_id is not NULL, the values supplied to the group_order argument must correspond to group_id values.
Vector of group identification labels (e.g., Treatment1, Treatment2, Control)
Optional vector indicating the order in which (1) group1 and group2 values or (2) group_ids should be arranged.
If group_order is NULL, the order of group pairings will be determined internally using alpha-numeric ordering.
Vector of construct names for construct initially designated as Y.
Vector of names for measures associated with constructs initially designated as "Y".
Vector indicating the order in which Y variables should be arranged.
Type of weight to use in the meta-analysis: options are "sample_size", "inv_var_mean" (inverse variance computed using mean effect size), and "inv_var_sample" (inverse variance computed using sample-specific effect sizes). Supported options borrowed from metafor are "DL", "HE", "HS", "SJ", "ML", "REML", "EB", and "PM" (see metafor documentation for details about the metafor methods).
Method to be used to estimate error variances: "mean" uses the mean effect size to estimate error variances and "sample" uses the sample-specific effect sizes.
Logical scalar that determines whether to correct correlations for small-sample bias (TRUE) or not (FALSE).
Optional named vector that supersedes correct_rGg and correct_ryy. Names should correspond to construct names in group_id and construct_y to determine which constructs should be corrected for unreliability.
Logical scalar or vector that determines whether to correct the grouping variable variable for measurement error (TRUE) or not (FALSE).
Logical scalar or vector that determines whether to correct the Y variable for measurement error (TRUE) or not (FALSE).
Optional named vector that supersedes correct_rr_g and correct_rr_y. Names should correspond to construct names in group_id and construct_y to determine which constructs should be corrected for range restriction.
Logical scalar or vector or column name determining whether each d value should be corrected for range restriction in the grouping variable (TRUE) or not (FALSE).
Logical scalar or vector or column name determining whether each d should be corrected for range restriction in Y (TRUE) or not (FALSE).
Optional named vector that supersedes indirect_rr_g and indirect_rr_y. Names should correspond to construct names in group_id and construct_y to determine which constructs should be corrected for indirect range restriction.
Logical vector or column name determining whether each d should be corrected for indirect range restriction in the grouping variable (TRUE) or not (FALSE).
Superseded in evaluation by correct_rr_g (i.e., if correct_rr_g == FALSE, the value supplied for indirect_rr_g is disregarded).
Logical vector or column name determining whether each d should be corrected for indirect range restriction in Y (TRUE) or not (FALSE).
Superseded in evaluation by correct_rr_y (i.e., if correct_rr_y == FALSE, the value supplied for indirect_rr_y is disregarded).
Vector or column name of reliability estimates for X.
Scalar or vector containing the restricted-group proportions of group membership. If a vector, it must either (1) have as many elements as there are d values or (2) be named so as to match with levels of the group_id argument.
Scalar or vector containing the unrestricted-group proportions of group membership (default = .5). If a vector, it must either (1) have as many elements as there are d values or (2) be named so as to match with levels of the group_id argument.
Vector or column name of reliability estimates for Y.
Logical vector or column name determining whether each element of ryy is an incumbent reliability (TRUE) or an applicant reliability (FALSE).
String vector identifying the types of reliability estimates supplied (e.g., "alpha", "retest", "interrater_r", "splithalf"). See the documentation for ma_r for a full list of acceptable reliability types.
Vector or column name of u ratios for Y.
Logical vector or column name determining whether each element of uy is an observed-score u ratio (TRUE) or a true-score u ratio (FALSE).
Optional named vector that supersedes sign_rgz and sign_ryz. Names should correspond to construct names in group_id and construct_y to determine the sign of each construct's relationship with the selection mechanism.
Sign of the relationship between X and the selection mechanism (for use with bvirr corrections only).
Sign of the relationship between Y and the selection mechanism (for use with bvirr corrections only).
Confidence level to define the width of the confidence interval (default = .95).
Credibility level to define the width of the credibility interval (default = .80).
Distribution to be used to compute the width of confidence intervals. Available options are "t" for t distribution or "norm" for normal distribution.
Distribution to be used to compute the width of credibility intervals. Available options are "t" for t distribution or "norm" for normal distribution.
Logical scalar determining whether variances should be unbiased (TRUE) or maximum-likelihood (FALSE).
Matrix or column names of moderator variables to be used in the meta-analysis (can be a vector in the case of one moderator).
Logical scalar or vector identifying whether variables in the moderators argument are categorical variables (TRUE) or continuous variables (FALSE).
Type of moderator analysis: "none" means that no moderators are to be used, "simple" means that moderators are to be examined one at a time, "hierarchical" means that all possible combinations and subsets of moderators are to be examined, and "all" means that simple and hierarchical moderator analyses are to be performed.
Logical value that determines whether to compute artifact distributions in a construct-pair-wise fashion (TRUE) or separately by construct (FALSE, default).
Logical argument that determines whether to use residualized variances (TRUE) or observed variances (FALSE) of artifact distributions to estimate sd_delta.
Logical scalar that determines whether database should be checked for violations of independence (TRUE) or not (FALSE).
Character argument that determines how to collapase dependent studies. Options are "composite" (default), "average," and "stop."
The intercorrelation(s) among variables to be combined into a composite. Can be a scalar or a named vector with element named according to the names of constructs.
Logical value that determines whether to compute artifact distributions in a construct-pair-wise fashion (TRUE) or separately by construct (FALSE, default).
If TRUE, multiple instances of the same construct (or construct-measure pair, if measure is provided) in the database are compared and reconciled with each other
in the case that any of the matching entries within a study have different artifact values. When impute_method is anything other than "stop", this method is always implemented to prevent discrepancies among imputed values.
If TRUE, artifact imputation will be performed (see impute_method for imputation procedures). Default is FALSE for artifact-distribution meta-analyses and TRUE otherwise.
When imputation is performed, clean_artifacts is treated as TRUE so as to resolve all discrepancies among artifact entries before and after imputation.
Method to use for imputing artifacts. See the documentation for ma_r for a list of available imputation methods.
Seed value to use for imputing artifacts. Default value is 42.
Number of decimal places to which results should be rounded (default is to perform no rounding).
When TRUE, this will override settings for wt_type (will set to "sample_size"), error_type (will set to "mean"),
correct_bias (will set to TRUE), conf_method (will set to "norm"), cred_method (will set to "norm"), and var_unbiased (will set to FALSE).
Logical scalar that determines whether artifact values from studies without valid effect sizes should be used in artifact distributions (TRUE) or not (FALSE).
Logical scalar that determines whether the unrestricted subgroup proportions associated with univariate-range-restricted effect sizes should be estimated by rescaling the range-restricted subgroup proportions as a function of the range-restriction correction (TRUE) or not (FALSE; default).
Named list (named according to the constructs included in the meta-analysis) of supplemental artifact distribution information from studies not included in the meta-analysis. This is a list of lists, where the elements of a list associated with a construct are named like the arguments of the create_ad() function.
Data frame containing columns whose names may be provided as arguments to vector arguments and/or moderators.
Further arguments to be passed to functions called within the meta-analysis.
A list object of the classes psychmeta, ma_d_as_r or ma_d_as_d, ma_bb (and ma_ic or ma_ad, as appropriate).
Components of output tables for bare-bones meta-analyses:
Pair_ID Unique identification number for each construct-contrast pairing.
Group_Contrast Name of the variable analyzed as the group-contrast variable.
Construct_Y Name of the variable analyzed as construct Y.
Analysis_ID Unique identification number for each moderator analysis within a construct-contrast pairing.
Analysis_Type Type of moderator analyses: Overall, Simple Moderator, or Hierarchical Moderator.
k Number of effect sizes meta-analyzed.
N Total sample size of all effect sizes in the meta-analysis.
mean_d Mean observed d value.
var_d Weighted variance of observed d values.
var_e Predicted sampling-error variance of observed d values.
var_res Variance of observed d values after removing predicted sampling-error variance.
sd_d Square root of var_r.
se_d Standard error of mean_d.
sd_e Square root of var_e.
sd_res Square root of var_res.
CI_LL_XX Lower limit of the confidence interval around mean_d, where "XX" represents the confidence level as a percentage.
CI_UL_XX Upper limit of the confidence interval around mean_d, where "XX" represents the confidence level as a percentage.
CV_LL_XX Lower limit of the credibility interval around mean_d, where "XX" represents the credibility level as a percentage.
CV_UL_XX Upper limit of the credibility interval around mean_d, where "XX" represents the credibility level as a percentage.
Components of output tables for individual-correction meta-analyses:
Pair_ID Unique identification number for each construct-contrast pairing.
Group_Contrast Name of the variable analyzed as the group-contrast variable.
Construct_Y Name of the variable analyzed as construct Y.
Analysis_ID Unique identification number for each moderator analysis within a construct-contrast pairing.
Analysis_Type Type of moderator analyses: Overall, Simple Moderator, or Hierarchical Moderator.
k Number of effect sizes meta-analyzed.
N Total sample size of all effect sizes in the meta-analysis.
mean_d Mean observed d value.
var_d Weighted variance of observed d values.
var_e Predicted sampling-error variance of observed d values.
var_res Variance of observed d values after removing predicted sampling-error variance.
sd_d Square root of var_r.
se_d Standard error of mean_d.
sd_e Square root of var_e.
sd_res Square root of var_res.
mean_delta Mean artifact-corrected d value.
var_d_c Variance of artifact-corrected d values.
var_e_c Predicted sampling-error variance of artifact-corrected d values.
var_delta Variance of artifact-corrected d values after removing predicted sampling-error variance.
sd_d_c Square root of var_r_c.
se_d_c Standard error of mean_delta.
sd_e_c Square root of var_e_c.
sd_delta Square root of var_delta.
CI_LL_XX Lower limit of the confidence interval around mean_delta, where "XX" represents the confidence level as a percentage.
CI_UL_XX Upper limit of the confidence interval around mean_delta, where "XX" represents the confidence level as a percentage.
CV_LL_XX Lower limit of the credibility interval around mean_delta, where "XX" represents the credibility level as a percentage.
CV_UL_XX Upper limit of the credibility interval around mean_delta, where "XX" represents the credibility level as a percentage.
Components of output tables for artifact-distribution meta-analyses:
Pair_ID Unique identification number for each construct-contrast pairing.
Group_Contrast Name of the variable analyzed as the group-contrast variable.
Construct_Y Name of the variable analyzed as construct Y.
Analysis_ID Unique identification number for each moderator analysis within a construct-contrast pairing.
Analysis_Type Type of moderator analyses: Overall, Simple Moderator, or Hierarchical Moderator.
k Number of effect sizes meta-analyzed.
N Total sample size of all effect sizes in the meta-analysis.
mean_d Mean observed d value.
var_d Weighted variance of observed d values.
var_e Predicted sampling-error variance of observed d values.
var_art Amount of variance in observed d values that is attributable to measurement-error and range-restriction artifacts.
var_pre Total predicted artifactual variance (i.e., the sum of var_e and var_art).
var_res Variance of observed d values after removing predicted sampling-error variance and predicted artifact variance.
sd_d Square root of var_d.
se_d Standard error of mean_d.
sd_e Square root of var_e.
sd_art Square root of var_art.
sd_pre Square root of var_pre.
sd_res Square root of var_res.
mean_delta Mean artifact-corrected d value.
var_d Weighted variance of observed d values corrected to the metric of delta.
var_e Predicted sampling-error variance of observed d values corrected to the metric of delta.
var_art Amount of variance in observed d values that is attributable to measurement-error and range-restriction artifacts corrected to the metric of delta.
var_pre Total predicted artifactual variance (i.e., the sum of var_e and var_art) corrected to the metric of delta.
var_delta Variance of artifact-corrected d values after removing predicted sampling-error variance and predicted artifact variance.
sd_d Square root of var_d corrected to the metric of delta.
se_d Standard error of mean_d corrected to the metric of delta.
sd_e Square root of var_e corrected to the metric of delta.
sd_art Square root of var_art corrected to the metric of delta.
sd_pre Square root of var_pre corrected to the metric of delta.
sd_delta Square root of var_delta.
CI_LL_XX Lower limit of the confidence interval around mean_delta, where "XX" represents the confidence level as a percentage.
CI_UL_XX Upper limit of the confidence interval around mean_delta, where "XX" represents the confidence level as a percentage.
CV_LL_XX Lower limit of the credibility interval around mean_delta, where "XX" represents the credibility level as a percentage.
CV_UL_XX Upper limit of the credibility interval around mean_delta, where "XX" represents the credibility level as a percentage.
Schmidt, F. L., & Hunter, J. E. (2015). Methods of meta-analysis: Correcting error and bias in research findings (3rd ed.). Thousand Oaks, CA: Sage. https://doi.org/10/b6mg. Chapter 3.
Dahlke, J. A., & Wiernik, B. M. (2018). One of these artifacts is not like the others: Accounting for indirect range restriction in organizational and psychological research. Manuscript submitted for review.
# NOT RUN {
## The 'ma_d' function can compute multi-construct bare-bones meta-analyses:
ma_d(d = d, n1 = n1, n2 = n2, construct_y = construct, data = data_d_meas_multi)
## It can also perform multiple individual-correction meta-analyses:
ma_d(ma_method = "ic", d = d, n1 = n1, n2 = n2, ryy = ryyi,
construct_y = construct, data = data_d_meas_multi)
## And 'ma_d' can also curate artifact distributions and compute multiple
## artifact-distribution meta-analyses:
ma_d(ma_method = "ad", d = d, n1 = n1, n2 = n2,
ryy = ryyi, correct_rr_y = FALSE,
construct_y = construct, data = data_d_meas_multi)
# }
Run the code above in your browser using DataLab