This function evaluates whether the difference between two correlation coefficients can be considered statistically and practically negligible
neg.twocors(
data = NULL,
r1v1 = NULL,
r1v2 = NULL,
r2v1 = NULL,
r2v2 = NULL,
r1 = NULL,
n1 = NULL,
r2 = NULL,
n2 = NULL,
dep = FALSE,
r3 = NA,
test = "AH",
eiu,
eil,
alpha = 0.05,
bootstrap = TRUE,
nboot = 1000,
seed = NA,
plots = TRUE,
saveplots = FALSE,
...
)# S3 method for neg.twocors
print(x, ...)
A list containing the following:
r1v1 Name of the 1st variable included in the 1st correlation coefficient (r1, variable 1 ; if applicable)
r1v2 Name of the 2nd variable included in the 1st correlation coefficient (r1, variable 2; if applicable)
r2v1 Name of the 1st variable included in the 2nd correlation coefficient (r2, variable 1; if applicable)
r2v2 Name of the 2nd variable included in the 2nd correlation coefficient (r2, variable 2; if applicable)
r1 Effect size of the first bivariate relationship (1st correlation coefficient)
n1 Sample size in each variable included in the first correlation (r1)
r2 Effect size of the second bivariate relationship (2nd correlation coefficient)
n2 Sample size in each variable included in the second correlation (r2)
r3 If the correlation coefficients (r1 and r2) are dependent, r3 is then the correlation coefficient of the two, non-intersecting variables (e.g. if r1 = r12 and r2 = r13, then r3 = r23; if applicable)
rsdiff The difference between the two correlation coefficients. Specifically, r1 - r2.
se Standard error associated with the effect size point estimate (the difference between r1 and r2). The SE calculations are based on Kraatz (2007) and can be found in Counsell & Cribbie (2015)
dep Logical. TRUE if r1 and r2 are dependent, otherwise FALSE
eil Lower bound of the negligible effect (equivalence) interval
eiu Upper bound of the negligible effect (equivalence) interval
u.ci.a Upper bound of the 1-alpha CI for the effect size
l.ci.a Lower bound of the 1-alpha CI for the effect size
pd Proportional distance
pd.l.ci Lower bound of the 1-alpha CI for the PD
pd.u.ci Upper bound of the 1-alpha CI for the PD
seed Seed identity if bootstrapping is used (if applicable)
nboot Number of bootstrapping iterations, if bootstrapping was used (if applicable)
which.test Test type, e.g., AH-rho-D, KTOST-rho etc. See Counsell & Cribbie (2015) for details
degfree Degrees of freedom associated with the test statistic
pv p value associated with the test statistic. If TOST was specified, only the larger of the p values will be presented
NHSTdecision NHST decision
alpha Nominal Type I error rate
a data.frame or matrix which includes the variables in r1 and r2
the name of the 1st variable included in the 1st correlation coefficient (r1, variable 1)
the name of the 2nd variable included in the 1st correlation coefficient (r1, variable 2)
the name of the 1st variable included in the 2nd correlation coefficient (r2, variable 1)
the name of the 2nd variable included in the 2st correlation coefficient (r2, variable 2)
entered 1st correlation coefficient manually, without a dataset
entered sample size associated with r1 manually, without a dataset
entered 2nd correlation coefficient manually, without a dataset
entered sample size associated with r2 manually, without a dataset
are the correlation coefficients dependent (overlapping)?
if the correlation coefficients are dependent and no datasets were entered, specify the correlation between the two, non-intersecting variables (e.g. if r1 = r12 and r2 = r13, then r3 = r23)
'AH' is the default based on recommendation in Counsell & Cribbie (2015), 'TOST' is an additional (albeit, more conservative) option.
upper bound of the equivalence interval measured as the largest difference between the two correlations for which the two coefficients would still be considered equivalent
lower bound of the equivalence interval measured as the largest difference between the two correlations for which the two coefficients would still be considered equivalent
desired alpha level, defualt is .05
logical, default is TRUE, incorporating bootstrapping when calculating regression coefficients, SE, and CIs
1000 is the default. indicate if other number of bootstrapping iterations is desired
to reproduce previous analyses using bootstrapping, the user can set their seed of choice
logical, plotting the results. TRUE is set as default
FALSE for no, "png" and "jpeg" for different formats
extra arguments
object of class neg.twocors
Rob Cribbie cribbie@yorku.ca and Alyssa Counsell a.counsell@ryerson.ca and Udi Alter udialter@gmail.com
This function evaluates whether the difference between two correlation coefficients can be considered statistically and practically negligible according to a predefined interval. i.e., minimally meaningful effect size (MMES)/smallest effect size of interest (SESOI). The effect size tested is the difference between two correlation coefficients (i.e., r1 - r2).
Unlike the most common null hypothesis significance tests looking to detect a difference or the existence of an effect statistically different than zero, in negligible effect testing, the hypotheses are flipped: In essence, H0 states that the effect is non-negligible, whereas H1 states that the effect is in fact statistically and practically negligible.
The statistical tests are based on Anderson-Hauck (1983) and Schuirmann's (1987) Two One-Sided Test (TOST) equivalence testing procedures; namely addressing the question of whether the estimated effect size (and its associated uncertainty) of a difference between two correlation coefficients (i.e., r1 and r2) is smaller than the what the user defines as negligible effect size. Defining what is considered negligible effect is done by specifying the negligible (equivalence) interval: its upper (eiu) and lower (eil) bounds.
The negligible (equivalence) interval should be set based on the context of the research. Because the two correlations (and, therefore their difference) are in standardized units, setting eil and eiu is a matter of determining what is the smallest difference between the two correlations that can be considered of practical significance. For example, if the user determines that the smallest effect of interest is 0.1 – that is, any difference between the two correlation coefficient larger than 0.1 is meaningful in this context - then eil will be set to -0.1 and eiu to 0.1. Therefore, any observable difference that is larger than -0.1 AND smaller than 0.1, will be considered practically negligible.
There are two main approaches to using neg.twocors. The first (and more recommended) is by inserting a dataset ('data' argument) into the function. If the user/s have access to the dataset, they should use the following set of arguments: data, formula, r1v1, r1v2, r2v1, r2v2, dep (if applicable), bootstrap (optional), nboot (optional), and seed (optional). However, this function also accommodates cases where no dataset is available. In this case, users should use the following set of arguments instead: r1, n1, r2, n2, and r3 (if applicable). In either situation, users must specify the negligible interval bounds (eiu and eil). Other optional arguments and features include: alpha, test, plots, and saveplots.
This function accommodates both independent and dependent correlations. A user might want to compare two independent correlations. For example, the correlation between X and Y in one group (e.g., Control group; rXYc) with the correlation between X and Y in a different, independent group (e.g., Treatment group; rXYt). The ‘independent correlations’ setting (i.e., dep=FALSE) is the default in this function. However, in other cases, a user might want to compare two dependent correlation coefficients. That is, the two correlations share a common variable (i.e., same variable values). For example, the correlation between X and Y in one group (e.g., Treatment group; rXYt) with the correlation between X and B in the same group (e.g., Treatment group; rXBt). Because values in variable X are shared among the two correlations, the two correlations (e.g., rXYt and rXBt) are not independent from one another, but, in fact, dependent. To compare two dependent correlation coefficients, users need only to specify dep=TRUE. If no dataset is entered into the function, users should also use the argument r3, which will hold the correlation between the two non-shared variables. In the example above (i.e., rXYt and rXBt), the two non-shared variables are Y and B. In this case, r3 = rYBt. If dep=TRUE is entered into the function, test statistics and p values will be calculated differently to account for the shared variable. The negligible testing methods for comparing dependent correlations in this function are based on Williams’s (1959) modification to Hotelling’s (1931) test for comparing overlapping dependent correlations. For more details see Counsell and Cribbie (2015).
The proportional distance (PD; effect size/eiu) estimates the proportional distance of the estimated effect to eiu, and acts as an alternative effect size measure.
The confidence interval for the PD is computed via bootstrapping (percentile bootstrap), unless the user does not insert a dataset. In which case, the PD confidence interval is calculated by dividing the upper and lower CI bounds for the effect size estimate by the absolute value of the negligible interval bounds.
# Negligible difference between two correlation coefficients
# Equivalence interval: -.15 to .15
v1a<-stats::rnorm(10)
v2a<-stats::rnorm(10)
v1b <- stats::rnorm(10)
v2b <- stats::rnorm(10)
dat<-data.frame(v1a, v2a, v1b, v2b)
# dataset available (independent correlation coefficients):
neg.twocors(r1v1=v1a,r1v2=v2a,r2v1=v1b,r2v2=v2b,data=dat,eiu=.15,eil=-.15,nboot=50, dep=FALSE)
neg.twocors(r1=0.5,n1=300,r2=0.6,n2=500,eiu=.15,eil=-0.15, dep=TRUE, r3=0.51)
# end.
Run the code above in your browser using DataLab