Data engineering and BI courses are free!
Computes the correlation of two variables on an edsurvey.data.frame,
a light.edsurvey.data.frame, or an edsurvey.data.frame.list.
The correlation accounts for plausible values and the survey design.
cor.sdf(
x,
y,
data,
method = c("Pearson", "Spearman", "Polychoric", "Polyserial"),
weightVar = "default",
reorder = NULL,
omittedLevels = TRUE,
defaultConditions = TRUE,
recode = NULL,
condenseLevels = TRUE,
fisherZ = if (match.arg(method) %in% "Pearson") {
TRUE
} else {
FALSE
},
jrrIMax = Inf,
verbose = TRUE
)An edsurvey.cor that has print and summary methods.
The class includes the following elements:
numeric estimated correlation coefficient
standard error of the correlation (Vimp + Vjrr). In the case of Pearson, this is calculated in the linear atanh space and is not a standard error in the usual sense.
a vector of length two showing the columns for which the correlation coefficient was calculated
correlates that are discrete
a list that shows the order of each variable
the type of correlation estimated
the jackknife component of the variance estimate. For Pearson, in the atanh space.
the imputation component of the variance estimate. For Pearson, in the atanh space.
the weight variable used
the number of plausible values used
the number of the jackknife replicates used
the original number of observations
the number of observations used in the analysis---after any conditions and any listwise deletion of missings is applied
the standard error of the correlation, in the correlation ([-1,1]) space
the confidence interval of the correlation in the transformation space
the confidence interval of the correlation in the correlation ([-1,1]) space
the name of the transformation used when calculating standard errors
a character variable name from the data to be correlated with y
a character variable name from the data to be correlated with x
an edsurvey.data.frame, a light.edsurvey.data.frame, or an edsurvey.data.frame.list
a character string indicating which correlation coefficient (or covariance) is to be computed.
One of Pearson (default), Spearman, Polychoric, or Polyserial. For Polyserial, the continuous argument must be x.
character indicating the weight variable to use. See Details section in lm.sdf.
a list of variables to reorder. Defaults to NULL (no variables are reordered). Can be set as
reorder = list(var1 = c("a","b","c"), var2 = c("4", "3", "2", "1")). See Examples.
a logical value. When set to the default value of TRUE, drops those levels of all factor variables that are specified
in an edsurvey.data.frame. Use print on an edsurvey.data.frame to see the omitted levels.
a logical value. When set to the default value of TRUE, uses the default conditions stored in an edsurvey.data.frame
to subset the data. Use print on an edsurvey.data.frame to see the default conditions.
a list of lists to recode variables. Defaults to NULL. Can be set as
recode = list(var1 = list(from = c("a","b","c"), to = "d")). See Examples.
a logical value. When set to the default value of
TRUE and either x or y is a
categorical variable, the function will drop all unused
levels and rank the levels of the variable before
calculating the correlation. When set to FALSE,
the numeric levels of the variable remain the same as
in the codebook. See Examples.
for standard error and mean calculations, set to TRUE to use
the Fisher Z-transformation (see details), or FALSE
to use no transformation of the data. The fisherZ argument defaults
to Fisher Z-transformation for Pearson and no transformation
for other correlation types.
a numeric value; when using the jackknife variance estimation method, the default estimation option, jrrIMax=Inf, uses the
sampling variance from all plausible values as the component for sampling variance estimation. The Vjrr
term (see
Statistical Methods Used in EdSurvey)
can be estimated with any number of plausible values, and values larger than the number of
plausible values on the survey (including Inf) will result in all plausible values being used.
Higher values of jrrIMax lead to longer computing times and more accurate variance estimates.
a logical value. Set to FALSE to avoid messages about variable conversion.
Paul Bailey; relies heavily on the wCorr package, written by Ahmad Emad and Paul Bailey
The getData arguments and recode.sdf may be useful. (See Examples.)
The correlation methods are calculated as described in the documentation for the wCorr package---see browseVignettes(package="wCorr").
When method is set to polyserial, all x arguments are assumed to be continuous and all y assumed discrete. Therefore,
be mindful of variable selection as this may result in calculations taking a very long time to complete.
The Fisher Z-transformation is both a variance stabilizing and normalizing transformation for the Pearson correlation coefficient (Fisher, 1915). The transformation takes the inverse hybarbolic tangent of the correlation coefficients and then calculates all variances and confidence intervals. These are then transformed back to the correlation space (values between -1 and 1, inclusive) using the hyperbolic tangent function. The Taylor series approximation (or delta method) is applied for the standard errors.
Fisher, R. A. (1915). Frequency distribution of the values of the correlation coefficient in samples from an indefinitely large population. Biometrika, 10(4), 507--521.
cor and weightedCorr
if (FALSE) {
# read in the example data (generated, not real student data)
sdf <- readNAEP(system.file("extdata/data", "M36NT2PM.dat", package = "NAEPprimer"))
# for two categorical variables any of the following work
c1_pears <- cor.sdf(x="b017451", y="b003501", data=sdf, method="Pearson",
weightVar="origwt")
c1_spear <- cor.sdf(x="b017451", y="b003501", data=sdf, method="Spearman",
weightVar="origwt")
c1_polyc <- cor.sdf(x="b017451", y="b003501", data=sdf, method="Polychoric",
weightVar="origwt")
c1_pears
c1_spear
c1_polyc
# for categorical variables, users can either keep the original numeric levels of the variables
# or condense the levels (default)
# the following call condenses the levels of the variable 'c046501'
cor.sdf(x="c046501", y="c044006", data=sdf)
# the following call keeps the original levels of the variable 'c046501'
cor.sdf(x="c046501", y="c044006", data=sdf, condenseLevels = FALSE)
# these take awhile to calculate for large datasets, so limit to a subset
sdf_dnf <- subset(sdf, b003601 == 1)
# for a categorical variable and a scale score any of the following work
c2_pears <- cor.sdf(x="composite", y="b017451", data=sdf_dnf, method="Pearson",
weightVar="origwt")
c2_spear <- cor.sdf(x="composite", y="b017451", data=sdf_dnf, method="Spearman",
weightVar="origwt")
c2_polys <- cor.sdf(x="composite", y="b017451", data=sdf_dnf, method="Polyserial",
weightVar="origwt")
c2_pears
c2_spear
c2_polys
# recode two variables
cor.sdf(x="c046501", y="c044006", data=sdf, method="Spearman", weightVar="origwt",
recode=list(c046501=list(from="0%",to="None"),
c046501=list(from=c("1-5%", "6-10%", "11-25%", "26-50%",
"51-75%", "76-90%", "Over 90%"),
to="Between 0% and 100%"),
c044006=list(from=c("1-5%", "6-10%", "11-25%", "26-50%",
"51-75%", "76-90%", "Over 90%"),
to="Between 0% and 100%")))
# reorder two variables
cor.sdf(x="b017451", y="sdracem", data=sdf, method="Spearman", weightVar="origwt",
reorder=list(sdracem=c("White", "Hispanic", "Black", "Asian/Pacific Island",
"Amer Ind/Alaska Natv", "Other"),
b017451=c("Every day", "2 or 3 times a week", "About once a week",
"Once every few weeks", "Never or hardly ever")))
# recode two variables and reorder
cor.sdf(x="pared", y="b013801", data=subset(sdf, !pared %in% "I Don\'t Know"),
method="Spearman", weightVar = "origwt",
recode=list(pared=list(from="Some ed after H.S.", to="Graduated H.S."),
pared=list(from="Graduated college", to="Graduated H.S."),
b013801=list(from="0-10", to="Less than 100"),
b013801=list(from="11-25", to="Less than 100"),
b013801=list(from="26-100", to="Less than 100")),
reorder=list(b013801=c("Less than 100", ">100")))
}
Run the code above in your browser using DataLab