difNLR: DIF detection using non-linear regression method.

Description

Performs DIF detection procedure for dichotomous data based on non-linear regression model (generalized logistic regression) and either likelihood-ratio test, F-test, or Wald's test of a submodel.

Usage

difNLR(
  Data, group, focal.name, model, constraints, type = "all", method = "nls",
  match = "zscore", anchor = NULL, purify = FALSE, nrIter = 10, test = "LR",
  alpha = 0.05, p.adjust.method = "none", start, initboot = TRUE, nrBo = 20,
  sandwich = FALSE
)

Arguments

Data

data.frame or matrix: dataset which rows represent scored examinee answers ("1" correct, "0" incorrect) and columns correspond to the items. In addition, Data can hold the vector of group membership.

group

numeric or character: a dichotomous vector of the same length as nrow(Data) or a column identifier of Data.

focal.name

numeric or character: indicates the level of group which corresponds to focal group.

model

character: generalized logistic regression model to be fitted. See Details.

constraints

character: which parameters should be the same for both groups. Possible values are any combinations of parameters "a", "b", "c", and "d". See Details.

type

character: type of DIF to be tested. Possible values are "all" for detecting difference in any parameter (default), "udif" for uniform DIF only (i.e., difference in difficulty parameter "b"), "nudif" for non-uniform DIF only (i.e., difference in discrimination parameter "a"), "both" for uniform and non-uniform DIF (i.e., difference in parameters "a" and "b"), or combination of parameters "a", "b", "c", and "d". Can be specified as a single value (for all items) or as an item-specific vector.

method

character: method used to estimate parameters. Either "nls" for non-linear least squares (default), or "likelihood" for maximum likelihood method.

match

numeric or character: matching criterion to be used as an estimate of trait. Can be either "zscore" (default, standardized total score), "score" (total test score), or vector of the same length as number of observations in Data.

anchor

numeric or character: specification of DIF free items. Either NULL (default), or a vector of item names (column names of Data), or item identifiers (integers specifying the column number) determining which items are currently considered as anchor (DIF free) items. Argument is ignored if match is not "zscore" or "score".

purify

logical: should the item purification be applied? (default is FALSE).

nrIter

numeric: the maximal number of iterations in the item purification (default is 10).

test

character: test to be performed for DIF detection. Can be either "LR" for likelihood ratio test of a submodel (default), "W" for Wald test, or "F" for F-test of a submodel.

alpha

numeric: significance level (default is 0.05).

p.adjust.method

character: method for multiple comparison correction. Possible values are "holm", "hochberg", "hommel", "bonferroni", "BH", "BY", "fdr", and "none" (default). For more details see p.adjust.

start

numeric: initial values for estimation of parameters. If not specified, starting values are calculated with startNLR function. Otherwise, list with as many elements as a number of items. Each element is a named numeric vector of length 8 representing initial values for parameter estimation. Specifically, parameters "a", "b", "c", and "d" are initial values for discrimination, difficulty, guessing, and inattention for reference group. Parameters "aDif", "bDif", "cDif", and "dDif" are then differences in these parameters between reference and focal group.

initboot

logical: in case of convergence issues, should be starting values re-calculated based on bootstrapped samples? (default is TRUE; newly calculated initial values are applied only to items/models with convergence issues).

nrBo

numeric: the maximal number of iterations for calculation of starting values using bootstrapped samples (default is 20).

sandwich

logical: should be sandwich estimator used for covariance matrix of parameters when using method = "nls"? Default is FALSE.

Value

The difNLR() function returns an object of class "difNLR". The output including values of the test statistics, p-values, and items detected as function differently is displayed by the print() method.

Object of class "difNLR" is a list with the following components:

Sval: the values of the test statistics.
nlrPAR: the estimates of final model.
nlrSE: the standard errors of estimates of final model.
parM0: the estimates of null model.
seM0: the standard errors of estimates of null model.
covM0: the covariance matrices of estimates of null model.
llM0: log-likelihood of null model.
parM1: the estimates of alternative model.
seM1: the standard errors of estimates of alternative model.
covM1: the covariance matrices of estimates of alternative model.
llM1: log-likelihood of alternative model.
DIFitems: either the column identifiers of the items which were detected as DIF, or "No DIF item detected" in case no item was detected as function differently.
model: fitted model.
constraints: constraints for the model.
type: character: type of DIF that was tested. If parameters were specified, the value is "other".
types: character: the parameters (specified by user, type has value "other") which were tested for difference.
p.adjust.method: character: method for multiple comparison correction which was applied.
pval: the p-values of the test.
adj.pval: the adjusted p-values of the test using p.adjust.method.
df: the degress of freedom of the test.
test: used test.
purification: purify value.
nrPur: number of iterations in item purification process. Returned only if purify is TRUE.
difPur: a binary matrix with one row per iteration of item purification and one column per item. "1" in i-th row and j-th column means that j-th item was identified as DIF in i-th iteration. Returned only if purify is TRUE.
conv.puri: logical: indicating whether item purification process converged before the maximal number nrIter of iterations. Returned only if purify is TRUE.
method: used estimation method.
conv.fail: numeric: number of convergence issues.
conv.fail.which: the identifiers of the items which did not converge.
alpha: numeric: significance level.
Data: the data matrix.
group: the vector of group membership.
group.names: names of groups.
match: matching criterion.

For an object of class "difNLR" several methods are available (e.g., methods(class = "difNLR")).

Details

DIF detection procedure based on non-linear regression is the extension of the logistic regression procedure (Swaminathan & Rogers, 1990) accounting for possible guessing and/or inattention when responding (Drabinova & Martinkova, 2017; Hladka & Martinkova, 2020).

The unconstrained form of 4PL generalized logistic regression model for probability of correct answer (i.e., $y = 1$) is $$P(y = 1) = (c + cDif * g) + (d + dDif * g - c - cDif * g) / (1 + exp(-(a + aDif * g) * (x - b - bDif * g))), $$ where $x$ is by default standardized total score (also called Z-score) and $g$ is a group membership. Parameters $a$, $b$, $c$, and $d$ are discrimination, difficulty, guessing, and inattention. Terms $aDif$, $bDif$, $cDif$, and $dDif$ then represent differences between two groups (reference and focal) in relevant parameters.

This 4PL model can be further constrained by model and constraints arguments. The arguments model and constraints can be also combined. Both arguments can be specified as a single value (for all items) or as an item-specific vector (where each element correspond to one item).

The model argument offers several predefined models. The options are as follows: Rasch for 1PL model with discrimination parameter fixed on value 1 for both groups, 1PL for 1PL model with discrimination parameter fixed for both groups, 2PL for logistic regression model, 3PLcg for 3PL model with fixed guessing for both groups, 3PLdg for 3PL model with fixed inattention for both groups, 3PLc (alternatively also 3PL) for 3PL regression model with guessing parameter, 3PLd for 3PL model with inattention parameter, 4PLcgdg for 4PL model with fixed guessing and inattention parameter for both groups, 4PLcgd (alternatively also 4PLd) for 4PL model with fixed guessing for both groups, 4PLcdg (alternatively also 4PLc) for 4PL model with fixed inattention for both groups, or 4PL for 4PL model.

The model can be specified in more detail with constraints argument which specifies what parameters should be fixed for both groups. For example, choice "ad" means that discrimination (parameter "a") and inattention (parameter "d") are fixed for both groups and other parameters ("b" and "c") are not. The NA value for constraints means no constraints.

Missing values are allowed but discarded for item estimation. They must be coded as NA for both, Data and group arguments.

In case that model considers difference in guessing or inattention parameter, the different parameterization is used and parameters with standard errors are re-calculated by delta method. However, covariance matrices stick with alternative parameterization.

References

Drabinova, A. & Martinkova, P. (2017). Detection of differential item functioning with nonlinear regression: A non-IRT approach accounting for guessing. Journal of Educational Measurement, 54(4), 498--517, 10.1111/jedm.12158.

Hladka, A. & Martinkova, P. (2020). difNLR: Generalized logistic regression models for DIF and DDF detection. The R journal, 12(1), 300--323, 10.32614/RJ-2020-014.

Swaminathan, H. & Rogers, H. J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27(4), 361--370, 10.1111/j.1745-3984.1990.tb00754.x

Examples

Run this code

# NOT RUN {
# Loading data based on GMAT
data(GMAT)

Data <- GMAT[, 1:20]
group <- GMAT[, "group"]

# Testing both DIF effects using likelihood-ratio test and
# 3PL model with fixed guessing for groups
(x <- difNLR(Data, group, focal.name = 1, model = "3PLcg"))
# }
# NOT RUN {
# Graphical devices
plot(x, item = x$DIFitems)
plot(x, item = "Item1")
plot(x, item = 1, group.names = c("Group 1", "Group 2"))
plot(x, plot.type = "stat")

# Coefficients
coef(x)
coef(x, SE = TRUE)
coef(x, SE = TRUE, simplify = TRUE)

# Fitted values
fitted(x)
fitted(x, item = 1)

# Residuals
residuals(x)
residuals(x, item = 1)

# Predicted values
predict(x)
predict(x, item = 1)

# Predicted values for new subjects
predict(x, item = 1, match = 0, group = 0)
predict(x, item = 1, match = 0, group = 1)

# AIC, BIC, log-likelihood
AIC(x)
BIC(x)
logLik(x)

# AIC, BIC, log-likelihood for the first item
AIC(x, item = 1)
BIC(x, item = 1)
logLik(x, item = 1)

# Testing both DIF effects using Wald test and
# 3PL model with fixed guessing for groups
difNLR(Data, group, focal.name = 1, model = "3PLcg", test = "W")
#' # Testing both DIF effects using F test and
# 3PL model with fixed guessing for groups
difNLR(Data, group, focal.name = 1, model = "3PLcg", test = "F")

# Testing both DIF effects using
# 3PL model with fixed guessing for groups and sandwich estimator
# of the covariance matrices
difNLR(Data, group, focal.name = 1, model = "3PLcg", sandwich = TRUE)

# Testing both DIF effects using LR test,
# 3PL model with fixed guessing for groups
# and Benjamini-Hochberg correction
difNLR(Data, group, focal.name = 1, model = "3PLcg", p.adjust.method = "BH")

# Testing both DIF effects using LR test,
# 3PL model with fixed guessing for groups
# and item purification
difNLR(Data, group, focal.name = 1, model = "3PLcg", purify = TRUE)

# Testing both DIF effects using 3PL model with fixed guessing for groups
# and total score as matching criterion
difNLR(Data, group, focal.name = 1, model = "3PLcg", match = "score")

# Testing uniform DIF effects using 4PL model with the same
# guessing and inattention
difNLR(Data, group, focal.name = 1, model = "4PLcgdg", type = "udif")

# Testing non-uniform DIF effects using 2PL model
difNLR(Data, group, focal.name = 1, model = "2PL", type = "nudif")

# Testing difference in parameter b using 4PL model with fixed
# a and c parameters
difNLR(Data, group, focal.name = 1, model = "4PL", constraints = "ac", type = "b")

# Testing both DIF effects using LR test,
# 3PL model with fixed guessing for groups
# with maximum likelihood estimation method
difNLR(Data, group, focal.name = 1, model = "3PLcg", method = "likelihood")
# }
# NOT RUN {
# }

Run the code above in your browser using DataLab