aberrance (version 0.2.1)

detect_nm: Detect nonparametric misfit

Description

Detect nonparametric misfit using person-fit statistics.

Usage

detect_nm(method, x = NULL, y = NULL)

Value

A list is returned with the following elements:

stat

A matrix of nonparametric person-fit statistics.

Arguments

method

The person-fit statistic(s) to compute. Options for score-based statistics are:

• "G_S" for the number of Guttman errors (Guttman, 1944; see also Molenaar, 1991).

• "NC_S" for the norm conformity index (Tatsuoka & Tatsuoka, 1983). Note: This statistic cannot be computed for polytomous item scores.

• "U1_S" for the \(U1\) statistic, also known as the \(G^*\) statistic (van der Flier, 1977; see also Emons, 2008).

• "U3_S" for the \(U3\) statistic (van der Flier, 1982; see also Emons, 2008).

• "ZU3_S" for the \(ZU3\) statistic (van der Flier, 1982). Note: This statistic cannot be computed for polytomous item scores.

• "A_S" for the agreement index (Kane & Brennan, 1980). Note: This statistic cannot be computed for polytomous item scores.

• "D_S" for the disagreement index (Kane & Brennan, 1980). Note: This statistic cannot be computed for polytomous item scores.

• "E_S" for the dependability index (Kane & Brennan, 1980). Note: This statistic cannot be computed for polytomous item scores.

• "C_S" for the caution index (Sato, 1975). Note: This statistic cannot be computed for polytomous item scores.

• "MC_S" for the modified caution index, also known as the \(C^*\) statistic (Harnisch & Linn, 1981). Note: This statistic cannot be computed for polytomous item scores.

• "PC_S" for the personal point-biserial correlation (Donlon & Fischer, 1968). Note: This statistic cannot be computed for polytomous item scores.

• "HT_S for the \(H^T\) statistic (Sijtsma, 1986). Note: This statistic cannot be computed for polytomous item scores.

Options for response time-based statistics are:

• "KL_T" for the Kullback-Leibler divergence (Man et al., 2018).

x, y

Matrices of raw data. x is for the item scores and y the item log response times.

References

Donlon, T. F., & Fischer, F. E. (1968). An index of an individual's agreement with group-determined item difficulties. Educational and Psychological Measurement, 28(1), 105--113.

Emons, W. H. M. (2008). Nonparametric person-fit analysis of polytomous item scores. Applied Psychological Measurement, 32(3), 224--247.

Guttman, L. (1944). A basis for scaling qualitative data. American Sociological Review, 9(2), 139--150.

Harnisch, D. L., & Linn, R. L. (1981). Analysis of item response patterns: Questionable test data and dissimilar curriculum practices. Journal of Educational Measurement, 18(3), 133--146.

Kane, M. T., & Brennan, R. L. (1980). Agreement coefficients as indices of dependability for domain referenced tests. Applied Psychological Measurement, 4(1), 105--126.

Man, K., Harring, J. R., Ouyang, Y., & Thomas, S. L. (2018). Response time based nonparametric Kullback-Leibler divergence measure for detecting aberrant test-taking behavior. International Journal of Testing, 18(2), 155--177.

Molenaar, I. W. (1991). A weighted Loevinger H-coefficient extending Mokken scaling to multicategory items. Kwantitatieve Methoden, 12(37), 97--117.

Sato, T. (1975). The construction and interpretation of S-P tables.

Sijtsma, K. (1986). A coefficient of deviance of response patterns. Kwantitatieve Methoden, 7(22), 131--145.

Tatsuoka, K. K., & Tatsuoka, M. M. (1983). Spotting erroneous rules of operation by the individual consistency index. Journal of Educational Measurement, 20(3), 221--230.

van der Flier, H. (1977) Environmental factors and deviant response patterns. In Y. H. Poortinga (Ed.), Basic problems in cross-cultural psychology. Swets & Zeitlinger Publishers.

van der Flier, H. (1982). Deviant response patterns and comparability of test scores. Journal of Cross-Cultural Psychology, 13(3), 267--298.

See Also

detect_pm() to detect parametric misfit.

Examples

# Setup for Examples 1 to 3 -------------------------------------------------

# Settings
set.seed(0)     # seed for reproducibility
N <- 500        # number of persons
n <- 40         # number of items

# Randomly select 10% examinees with preknowledge and 40% compromised items
cv <- sample(1:N, size = N * 0.10)
ci <- sample(1:n, size = n * 0.40)

# Create vector of indicators (1 = misfitting, 0 = fitting)
ind <- ifelse(1:N %in% cv, 1, 0)

# Example 1: Dichotomous Item Scores ----------------------------------------

# Generate person parameters for the 3PL model
xi <- cbind(theta = rnorm(N, mean = 0.00, sd = 1.00))

# Generate item parameters for the 3PL model
psi <- cbind(
  a = rlnorm(n, meanlog = 0.00, sdlog = 0.25),
  b = rnorm(n, mean = 0.00, sd = 1.00),
  c = runif(n, min = 0.05, max = 0.30)
)

# Simulate uncontaminated data
x <- sim(psi, xi)$x

# Modify contaminated data by changing the item scores
x[cv, ci] <- rbinom(length(cv) * length(ci), size = 1, prob = 0.90)

# Detect nonparametric misfit
out <- detect_nm(
  method = c("G_S", "NC_S", "U1_S", "U3_S", "ZU3_S", "A_S", "D_S", "E_S",
             "C_S", "MC_S", "PC_S", "HT_S"),
  x = x
)

# Example 2: Polytomous Item Scores -----------------------------------------

# Generate person parameters for the generalized partial credit model
xi <- cbind(theta = rnorm(N, mean = 0.00, sd = 1.00))

# Generate item parameters for the generalized partial credit model
psi <- cbind(
  a = rlnorm(n, meanlog = 0.00, sdlog = 0.25),
  c0 = 0,
  c1 = rnorm(n, mean = -1.00, sd = 0.50),
  c2 = rnorm(n, mean = 0.00, sd = 0.50),
  c3 = rnorm(n, mean = 1.00, sd = 0.50)
)

# Simulate uncontaminated data
x <- sim(psi, xi)$x

# Modify contaminated data by changing the item scores to the maximum score
x[cv, ci] <- 3

# Detect nonparametric misfit
out <- detect_nm(
  method = c("G_S", "U1_S", "U3_S"),
  x = x
)

# Example 3: Item Response Times --------------------------------------------

# Generate person parameters for the lognormal model
xi <- cbind(tau = rnorm(N, mean = 0.00, sd = sqrt(0.25)))

# Generate item parameters for the lognormal model
psi <- cbind(
  alpha = runif(n, min = 1.50, max = 2.50),
  beta = rnorm(n, mean = 3.50, sd = sqrt(0.15))
)

# Simulate uncontaminated data
y <- sim(psi, xi)$y

# Modify contaminated data by reducing the log response times
y[cv, ci] <- y[cv, ci] * 0.75

# Detect nonparametric misfit
out <- detect_nm(
  method = "KL_T",
  y = y
)