CopyDetect (version 1.3)

similarity1: Response Similarity for Dcihotomously Scored Items

Description

Computes the response similarity indices such as the Omega index (Wollack, 1996), Generalized Binomial Test ([GBT], van der Linden & Sotaridona (2006), K index (Holland, 1996), K1 and K2 indices (Sotaridona & Meijer, 2002), and S1 and S2 indices (Sotaridona & Meijer, 2003), and the M4 index (Maynes, 2014).

Usage

similarity1(data, item.par=NULL, model="1PL", prior = TRUE, person.id, center.id=NULL, 
item.loc, single.pair=NULL, many.pairs=NULL, centers=NULL)

Arguments

data

a data frame with N rows and n columns. The data must include at least one column for unique individual IDs and item responses. All items should be scored dichotomously, with 0 indicating an incorrect response and 1 indicating a correct response. All columns must be "numeric". Missing values (NA) are allowed. Please see the details below for the treatment of missing data in the analysis.

item.par

a data matrix with n rows and three columns, where n denotes the number of items. The first, second, and third columns representitem discrimination, item difficulty, and item guessing parameters, respectively. If item parameters are not provided by user, the mirt package is internally called to estimate the parameeters of a chosen IRT model. The rows in the item parameter matrix must be in the same order as the columns in the response data.

model

IRT model to be used for computing IRT-based indices (omega, GBT, and M4). The available options are "1PL", "2PL", and "3PL". Default is "1PL".

prior

a logical argument if the model is equal 3PL. If TRUE, then a prior distribution is specified for the guessing parameter when fitting the 3PL model. Otherwise, this argument is ignored.

person.id

column label in the dataset for the variable indicating unique person ids.

center.id

column label in the dataset for the variable indicating center ids. This is used if the user asks computing the indices for all pairs in some centers. For a single pair or multiple pair calculation, this argument is ignored.

item.loc

a numeric vector indicating the location of item response in the dataset.

single.pair

a character vector of length 2 for the suspected pair of examinees. The first element of the vector indicates the unique ID of the suspected copier examinee, and the second element of the vector indicates the unique ID of the suspected source examinee.

many.pairs

a matrix with two columns. Users can request to compute the indices simultaneously as many pairs as they wish. Each row of this matrix represents a pair of examinees. The first column of each row indicates the unique ID of the suspected copier examinee, and the second column of each row indicates the unique ID of the suspected source examinee.

centers

a character vector including unique center ids. Users can request to compute the indices for all pairs in the centers listed using this argument.

Value

If a single-pair is requested, similarity1() returns a list containing the following components.

data

original data file provided by user

W.index

statistics for the W index

GBT.index

statistics for the GBT index

K.index

statistics for the K index

K.variants

statistics for the K1, K2, S1, and S2 indices

M4.index

statistics for the M4 index

item.loc

columns in the dataset that stores the item responses

item.par

estimated item parameter matrix for a chosen IRT model

center.id

column label for the unique center IDs

person.id

column label for the unique person IDs

single.pair

Unique IDs for a pair of examinee requested

single.pair2

corresponding row numbers for the pair of examinee requested

thetas

maximum likelihood estimates for IRT ability parameter for each individual

If a multiple pairs are requested in the matrix form, similarity1() returns a list containing the following components.

data

original data file provided by user

output.manypairs

a matrix including the IDs for each pair requested and corresponding indices computed for these pairs

item.loc

columns in the dataset that stores the item responses

item.par

estimated item parameter matrix for a chosen IRT model

center.id

column label for the unique center IDs

person.id

column label for the unique person IDs

thetas

maximum likelihood estimates for IRT ability parameter for each individual

If all possible pairs in certain test centers are requested in the matrix form, similarity1() returns a list containing the following components.

data

original data file provided by user

output.centers

a matrix including the IDs for all pairs requested and corresponding indices computed for these pairs

item.loc

columns in the dataset that stores the item responses

item.par

estimated item parameter matrix for a chosen IRT model

center.id

column label for the unique center IDs

person.id

column label for the unique person IDs

thetas

maximum likelihood estimates for IRT ability parameter for each individual

Details

Test fraud has recently been receiving increased attention in the field of educational testing. The current R package provides a set of useful statistical indices recently proposed in the literature for detecting a specific type of test fraud - answer copying from a nearby examinee on multiple-choice examinations. The information obtained from these procedures may provide additional statistical evidence of answer copying, but they should be used cautiously. These statistical procedures should not be used as sole evidence of answer copying, especially when used for general screening purposes.

There are more than twenty different statistical procedures recommended in the literature for detecting answer copying on multiple-choice examinations. However, the CopyDetect package includes the indices that have been shown as effective and reliable based on the simulation studies in the literature (Sotaridona & Meijer, 2002, 2003; van der Linden & Sotaridona, 2006; Wollack, 1996, 2003, 2006; Wollack & Cohen, 1998; Maynes, 2014). Among these indices, \(\mathrel\omega\), GBT, and M4 use IRT models, and \(K\) and \(K\) variants are the non-IRT counterparts.

Since similarity1 uses dichotomous responses as input, any (0,0) response combination between two response vectors is counted as an "identical incorrect response", and any (1,1) response combination between two response vectors is counted as an "identical correct response". similarity1 also counts any (NA,NA) response combination between two response vectors as an "identical incorrect response". Other response combinations such as (0,1),(1,0),(0,NA),(1,NA) between two response vectors are not counted as identical responses. When computing the number-correct/number-incorrect scores or estimating the IRT ability parameters, missing values (NA) are counted as an incorrect response.

References

Sotaridona, L.S., & Meijer, R.R.(2002). Statistical properties of the K-index for detecting answer copying. Journal of Educational Measurement, 39, 115-132.

Sotaridona, L.S., & Meijer, R.R.(2003). Two new statistics to detect answer copying. Journal of Educational Measurement, 40, 53-69.

van der Linden, W.J., & Sotaridona, L.S.(2006). Detecting answer copying when the regular response process follows a known response model. Journal of Educational and Behavioral Statistics, 31, 283-304.

Wollack, J.A.(1996). Detection of answer copying using item response theory. Dissertation Abstracts International, 57/05, 2015.

Wollack, J.A.(2003). Comparison of answer copying indices with real data. Journal of Educational Measurement, 40, 189-205.

Wollack, J.A.(2006). Simultaneous use of multiple answer copying indexes to improve detection rates. Applied Measurement in Education, 19, 265-288.

Wollack, J.A., & Cohen, A.S.(1998). Detection of answer copying with unknown item and trait parameters. Applied Psychological Measurement, 22, 144-152.

Examples

# NOT RUN {
data(form1)
dim(form1)
head(form1)

  # the first column of this dataset is unique individual IDs
  # the second column of this dataset is unique center IDs
  # From Column 3 to Column 172, dichotomous item responses


# For the sake of reducing the computational time,
# I will analyze a subset of this dataset (first 20 items)

 subset <- form1[1:1000,1:22]
 
 dim(subset)
 head(subset)


# Computing similarity for a single pair

a <- similarity1(data       = subset, 
                 model      = "1PL", 
                 person.id  = "EID", 
                 item.loc   = 3:22, 
                 single.pair= c("e100287","e100869")) 

print(a)


# }
# NOT RUN {
# Computing for multiple pairs

pairs <- matrix(as.character(sample(subset$EID,20)),nrow=10,ncol=2)

a <- similarity1(data       = subset, 
                 model      = "1PL", 
                 person.id  = "EID", 
                 item.loc   = 3:20, 
                 many.pairs = pairs)

print(a)


# Computing all possible pairs in the requested centers

a <- similarity1(data       = subset, 
                 model      = "1PL", 
                 person.id  = "EID",
                 center.id  = "cent_id",
                 item.loc   = 3:20, 
                 centers    = c(1802,5130,67,9056)) 
                 
print(a)

# }
# NOT RUN {
	
# }