rapidsplithalf (version 0.6)

rapidsplit: rapidsplit

Description

A very fast algorithm for computing stratified permutation-based split-half reliability.

Usage

rapidsplit(
  data,
  subjvar,
  diffvars = NULL,
  stratvars = NULL,
  subscorevar = NULL,
  aggvar,
  splits = 6000,
  aggfunc = c("means", "medians"),
  errorhandling = list(type = c("none", "fixedpenalty"), errorvar = NULL, fixedpenalty =
    600, blockvar = NULL),
  standardize = FALSE,
  include.scores = TRUE,
  verbose = TRUE,
  check = TRUE
)
# S3 method for rapidsplit
print(x, ...)
# S3 method for rapidsplit
plot(
  x,
  type = c("average", "minimum", "maximum", "random", "all"),
  show.labels = TRUE,
  ...
)
rapidsplit.chunks(
  data,
  subjvar,
  diffvars = NULL,
  stratvars = NULL,
  subscorevar = NULL,
  aggvar,
  splits = 6000,
  aggfunc = c("means", "medians"),
  errorhandling = list(type = c("none", "fixedpenalty"), errorvar = NULL, fixedpenalty =
    600, blockvar = NULL),
  standardize = FALSE,
  include.scores = TRUE,
  verbose = TRUE,
  check = TRUE,
  split.chunksize = 10000,
  sample.chunksize = 200
)

Value

A list containing

• r: the averaged reliability.

• ci: the 95% confidence intervals.

• allcors: a vector with the reliability of each iteration.

• nobs: the number of participants.

• rcomponents: a list containing the mean variance of the scores of both halves, as well as their mean covariance.

• scores: the individual participants scores in each split-half, contained in a list with two matrices (Only included if requested with include.scores).

Arguments

data

Dataset, a data.frame.

subjvar

Subject ID variable name, a character.

diffvars

Names of variables that determine which conditions need to be subtracted from each other, character.

stratvars

Additional variables that the splits should be stratified by; a character.

subscorevar

A character variable identifying subgroups within a participant's data from which separate scores should be computed. To compute a participant's final score, these subscores will be averaged together. A typical use case is the D-score of the implicit association task.

aggvar

Name of variable whose values to aggregate, a character. Examples include reaction times and error rates.

splits

Number of split-halves to average, an integer. It is recommended to use around 5000.

aggfunc

The function by which to aggregate the variable defined in aggvar; can be "means", "medians", or a custom function (not a function name). This custom function must take a numeric vector and output a single value.

errorhandling

A list with 4 named items, to be used to replace error trials with the block mean of correct responses plus a fixed penalty, as in the IAT D-score. The 4 items are type which can be set to "none" for no error replacement, or "fixedpenalty" to replace error trials as described; errorvar requires name of the logical variable indicating an incorrect response (as TRUE); fixedpenalty indicates how much of a penalty should be added to said block mean; and blockvar indicates the name of the block variable.

standardize

Whether to divide by scores by the subject's SD; a logical. Regardless of whether error penalization is utilized, this standardization will be based on the unpenalized SD of correct and incorrect trials, as in the IAT D-score.

include.scores

Include all individual split-half scores?

verbose

Display progress bars? Defaults to TRUE.

check

Check input for possible problems?

x

rapidsplit object to print or plot.

...

Ignored.

type

Character argument indicating what should be plotted. By default, this plots the random split whose correlation is closest to the average. However, this can also plot the random split with the "minimum" or "maximum" split-half correlation, or any "random" split. "all" splits can also be plotted together in one figure.

show.labels

Should participant IDs be shown above their points in the scatterplot? Defaults to TRUE and is ignored when type is "all".

split.chunksize, sample.chunksize

Number of chunks to divide the splits and sample in for more memory-efficient computation. This has no bearing on the result.

Author

Sercan Kahveci

Details

The order of operations (with optional steps between brackets) is:

• Splitting

• (Replacing error trials within block within split)

• Computing aggregates per condition (per subscore) per person

• Subtracting conditions from each other

• (Dividing the resulting (sub)score by the SD of the data used to compute that (sub)score)

• (Averaging subscores together into a single score per person)

• Computing the covariances of scores from one half with scores from the other half for every split

• Computing the variances of scores within each half for every split

• Computing the average split-half correlation with the average variances and covariance across all splits, using corStatsByColumns()

• Applying the Spearman-Brown formula to the absolute correlation using spearmanBrown(), and restoring the original sign after

cormean() was used to aggregate correlations in previous versions of this package & in the associated manuscript, but the method based on (co)variance averaging was found to be more accurate. This was suggested by prof. John Christie of Dalhousie University.

References

Kahveci, S., Bathke, A.C. & Blechert, J. (2024) Reaction-time task reliability is more accurately computed with permutation-based split-half correlations than with Cronbach’s alpha. Psychonomic Bulletin and Review. tools:::Rd_expr_doi("10.3758/s13423-024-02597-y")

Examples

data(foodAAT)
# Reliability of the double difference score:
# [RT(push food)-RT(pull food)] - [RT(push object)-RT(pull object)]

frel<-rapidsplit(data=foodAAT,
                 subjvar="subjectid",
                 diffvars=c("is_pull","is_target"),
                 stratvars="stimid",
                 aggvar="RT",
                 splits=100)
                 
print(frel)

plot(frel,type="average")

           
# Compute a single random split-half reliability of the error rate
rapidsplit(data=foodAAT,
           subjvar="subjectid",
           aggvar="error",
           splits=1,
           aggfunc="means")

# Compute the reliability of an IAT D-score
data(raceIAT)
rapidsplit(data=raceIAT,
           subjvar="session_id",
           diffvars="congruent",
           subscorevar="blocktype",
           aggvar="latency",
           errorhandling=list(type="fixedpenalty",errorvar="error",
                              fixedpenalty=600,blockvar="block_number"),
           splits=10,
           standardize=TRUE)


# Compute the reliability of mean RT
# in subsets of 200 splits and 100 participants per run
rapidsplit.chunks(data=foodAAT,
                  subjvar="subjectid",
                  aggvar="RT",
                  splits=400,
                  split.chunksize=200,
                  sample.chunksize=50)