tcl (version 1.0.1)

post_hocChange: Power analysis of tests in context of measurement of change using LLTM

Description

Returns post hoc power of Wald (W), likelihood ratio (LR), Rao score (RS) and gradient (GR) test given data and probability of error of first kind \(\alpha\). The hypothesis to be tested states that the shift parameter quantifying the constant change for all items between time points 1 and 2 equals 0. The alternative states that the shift parameter is not equal to 0. It is assumed that the same items are presented at both time points. See function change_test.

Usage

post_hocChange(data, alpha = 0.05)

Value

A list of results of class tcl_post_hoc.

test

A numeric vector of Wald (W), likelihood ratio (LR), Rao score (RS), and gradient (GR) test statistics.

power

Posthoc power value for each test.

dev_obs

CML estimate of shift parameter expressing observed deviation from the hypothesis to be tested.

score_dist

Relative frequencies of person scores. Uninformative scores, i.e., minimum and maximum scores, are omitted. Note that the person score distribution also influences the power of the tests.

df

Degrees of freedom \(df\).

ncp

Noncentrality parameter \(\lambda\) of \(\chi^2\) distribution from which power is determined. It equals the observed value of the test statistic.

call

The matched call.

Arguments

data

Data matrix as required for function change_test.

alpha

Probability of error of first kind.

Details

The power of the tests (Wald, LR, score, and gradient) is determined from the assumption that the approximate distributions of the four test statistics are from the family of noncentral \(\chi^2\) distributions with \(df = 1\) and noncentrality parameter \(\lambda\). In case of evaluating the post hoc power, \(\lambda\) is assumed to be given by the observed value of the test statistic. Given the probability of the error of the first kind \(\alpha\) the post hoc power of the tests can be determined from \(\lambda\). More details about the distributions of the test statistics and the relationship between \(\lambda\), power, and sample size can be found in Draxler and Alexandrowicz (2015).

In particular, let \(q_{1- \alpha}\) be the \(1- \alpha\) quantile of the central \(\chi^2\) distribution with df = 1. Then,

$$power = 1 - F_{df, \lambda} (q_{1- \alpha}),$$

where \(F_{df, \lambda}\) is the cumulative distribution function of the noncentral \(\chi^2\) distribution with \(df = 1\) and \(\lambda\) equal to the observed value of the test statistic.

References

Draxler, C., & Alexandrowicz, R. W. (2015). Sample size determination within the scope of conditional maximum likelihood estimation with special focus on testing the Rasch model. Psychometrika, 80(4), 897-919. Fischer, G. H. (1995). The Linear Logistic Test Model. In G. H. Fischer & I. W. Molenaar (Eds.), Rasch models: Foundations, Recent Developments, and Applications (pp. 131-155). New York: Springer. Fischer, G. H. (1983). Logistic Latent Trait Models with Linear Constraints. Psychometrika, 48(1), 3-26.

See Also

sa_sizeChange, and powerChange.

Examples

if (FALSE) {
# Numerical example with 200 persons and 4 items
# presented twice, thus 8 virtual items

# Data y generated under the assumption that shift parameter equals 0.5
# (change from time point 1 to 2)

# design matrix W used only for exmaple data generation
#     (not used for estimating in change_test function)
W <- rbind(c(1,0,0,0,0), c(0,1,0,0,0), c(0,0,1,0,0), c(0,0,0,1,0),
           c(1,0,0,0,1), c(0,1,0,0,1), c(0,0,1,0,1), c(0,0,0,1,1))

# eta parameter vector, first 4 are nuisance, i.e., item parameters at time point 1.
# (easiness parameters of the 4 items at time point 1),
# last one is the shift parameter
eta <- c(-2,-1,1,2,0.5)

y <- eRm::sim.rasch(persons=rnorm(150), items=colSums(-eta*t(W)))

res <- post_hocChange(data = y, alpha = 0.05)

# > res
# $test
#     W     LR     RS     GR
# 9.822 10.021  9.955 10.088
#
# $power
#     W    LR    RS    GR
# 0.880 0.886 0.884 0.888
#
# $dev_obs #`observed deviation (estimate of shift parameter)`
# [1] 0.504
#
# $score_dist #`person score distribution`
#
#     1     2     3     4     5     6     7
# 0.047 0.047 0.236 0.277 0.236 0.108 0.047
#
# $df #`degrees of freedom`
# [1] 1
#
# $ncp # `noncentrality parameter`
#     W     LR     RS     GR
# 9.822 10.021  9.955 10.088
#
# $call
# post_hocChange(alpha = 0.05, data = y)
}