ReplicationSuccess (version 1.3.3)

T1EpSceptical: Compute overall type-I error rate of the sceptical p-value

Description

The overall type-I error rate of the sceptical p-value is computed for a specified level, the relative variance, and the alternative hypothesis.

Usage

T1EpSceptical(
  level,
  c,
  alternative = c("one.sided", "two.sided"),
  type = c("golden", "nominal", "controlled")
)

Value

The overall type-I error rate.

Arguments

level

Threshold for the calibrated sceptical p-value. Default is 0.025.

c

Numeric vector of variance ratios of the original and replication effect estimates. This is usually the ratio of the sample size of the replication study to the sample size of the original study.

alternative

Specifies if level is "two.sided" or "one.sided".

type

Type of recalibration. Recalibration type can be either "golden" (default), "nominal" (no recalibration), or "controlled".

Author

Leonhard Held, Samuel Pawel

Details

T1EpSceptical is the vectorized version of the internal function .T1EpSceptical_. Vectorize is used to vectorize the function.

References

Held, L. (2020). The harmonic mean chi-squared test to substantiate scientific findings. Journal of the Royal Statistical Society: Series C (Applied Statistics), 69, 697-708. tools:::Rd_expr_doi("10.1111/rssc.12410")

Held, L., Micheloud, C., Pawel, S. (2022). The assessment of replication success based on relative effect size. The Annals of Applied Statistics. 16:706-720. tools:::Rd_expr_doi("10.1214/21-AOAS1502")

Micheloud, C., Balabdaoui, F., Held, L. (2023). Assessing replicability with the sceptical p-value: Type-I error control and sample size planning. Statistica Neerlandica. tools:::Rd_expr_doi("10.1111/stan.12312")

See Also

pSceptical, levelSceptical, PPpSceptical

Examples

## compare type-I error rate for different recalibration types
types <- c("nominal", "golden", "controlled")
c <- seq(0.2, 5, by = 0.05)
t1 <- sapply(X = types, FUN = function(t) {
  T1EpSceptical(type = t, c = c, alternative = "one.sided", level = 0.025)
})
matplot(
  x = c, y = t1*100, type = "l", lty = 1, lwd = 2, las = 1, log = "x",
  xlab = bquote(italic(c)), ylab = "Type-I error (%)",
  xlim = c(0.2, 5)
)
legend("topright", legend = types, lty = 1, lwd = 2, col = seq_along(types))