This function uses a bootstrap approach to calculate the replication probability of significance, which answers the question "if we repeat this linear regression under identical conditions (similar sample size, similar residual variance), what is the probability of observing significance (or non-significance) similar to the original data?".
rpLM(model, alpha = 0.05, R = 10000, plot = TRUE, verbose = TRUE, ...)A vector with the three different bootstrap results as described above.
a linear model of class lm.
the \(\alpha\)-level to use as the threshold border.
the number of bootstrap resamples, see bootLM.
logical. If TRUE, a stripchart of the bootstrap P-values, the original P-value and the \(\alpha\)-level is displayed.
logical. If TRUE, the analysis steps are written to the console.
other parameters to be supplied to bootLM.
Andrej-Nikolai Spiess
The approach here is along the lines of Boos & Stefanski (2011), which investigated the replication probability of the P-value, as opposed to the works of Goodman (1992), Shao & Chow (2002) and Miller (2009), where the effect size is used. In our context, for a given linear model and using a bootstrap approach, the replication probability is the proportion of bootstrap P-values with \(\tilde{P} \leq \alpha\) when the original model is significant, or \(\tilde{P} > \alpha\) when not. Hence, we employ the bootstrap to assess the sampling variability of the P-value, not the sampling variability of the P-value under \(H_0\), as is common, thereby preserving the non-null property of the data.
Bootstrap results are obtained from non-parametric cases bootstrapping ("np.cases"), non-parametric residuals bootstrapping ("np.resid") and parametric residuals bootstrapping ("p.resid"), see bootLM.
Ecological Models and Data in R.
Chapter 5: Stochastic simulation and power analysis.
Benjamin M. Bolker.
Princeton University Press (2008).
P-Value Precision and Reproducibility.
Boos DD & Stefanski LA.
Am Stat, 65, 2011, 213-212.
A comment on replication, p-values and evidence.
Goodman SN.
Stat Med, 11, 1992, 875-879.
Reproducibility probability in clinical trials.
Shao J & Chow SC.
Stat Med, 21, 2002, 1727-1742.
What is the probability of replicating a statistically significant effect?
Miller J.
Psych Bull & Review, 16, 2009, 617-640.
set.seed(125)
a <- 1:20
b <- 5 + 0.08 * a + rnorm(length(a), 0, 1)
LM1 <- lm(b ~ a)
summary(LM1)
rpLM(LM1, R = 100)
Run the code above in your browser using DataLab