`sca_minp()` adds, to an existing [sca_test()] result, a multiple-comparison- corrected p-value for *every* specification in the curve. It answers the question the per-specification null band only shows descriptively: *which* individual specifications are more extreme than chance once you account for having searched all of them?
It reuses the permutation null [sca_test()] already computed (so it never re-runs the permutations) and requires that result to have been produced with `keep_curves = TRUE` and a confound-preserving null (`null_type = "freedman_lane"` or `"residual_bootstrap"`). When those prerequisites are met, [sca_test()] already attaches the single-step adjustment automatically; call `sca_minp()` to recompute it with a different method or significance threshold.
sca_minp(result, method = c("single_step", "step_down"), alpha = NULL)The input `result`, augmented with a `fwer` element under `null_curves`: a list with `specs` (a data frame with one row per specification giving its term-set key `spec`, `observed` coefficient, uncorrected own-null p-value `p_raw`, FWER-adjusted p-value `p_adj`, and logical `significant_adj`) and `summary` (the method, alpha, whether weak or strong FWER is controlled, the number of distinct specifications tested, the number significant after correction, and the smallest adjusted p-value).
An object of class `"sca_test"` returned by [sca_test()] with `keep_curves = TRUE` and a confound-preserving `null_type`.
One of `"single_step"` (default) or `"step_down"`; see Details.
The significance threshold for `significant_adj`. Defaults to `NULL`, which inherits the `alpha` used by [sca_test()].
Reporting a specification as "significant" because its ordinary p-value is below 0.05 is misleading when the curve contains many correlated specifications: searching dozens of them all but guarantees some will look significant by chance. `sca_minp()` corrects for this with the min-P / max-statistic permutation method of Westfall and Young (1993). For each permutation it records the most extreme specification anywhere in the curve, building the null distribution of "the best result a search could turn up by chance", and compares each observed specification to that distribution.
Each specification is compared to its *own* permutation null rather than to zero. This matters under the confound-preserving nulls: an under-controlled specification's null distribution is centred on its confounded value, not on zero, so a specification is flagged only when its estimate is more extreme than that specification could itself produce under the null. Consequently a flagged under-controlled specification indicates an association beyond the superset-conditional sharp null -- it is **not** a causal effect of size (estimate minus null centre), because omitting a confounder re-routes part of the focal variable's coefficient.
`method = "single_step"` (the default, and what [sca_test()] attaches automatically) controls the family-wise error rate in the weak sense, under the global null of no effect in any specification, with no further assumptions. `method = "step_down"` applies Westfall and Young's free step-down refinement, which is more powerful and controls the family-wise error rate in the strong sense under subset pivotality (exact for `"freedman_lane"` via the common control-superset conditioning; approximate for `"residual_bootstrap"` and under partial alternatives). Adjusted p-values have a resolution floor of `1 / (n_eff + 1)`, where `n_eff` is the number of permutations that estimated at least one specification (equal to `n_used` unless some permutations were wholly non-estimable).
Westfall, P. H., & Young, S. S. (1993). Resampling-based multiple testing: Examples and methods for p-value adjustment. Wiley.
[sca_test()], which attaches the single-step adjustment automatically; [plot_sca_test_specs()] to see which specifications survive correction.
# \donttest{
result <- sca_test(y = "Salnty", x = "T_degC",
controls = c("ChlorA", "O2Sat", "NO2uM"), data = bottles,
null_type = "freedman_lane", n_permutations = 199,
keep_curves = TRUE, seed = 1, progress_bar = FALSE)
# The single-step adjustment is already attached; recompute it step-down.
result <- sca_minp(result, method = "step_down")
as.data.frame(result, what = "specs")
# }
Run the code above in your browser using DataLab