PTE_bootstrap_inference's m_pow_of_n argument sets each bootstrap
replicate's resample size to \(m = \lfloor n^{\gamma} \rfloor\), which is
required to be less than \(n\) (i.e. \(\gamma < 1\)) for the bootstrap to be
consistent for our non-smooth policy-value estimands (see PTE_bootstrap_inference).
There is, however, no single population-optimal \(\gamma\) -- asymptotic theory only
requires \(m \to \infty\) and \(m / n \to 0\). Politis, Romano &
Wolf (Subsampling, 1999, Ch. 9) and Bickel & Sakov (2008) address this by proposing the
"minimum volatility" (MV) method: compute the bootstrap answer (here, the width of the
rate-corrected percentile confidence interval for one of the three PTE estimands) over a grid of
candidate \(\gamma\) (equivalently \(m\)) values, then, for each grid point, measure
how much that answer varies over a small centered neighborhood of adjacent grid points ("volatility").
The chosen \(\gamma\) is the one whose neighborhood is most stable, i.e. the region of the
grid where the bootstrap's answer has stopped changing in response to further tuning of \(m\) --
this is the practical, data-driven proxy for "\(m\) large enough for the CLT-type approximation
underlying the bootstrap to have kicked in, but still small enough relative to \(n\) for
consistency."
select_optimal_m_prop(
X,
y,
regression_type = "continuous",
incidence_metric = "odds_ratio",
personalized_model_build_function = NULL,
censored = NULL,
predict_function = NULL,
difference_function = NULL,
cleanup_mod_function = NULL,
y_higher_is_better = TRUE,
pct_leave_out = 0.1,
m_pow_of_n_grid = seq(0.5, 1, by = 0.05),
estimand = "average",
B = 250,
alpha = 0.05,
volatility_window = 3,
num_cores = NULL
)A list with class "PTE_optimal_m_selection" containing:
m_pow_of_n_optimalThe selected \(\gamma\), ready to pass as
m_pow_of_n to PTE_bootstrap_inference.
m_optimalThe corresponding resample size, round(n^m_pow_of_n_optimal).
grid_tableA data.frame with one row per grid point: m_pow_of_n,
m, the point estimate and ci_width for estimand, and its
volatility (NA for the non-eligible boundary points described above).
estimand, B, alpha, volatility_windowEchoed back for reference.
A \(n \times p\) dataframe of covariates where one column is labeled "treatment" and it
is a binary vector of treatment allocations in the study. See PTE_bootstrap_inference.
An \(n\)-length numeric vector which is the response. See PTE_bootstrap_inference.
See PTE_bootstrap_inference. Default "continuous".
See PTE_bootstrap_inference. Default "odds_ratio".
See PTE_bootstrap_inference. Default NULL (use the built-in default for regression_type).
See PTE_bootstrap_inference. Required if regression_type is "survival".
See PTE_bootstrap_inference. Default NULL (use the built-in default).
See PTE_bootstrap_inference. Default NULL (use the built-in default for regression_type).
See PTE_bootstrap_inference. Default NULL (no cleanup).
See PTE_bootstrap_inference. Default TRUE.
See PTE_bootstrap_inference. Default 0.10.
The grid of candidate \(\gamma\) exponents to evaluate (each entry is a candidate value
for PTE_bootstrap_inference's m_pow_of_n argument). Must have at least
volatility_window entries so every interior grid point has a complete neighborhood. The
default, seq(0.5, 1, by = 0.05), brackets the Politis-Romano-Wolf-recommended
\(\gamma = 0.75\) on both sides, up to (but not below) \(\gamma = 0.5\)
(\(m = \sqrt{n}\), a common lower-rate benchmark) and \(\gamma = 1\)
(the classical, here potentially inconsistent, \(n\)-out-of-\(n\) bootstrap, included only as a
reference point -- see volatility_window).
Which of the three PTE estimands' confidence interval width to stabilize: "adversarial",
"average" (the default) or "best"; see PTE_bootstrap_inference.
The number of bootstrap samples to take at each grid point. Since the total cost is
length(m_pow_of_n_grid) * B bootstrap replicates, this defaults to a much smaller
250 than PTE_bootstrap_inference's own default of 3000 -- this
function is for calibrating m_pow_of_n, not for the final reported inference.
Confidence interval size (1 - alpha) used to measure CI width at each grid point. Defaults to 0.05.
The number of adjacent grid points (must be odd and \(\geq 3\)) averaged into the
volatility measure at each interior grid point. Only grid points with a complete, centered
window are eligible to be selected (this excludes the (volatility_window - 1) / 2 grid
points at each end of m_pow_of_n_grid from selection, since their neighborhoods would
otherwise be one-sided and their apparent volatility artificially low). Default 3.
See PTE_bootstrap_inference. Passed through unchanged to each grid point's call
(grid points are evaluated sequentially, not in additional parallel, to avoid nested
parallelism on top of PTE_bootstrap_inference's own).
Adam Kapelner
This function is a calibration step: it runs PTE_bootstrap_inference once per grid
point (so its total cost is roughly length(m_pow_of_n_grid) times that of a single call) and
does not itself return final inference -- rerun PTE_bootstrap_inference at the
selected m_pow_of_n_optimal (with a larger B) to get the confidence intervals and
p-values you report.
Politis, D.N., Romano, J.P. and Wolf, M. (1999) Subsampling. Springer Series in Statistics.
Bickel, P.J. and Sakov, A. (2008) On the choice of m in the m out of n bootstrap and confidence bounds for extrema. Statistica Sinica, 18(3), 967-985.
if (FALSE) {
library(PTE)
data(continuous_example)
X = continuous_example$X
y = continuous_example$y
m_selection = select_optimal_m_prop(
X, y,
regression_type = "continuous",
m_pow_of_n_grid = seq(0.5, 1, by = 0.05),
B = 250,
num_cores = 1
)
m_selection$grid_table
m_selection$m_pow_of_n_optimal
#now rerun inference at the selected m_pow_of_n with a production-sized B
pte_results = PTE_bootstrap_inference(
X, y,
regression_type = "continuous",
m_pow_of_n = m_selection$m_pow_of_n_optimal,
B = 3000
)
pte_results
}
Run the code above in your browser using DataLab