Configure split conformal prediction for conformal.
setup_SplitConformal(alpha = 0.1, score = NULL, seed = NULL)SplitConformalConfig object.
Numeric (0, 1): Miscoverage rate. The region covers with
probability at least 1 - alpha.
Optional Character {"absolute", "LAC", "APS"}: Nonconformity score. NULL resolves to "absolute" for a regression and "APS" for a classification.
Optional Integer [0, Inf): Random seed for the APS draw. NULL draws one and records it.
EDG
The reference construction: score every calibration case, take q to be the
score at rank ceiling((n + 1) * (1 - alpha)) from the bottom, and return
every outcome whose score would not exceed it. Coverage is at least
1 - alpha, and needs only that the calibration cases and the test case be
exchangeable.
alpha has a floor set by the calibration count. A finite region needs
ceiling((n + 1) * (1 - alpha)) <= n, so alpha = 0.05 needs at least 19
calibration cases and alpha = 0.01 at least 99. conformal() aborts naming
the count.
score is a choice for a classification only. A regression has one
score, abs(y - yhat), which is what NULL resolves to.
"APS", the default, accumulates the predicted probabilities from the most
probable class down to the true one. It produces larger sets, with better
coverage of the cases where the model is uncertain.
"LAC" scores 1 - p(y). It produces the smallest sets attaining valid
marginal coverage, and under-covers the cases where the model is
uncertain.
APS is randomized (Romano, Sesia and Candes, 2020): the true label's own
probability enters its score multiplied by a uniform draw, one per case.
Without it, every case whose true label ranks last
scores exactly 1, so a model erring on more than alpha of its cases puts
the threshold at 1 and every set holds every label -- valid coverage carrying
no information. seed keeps the draw reproducible: left NULL, one is drawn
and recorded on the returned config, so an unseeded region can still be
reproduced from what it reports. "LAC" draws nothing and ignores it.
setup_SplitConformal()
setup_SplitConformal(alpha = 0.05)
setup_SplitConformal(score = "LAC")
Run the code above in your browser using DataLab