Calculate p-values for feature sets or features relative to an empirical null or each other using resampled t-tests
stat_test(
data,
iter_data,
row_id,
by_set = FALSE,
hypothesis,
metric,
train_test_sizes,
n_resamples
)object of class data.frame
data.frame of raw classification accuracy results
data.frame containing the values to iterate over for seed and either feature name or set name
integer denoting the row ID for iter_data to filter to
Boolean specifying whether you want to compare feature sets (if TRUE) or individual features (if FALSE).
character denoting whether p-values should be calculated for each feature set or feature (depending on by_set argument) individually relative to the null if use_null = TRUE in tsfeature_classifier through "null", or whether pairwise comparisons between each set or feature should be conducted on main model fits only through "pairwise".
character denoting the classification performance metric to use in statistical testing. Can be one of "accuracy", "precision", "recall", "f1". Defaults to "accuracy"
integer vector containing the train and test set sample sizes
integer denoting the number of resamples that were calculated
Trent Henderson