powerPLS (version 0.2.1)

repeatedCV_test: Repeated k-Fold Cross-Validation with Custom Test Metrics

Description

This function performs repeated k-fold cross-validation and computes a selected performance metric across all repetitions and folds. It allows for different types of performance tests, such as MCC, sensitivity, specificity, R2, F1, and more.

Usage

repeatedCV_test(
  data,
  labels,
  k_folds = 5,
  repeats = 3,
  A = 1,
  test_type = "mccTest",
  seed = 1234
)

Value

A numeric value representing the average performance metric across the outer folds.

Arguments

data

A data frame or matrix of features (predictor variables).

labels

A vector of class labels corresponding to the rows of data.

k_folds

An integer specifying the number of cross-validation folds (default = 5).

repeats

An integer specifying the number of times the cross-validation is repeated (default = 3).

A

number of score components

test_type

A character string specifying the type of test to use. Options include:

• 'mccTest' for Matthews Correlation Coefficient (MCC),

• 'sensitivityTest' for Sensitivity,

• 'specificityTest' for Specificity,

• 'R2Test' for R-squared,

• 'scoreTest' for Score,

• 'F1Test' for F1 Score,

• 'FMTest' for Fowlkes-Mallows Index (FM),

• 'AUCTest' for Area Under the Curve (AUC),

• 'dQ2Test' for dQ2.

Default is 'mccTest'.

seed

An integer for setting the random seed to ensure reproducibility (default = 1234).

Examples

datas <- simulatePilotData(nvar = 30, clus.size = c(15,15),m = 6,nvar_rel = 5,A = 1)
data <- datas$X
labels <- datas$Y
mean_mcc <- repeatedCV_test(data, labels, A = 1, test_type = 'mccTest')
cat('Mean MCC:', mean_mcc, '\n')

mean_score <- repeatedCV_test(data, labels, A = 1, test_type = 'scoreTest')
cat('Mean Sensitivity:', mean_score, '\n')