PredPsych (version 0.1)

ClassPerm: Permutation Analysis for classification

Description

simple function to create permutation testing of a classifier

Usage

ClassPerm(Data, classCol, selectedCols, classifierFun, nSims = 1000,
  plot = TRUE, ...)

Arguments

Data

(dataframe) dataframe of the data

classCol

(numeric) column number that contains the variable to be predicted

selectedCols

(optional) (numeric) all the columns of data that would be used either as predictor or as feature

classifierFun

(optional) (function) classifier function

nSims

(optional) (numeric) number of simulations

plot

(optional) (logical) whether to plot null accuracy distribution

...

(optional) additional arguments for the function

Value

Returns actualAcc of the classification analysis, p-value from permutation testing, nullAcc distribution of the permutation figure containing null distribution

Details

The function implements Permutation tests for classification. Permutation tests are a set of non-parametric methods for hypothesis testing without assuming a particular distribution (Good, 2005). In case of classification analysis, this requires shuffling the labels of the dataset (i.e. randomly shuffling classes/conditions between observations) and calculating accuracies obtained.

Examples

# perform a permutation testing for 10% of the kinematics movement data
# not run
# PermutationResult <- ClassPerm(Data = KinData, classCol = 1,
#  selectedCols = c(1,2,12,22,32,42,52,62,72,82,92,102,112), nSims = 1000)