aif360 (version 0.1.0)

classification_metric: Classification Metric

Description

Class for computing metrics based on two BinaryLabelDatasets. The first dataset is the original one and the second is the output of the classification transformer (or similar)

Usage

classification_metric(dataset, classified_dataset, unprivileged_groups, privileged_groups)

Arguments

dataset

(BinaryLabelDataset) Dataset containing ground-truth labels

classified_dataset

(BinaryLabelDataset) Dataset containing predictions

unprivileged_groups

Unprivileged groups. List containing unprivileged protected attribute name and value of the unprivileged protected attribute.

privileged_groups

Privileged groups. List containing privileged protected attribute name and value of the privileged protected attribute.

See Also

Explore available classification metrics explanations here

Available metrics:

• accuracy

• average_abs_odds_difference

• average_odds_difference

• between_all_groups_coefficient_of_variation

• between_all_groups_generalized_entropy_index

• between_all_groups_theil_index

• between_group_coefficient_of_variation

• between_group_generalized_entropy_index

• between_group_theil_index

• binary_confusion_matrix

• coefficient_of_variation

• disparate_impact

• equal_opportunity_difference

• error_rate

• error_rate_difference

• error_rate_ratio

• false_discovery_rate

• false_discovery_rate_difference

• false_discovery_rate_ratio

• false_negative_rate

• false_negative_rate_difference

• false_negative_rate_ratio

• false_omission_rate

• false_omission_rate_difference

• false_omission_rate_ratio

• false_positive_rate

• false_positive_rate_difference

• false_positive_rate_ratio

• generalized_binary_confusion_matrix

• generalized_entropy_index

• generalized_false_negative_rate

• generalized_false_positive_rate

• generalized_true_negative_rate

• generalized_true_positive_rate

• negative_predictive_value

• num_false_negatives

• num_false_positives

• num_generalized_false_negatives

• num_generalized_false_positives

• num_generalized_true_negatives

• num_generalized_true_positives

• num_pred_negatives

• num_pred_positives

• num_true_negatives

• num_true_positives

• performance_measures

• positive_predictive_value

• power

• precision

• recall

• selection_rate

• sensitivity

• specificity

• statistical_parity_difference

• theil_index

• true_negative_rate

• true_positive_rate

• true_positive_rate_difference

Examples

# NOT RUN {
load_aif360_lib()
# Input dataset
data <- data.frame("feat" = c(0,0,1,1,1,1,0,1,1,0), "label" = c(1,0,0,1,0,0,1,0,1,1))
# Create aif compatible input dataset
act <- aif360::aif_dataset(data_path = data,  favor_label=0, unfavor_label=1,
                            unprivileged_protected_attribute=0,
                            privileged_protected_attribute=1,
                            target_column="label", protected_attribute="feat")
# Classified dataset
pred_data <- data.frame("feat" = c(0,0,1,1,1,1,0,1,1,0), "label" = c(1,0,1,1,1,0,1,0,0,1))
# Create aif compatible classified dataset
pred <- aif360::aif_dataset(data_path = pred_data,  favor_label=0, unfavor_label=1,
                             unprivileged_protected_attribute=0,
                             privileged_protected_attribute=1,
                             target_column="label", protected_attribute="feat")
# Create an instance of classification metric
cm <- classification_metric(act, pred, list('feat', 1), list('feat', 0))
# Access metric functions
cm$accuracy()
# }