ModelMetrics v1.2.2

0

Monthly downloads

0th

Percentile

Rapid Calculation of Model Metrics

Collection of metrics for evaluating models written in C++ using 'Rcpp'. Popular metrics include area under the curve, log loss, root mean square error, etc.

Readme

ModelMetrics: Rapid Calculation of Model Metrics

Build Status Build status Coverage Status Downloads

Tyler Hunt thunt@snapfinance.com

Introduction

ModelMetrics is a much faster and reliable package for evaluating models. ModelMetrics is written in using Rcpp making it faster than the other packages used for model metrics.

Installation

You can install this package from CRAN:

install.packages("ModelMetrics")

Or you can install the development version from Github with devtools:

devtools::install_github("JackStat/ModelMetrics")

Benchmark and comparison

N = 100000
Actual = as.numeric(runif(N) > .5)
Predicted = as.numeric(runif(N))

actual = Actual
predicted = Predicted

s1 <- system.time(a1 <- ModelMetrics::auc(Actual, Predicted))
s2 <- system.time(a2 <- Metrics::auc(Actual, Predicted))
# Warning message:
# In n_pos * n_neg : NAs produced by integer overflow
s3 <- system.time(a3 <- pROC::auc(Actual, Predicted))
s4 <- system.time(a4 <- MLmetrics::AUC(Predicted, Actual))
# Warning message:
# In n_pos * n_neg : NAs produced by integer overflow
s5 <- system.time({pp <- ROCR::prediction(Predicted, Actual); a5 <- ROCR::performance(pp, 'auc')})


data.frame(
  package = c("ModelMetrics", "pROC", "ROCR")
  ,Time = c(s1[[3]],s3[[3]],s5[[3]])
)

# MLmetrics and Metrics could not calculate so they are dropped from time comparison
#        package   Time
# 1 ModelMetrics  0.030
# 2         pROC 50.359
# 3         ROCR  0.358

Functions in ModelMetrics

Name Description
confusionMatrix Confusion Matrix
logLoss Log Loss
mae Mean absolute error
ppv Positive Predictive Value
recall Recall, Sensitivity, tpr
mauc Multiclass Area Under the Curve
msle Mean Squared Log Error
npv Negative Predictive Value
f1Score F1 Score
mcc Matthews Correlation Coefficient
fScore F Score
mlogLoss Multiclass Log Loss
gini GINI Coefficient
mse Mean Square Error
kappa kappa statistic
testDF Test data
auc Area Under the Curve
tnr Specificity, True negative rate
brier Brier Score
rmse Root-Mean Square Error
rmsle Root Mean Squared Log Error
ce Classification error
No Results!

Last month downloads

Details

Date 2018-11-03
License GPL (>= 2)
Encoding UTF-8
LazyData true
LinkingTo Rcpp
RoxygenNote 6.0.1
NeedsCompilation yes
Packaged 2018-11-03 16:37:41 UTC; tylerhunt
Repository CRAN
Date/Publication 2018-11-03 17:00:14 UTC
imports data.table , Rcpp
depends R (>= 3.2.2)
suggests testthat
Contributors

Include our badge in your README

[![Rdoc](http://www.rdocumentation.org/badges/version/ModelMetrics)](http://www.rdocumentation.org/packages/ModelMetrics)