precrec: A package for computing accurate ROC and Precision-Recall curves

The precrec package contains several functions and S3 generics to provide a robust platform for performance evaluation of binary classifiers.

Functions

The precrec package provides the following six functions.

Function	Description
evalmod	Main function to calculate evaluation measures
mmdata	Reformat input data for performance evaluation calculation
join_scores	Join scores of multiple models into a list
join_labels	Join observed labels of multiple test datasets into a list
create_sim_samples	Create random samples for simulations
format_nfold	Create n-fold cross validation dataset from data frame

S3 generics

The precrec package provides nine different S3 generics for the S3 objects generated by the evalmod function.

S3 generic	Library	Description
print	base	Print the calculation results and the summary of the test data
as.data.frame	base	Convert a precrec object to a data frame
plot	graphics	Plot performance evaluation measures
autoplot	ggplot2	Plot performance evaluation measures with ggplot2
fortify	ggplot2	Prepare a data frame for ggplot2
auc	precrec	Make a data frame with AUC scores
part	precrec	Calculate partial curves and partial AUC scores
pauc	precrec	Make a data frame with pAUC scores
auc_ci	precrec	Calculate confidence intervals of AUC scores

Performance measure calculations

The evalmod function calculates ROC and Precision-Recall curves and returns an S3 object. The generated S3 object can be used with several different S3 generics, such as print and plot. The evalmod function can also calculate basic evaluation measures - error rate, accuracy, specificity, sensitivity, precision, Matthews correlation coefficient, and F-Score.

Data preparation

The mmdata function creates an input dataset for the evalmod function. The generated dataset contains formatted scores and labels.

join_scores and join_labels are helper functions to combine multiple scores and labels.

The create_sim_samples function creates test datasets with five different performance levels.

Data visualization

plot takes an S3 object generated by evalmod as input and plot corresponding curves.

autoplot uses ggplot to plot curves.

Result retrieval

as.data.frame takes an S3 object generated by evalmod as input and and returns a data frame with calculated curve points.

auc and pauc returns a data frame with AUC scores and partial AUC scores, respectively. auc_ci returns confidence intervals of AUCs for both ROC and precision-recall curves.