The precrec package contains several functions and S3 generics to provide a robust platform for performance evaluation of binary classifiers.


The precrec package provides the following six functions.

evalmodMain function to calculate evaluation measures
mmdataReformat input data for performance evaluation calculation
join_scoresJoin scores of multiple models into a list
join_labelsJoin observed labels of multiple test datasets into a list
create_sim_samplesCreate random samples for simulations
format_nfoldCreate n-fold cross validation dataset from data frame

S3 generics

The precrec package provides nine different S3 generics for the S3 objects generated by the evalmod function.

S3 genericLibraryDescription
printbasePrint the calculation results and the summary of the test data a precrec object to a data frame
plotgraphicsPlot performance evaluation measures
autoplotggplot2Plot performance evaluation measures with ggplot2
fortifyggplot2Prepare a data frame for ggplot2
aucprecrecMake a data frame with AUC scores
partprecrecCalculate partial curves and partial AUC scores
paucprecrecMake a data frame with pAUC scores
auc_ciprecrecCalculate confidence intervals of AUC scores

Performance measure calculations

The evalmod function calculates ROC and Precision-Recall curves and returns an S3 object. The generated S3 object can be used with several different S3 generics, such as print and plot. The evalmod function can also calculate basic evaluation measures - error rate, accuracy, specificity, sensitivity, precision, Matthews correlation coefficient, and F-Score.

Data preparation

The mmdata function creates an input dataset for the evalmod function. The generated dataset contains formatted scores and labels.

join_scores and join_labels are helper functions to combine multiple scores and labels.

The create_sim_samples function creates test datasets with five different performance levels.

Data visualization

plot takes an S3 object generated by evalmod as input and plot corresponding curves.

autoplot uses ggplot to plot curves.

Result retrieval takes an S3 object generated by evalmod as input and and returns a data frame with calculated curve points.

auc and pauc returns a data frame with AUC scores and partial AUC scores, respectively. auc_ci returns confidence intervals of AUCs for both ROC and precision-recall curves.