The aim of the prcbench package is to provide a testing workbench for evaluating precision-recall curves under various conditions. It contains integrated interfaces for the following five tools. It also contains predefined test data sets.

Tool Link
ROCR Tool web site, CRAN
AUCCalculator Tool web site
PerfMeas CRAN
precrec Tool web site, CRAN




AUCCalculator requires a Java runtime (>= 6).

Bioconductor libraries

PerfMeas requires Bioconductor libraries.


  • Install the release version of prcbench from CRAN with install.packages("prcbench").

  • Alternatively, you can install a development version of prcbench from our GitHub repository. To install it:

    1. Make sure you have a working development environment.

      • Windows: Install Rtools (available on the CRAN website).
      • Mac: Install Xcode from the Mac App Store.
      • Linux: Install a compiler and various development libraries (details vary across different flavors of Linux).
    2. Install devtools from CRAN with install.packages("devtools").

    3. Install prcbench from the GitHub repository with devtools::install_github("evalclass/prcbench").


Bioconductor libraries

You can manually install the dependencies from Bioconductor if install.packages fails to access the Bioconductor repository.

if (!requireNamespace("BiocManager", quietly = TRUE))



microbenchmark does not work on some OSs. prcbench uses system.time when microbenchmark is not available.


  • Some OSs require further configuration for rJava.
sudo R CMD javareconf
  • JDKs for macOS Big Sur.
  1. AdoptOpenJDK
  2. AdoptOpenJDK with homebrew
  3. Oracle JDK
install.packages("rJava", configure.args="--disable-jri")


Following two examples show the basic usage of prcbench functions.


The run_benchmark function outputs the result of microbenchmark for specified tools.

## Load library

## Run microbenchmark for auc5 (five tools) on b10 (balanced 5 Ps and 5 Ns)
testset <- create_testset("bench", "b10")
toolset <- create_toolset(set_names = "auc5")
res <- run_benchmark(testset, toolset)
## [1] "microbenchmark is not available. system.time will be used instead."
## [1] "PerfMeas is not available."
## Use knitr::kable to show the result in a table format
knitr::kable(res$tab, digits = 2)
testset toolset toolname min lq mean median uq max neval
b10 auc5 AUCCalculator 4 5 7 11.2 9 31 5
b10 auc5 PerfMeas 0 0 0 0.2 0 1 5
b10 auc5 precrec 7 8 9 42.4 11 177 5
b10 auc5 PRROC 0 1 1 2.6 1 10 5
b10 auc5 ROCR 3 3 4 16.6 24 49 5

Evaluation of precision-recall curves

The run_evalcurve function evaluates precision-recall curves with predefined test datasets. The autoplot shows a plot with the result of the run_evalcurve function.

## ggplot2 is necessary to use autoplot

## Plot base points and the result of precrec on c1, c2, and c3 test sets
testset <- create_testset("curve", c("c1", "c2", "c3"))
toolset <- create_toolset("precrec")
scores1 <- run_evalcurve(testset, toolset)

## Plot the results of PerfMeas and PRROC on c1, c2, and c3 test sets
toolset <- create_toolset(c("PerfMeas", "PRROC"))
scores2 <- run_evalcurve(testset, toolset)
## [1] "PerfMeas is not available."
autoplot(scores2, base_plot = FALSE)


Precrec: fast and accurate precision-recall and ROC curve calculations in R

Takaya Saito; Marc Rehmsmeier

Bioinformatics 2017; 33 (1): 145-147.

doi: 10.1093/bioinformatics/btw570