Plot performance evaluation metrics with ggplot2

The autoplot function plots performance evaluation metrics by using ggplot2 instead of the general R plot.

Usage

# S3 method for class 'sscurves'
autoplot(object, curvetype = c("ROC", "PRC"), ...)

# S3 method for class 'mscurves'
autoplot(object, curvetype = c("ROC", "PRC"), ...)

# S3 method for class 'smcurves'
autoplot(object, curvetype = c("ROC", "PRC"), ...)

# S3 method for class 'mmcurves'
autoplot(object, curvetype = c("ROC", "PRC"), ...)

# S3 method for class 'sspoints'
autoplot(object, curvetype = .get_obj_metrics(object), ...)

# S3 method for class 'mspoints'
autoplot(object, curvetype = .get_obj_metrics(object), ...)

# S3 method for class 'smpoints'
autoplot(object, curvetype = .get_obj_metrics(object), ...)

# S3 method for class 'mmpoints'
autoplot(object, curvetype = .get_obj_metrics(object), ...)

# S3 method for class 'ssxycurves'
autoplot(object, ...)

# S3 method for class 'msxycurves'
autoplot(object, ...)

# S3 method for class 'smxycurves'
autoplot(object, ...)

# S3 method for class 'mmxycurves'
autoplot(object, ...)

Arguments

object

An S3 object generated by evalmod(). The autoplot function accepts the following S3 objects for two different modes, "rocprc" and "basic".

1. ROC and Precision-Recall curves (mode = "rocprc")

   S3 object	# of models	# of test datasets
   sscurves	single	single
   mscurves	multiple	single
   smcurves	single	multiple
   mmcurves	multiple	multiple

2. Basic evaluation metrics (mode = "basic")

   S3 object	# of models	# of test datasets
   sspoints	single	single
   mspoints	multiple	single
   smpoints	single	multiple
   mmpoints	multiple	multiple

See the Value section of evalmod() for more details.

curvetype

A character vector with the following curve types.

1. ROC and Precision-Recall curves (mode = "rocprc")

   curvetype	description
   ROC	ROC curve
   PRC	Precision-Recall curve

   Multiple curvetype can be combined, such as c("ROC", "PRC").

2. Basic evaluation metrics (mode = "basic")

   curvetype	description
   error	Normalized ranks vs. error rate
   accuracy	Normalized ranks vs. accuracy
   specificity	Normalized ranks vs. specificity
   sensitivity	Normalized ranks vs. sensitivity
   precision	Normalized ranks vs. precision
   mcc	Normalized ranks vs. Matthews correlation coefficient
   fscore	Normalized ranks vs. F-score
   balanced_accuracy	Normalized ranks vs. balanced accuracy
   npv	Normalized ranks vs. negative predictive value
   informedness	Normalized ranks vs. informedness (Youden's J)
   markedness	Normalized ranks vs. markedness
   kappa	Normalized ranks vs. Cohen's kappa

   Multiple curvetype can be combined, such as c("precision", "sensitivity").

...

Following additional arguments can be specified.

type

A character to specify the line type as follows.

"l"

lines

"p"

points

"b"

both lines and points

show_cb

A Boolean value to specify whether point-wise confidence bounds are drawn. It is effective only when calc_avg of the evalmod() function is set to TRUE .

raw_curves

A Boolean value to specify whether raw curves are shown instead of the average curve. It is effective only when raw_curves of the evalmod() function is set to TRUE.

show_legend

A Boolean value to specify whether the legend is shown.

ret_grob

A logical value to indicate whether autoplot returns a grob object. The grob object is internally generated by gridExtra::arrangeGrob(). The grid::grid.draw() function takes a grob object and shows a plot. It is effective only when a multiple-panel plot is generated, for example, when curvetype is c("ROC", "PRC").

reduce_points

A Boolean value to decide whether the points should be reduced. The number kept is x_bins of the evalmod() function, per curve for mode = "rocprc" and per metric for mode = "basic". The default is TRUE for the curves and FALSE for the basic metrics, whose points are the data rather than an interpolation. Reduction changes only what is drawn, never how it was calculated.

Value

The autoplot function returns a ggplot object for a single-panel plot and a frame-grob object for a multiple-panel plot.

See also

evalmod() for generating an S3 object. fortify() for converting a curves and points object to a data frame. plot() for plotting the equivalent curves with the general R plot.

Examples

if (FALSE) { # \dontrun{

## Load libraries
library(ggplot2)
library(grid)

##################################################
### Single model & single test dataset
###

## Load a dataset with 10 positives and 10 negatives
data(P10N10)

## Generate an sscurve object that contains ROC and Precision-Recall curves
sscurves <- evalmod(scores = P10N10$scores, labels = P10N10$labels)

## Plot both ROC and Precision-Recall curves
autoplot(sscurves)

## Reduced/Full supporting points
sampss <- create_sim_samples(1, 50000, 50000)
evalss <- evalmod(scores = sampss$scores, labels = sampss$labels)

# Reduced supporting point
system.time(autoplot(evalss))

# Full supporting points
system.time(autoplot(evalss, reduce_points = FALSE))

## Get a grob object for multiple plots
pp1 <- autoplot(sscurves, ret_grob = TRUE)
plot.new()
grid.draw(pp1)

## A ROC curve
autoplot(sscurves, curvetype = "ROC")

## A Precision-Recall curve
autoplot(sscurves, curvetype = "PRC")

## Generate an sspoints object that contains basic evaluation metrics
sspoints <- evalmod(
  mode = "basic", scores = P10N10$scores,
  labels = P10N10$labels
)

## Normalized ranks vs. basic evaluation metrics
autoplot(sspoints)

## Normalized ranks vs. precision
autoplot(sspoints, curvetype = "precision")


##################################################
### Multiple models & single test dataset
###

## Create sample datasets with 100 positives and 100 negatives
samps <- create_sim_samples(1, 100, 100, "all")
mdat <- mmdata(samps[["scores"]], samps[["labels"]],
  modnames = samps[["modnames"]]
)

## Generate an mscurve object that contains ROC and Precision-Recall curves
mscurves <- evalmod(mdat)

## ROC and Precision-Recall curves
autoplot(mscurves)

## Reduced/Full supporting points
sampms <- create_sim_samples(5, 50000, 50000)
evalms <- evalmod(scores = sampms$scores, labels = sampms$labels)

# Reduced supporting point
system.time(autoplot(evalms))

# Full supporting points
system.time(autoplot(evalms, reduce_points = FALSE))

## Hide the legend
autoplot(mscurves, show_legend = FALSE)

## Generate an mspoints object that contains basic evaluation metrics
mspoints <- evalmod(mdat, mode = "basic")

## Normalized ranks vs. basic evaluation metrics
autoplot(mspoints)

## Hide the legend
autoplot(mspoints, show_legend = FALSE)


##################################################
### Single model & multiple test datasets
###

## Create sample datasets with 100 positives and 100 negatives
samps <- create_sim_samples(10, 100, 100, "good_er")
mdat <- mmdata(samps[["scores"]], samps[["labels"]],
  modnames = samps[["modnames"]],
  dsids = samps[["dsids"]]
)

## Generate an smcurve object that contains ROC and Precision-Recall curves
smcurves <- evalmod(mdat, raw_curves = TRUE)

## Average ROC and Precision-Recall curves
autoplot(smcurves, raw_curves = FALSE)

## Hide confidence bounds
autoplot(smcurves, raw_curves = FALSE, show_cb = FALSE)

## Raw ROC and Precision-Recall curves
autoplot(smcurves, raw_curves = TRUE, show_cb = FALSE)

## Reduced/Full supporting points
sampsm <- create_sim_samples(4, 5000, 5000)
mdatsm <- mmdata(sampsm$scores, sampsm$labels, expd_first = "dsids")
evalsm <- evalmod(mdatsm, raw_curves = TRUE)

# Reduced supporting point
system.time(autoplot(evalsm, raw_curves = TRUE))

# Full supporting points
system.time(autoplot(evalsm, raw_curves = TRUE, reduce_points = FALSE))

## Generate an smpoints object that contains basic evaluation metrics
smpoints <- evalmod(mdat, mode = "basic")

## Normalized ranks vs. average basic evaluation metrics
autoplot(smpoints)


##################################################
### Multiple models & multiple test datasets
###

## Create sample datasets with 100 positives and 100 negatives
samps <- create_sim_samples(10, 100, 100, "all")
mdat <- mmdata(samps[["scores"]], samps[["labels"]],
  modnames = samps[["modnames"]],
  dsids = samps[["dsids"]]
)

## Generate an mscurve object that contains ROC and Precision-Recall curves
mmcurves <- evalmod(mdat, raw_curves = TRUE)

## Average ROC and Precision-Recall curves
autoplot(mmcurves, raw_curves = FALSE)

## Show confidence bounds
autoplot(mmcurves, raw_curves = FALSE, show_cb = TRUE)

## Raw ROC and Precision-Recall curves
autoplot(mmcurves, raw_curves = TRUE)

## Reduced/Full supporting points
sampmm <- create_sim_samples(4, 5000, 5000)
mdatmm <- mmdata(sampmm$scores, sampmm$labels,
  modnames = c("m1", "m2"),
  dsids = c(1, 2), expd_first = "modnames"
)
evalmm <- evalmod(mdatmm, raw_curves = TRUE)

# Reduced supporting point
system.time(autoplot(evalmm, raw_curves = TRUE))

# Full supporting points
system.time(autoplot(evalmm, raw_curves = TRUE, reduce_points = FALSE))

## Generate an mmpoints object that contains basic evaluation metrics
mmpoints <- evalmod(mdat, mode = "basic")

## Normalized ranks vs. average basic evaluation metrics
autoplot(mmpoints)


##################################################
### N-fold cross validation datasets
###

## Load test data
data(M2N50F5)

## Speficy nessesary columns to create mdat
cvdat <- mmdata(
  nfold_df = M2N50F5, score_cols = c(1, 2),
  lab_col = 3, fold_col = 4,
  modnames = c("m1", "m2"), dsids = 1:5
)

## Generate an mmcurve object that contains ROC and Precision-Recall curves
cvcurves <- evalmod(cvdat)

## Average ROC and Precision-Recall curves
autoplot(cvcurves)

## Show confidence bounds
autoplot(cvcurves, show_cb = TRUE)

## Generate an mmpoints object that contains basic evaluation metrics
cvpoints <- evalmod(cvdat, mode = "basic")

## Normalized ranks vs. average basic evaluation metrics
autoplot(cvpoints)
} # }