Skip to contents

Calculate the precision-recall break-even point

Source: R/g_prbe.R
prbe.Rd

The prbe function finds the points of a precision-recall curve at which precision and recall are equal. It takes an S3 object generated by evalmod() and returns a data frame with one row per break-even point.

Usage

prbe(curves)

# S3 method for class 'curve_info'
prbe(curves)

Arguments

curves

An S3 object generated by evalmod().

Value

The prbe function returns a data frame with the columns modnames, dsids, prbe and baselines. prbe is the common value of precision and recall at the break-even point.

A curve can cross the diagonal more than once, and then the data frame holds one row per crossing, ordered by recall. A curve that never reaches equal precision and recall gets a single row of NA, which at a low proportion of positives is a common and correct answer: such a curve can leave the origin below the diagonal and never catch up.

baselines is the proportion of positives, which is where a classifier that ranks at random breaks even - at chance the curve is flat at the prevalence, so it meets the diagonal at that recall. A break-even point of 0.2 is chance on data that is 20% positive and five times chance on data that is 4% positive. See Reading an area against its baseline in auc().

The origin is not reported as a break-even point. A curve is anchored at recall 0, where precision is 1 if the top-ranked instance is a positive and 0 if it is a negative; in the second case precision and recall are equal there, but nothing has been retrieved and the equality is an artifact of where the curve starts rather than a point at which the classifier balances the two.

How this differs from ROCR

ROCR::performance(pred, "prbe") interpolates linearly between adjacent raw precision-recall points to find the crossing. Linear interpolation between precision-recall points is not correct, which is the reason this package exists, so prbe reads the crossing off the curve evalmod() has already interpolated properly. The two agree wherever the crossing falls on a point both of them hold, and differ where it falls between two of them.

See also

evalmod() for generating S3 objects with performance evaluation metrics. auc() for the areas under the curves.

Examples


##################################################
### Single model & single test dataset
###
samps <- create_sim_samples(1, 50, 50, "good_er")
sscurves <- evalmod(scores = samps[["scores"]], labels = samps[["labels"]])
prbe(sscurves)
#>   modnames dsids prbe baselines
#> 1       m1     1  0.7       0.5
#> 2       m1     1  0.7       0.5

##################################################
### Multiple models & multiple test datasets
###
samps2 <- create_sim_samples(2, 50, 50, c("poor_er", "good_er"))
mmdat <- mmdata(samps2[["scores"]], samps2[["labels"]],
  modnames = samps2[["modnames"]], dsids = samps2[["dsids"]]
)
mmcurves <- evalmod(mmdat, raw_curves = TRUE)
prbe(mmcurves)
#>   modnames dsids prbe baselines
#> 1  poor_er     1 0.76       0.5
#> 2  good_er     1 0.74       0.5
#> 3  poor_er     2 0.74       0.5
#> 4  good_er     2 0.68       0.5