Evaluate Precision-Recall curves with specified tools and test sets

Source: R/main_evalcurves.R

The run_evalcurve function runs several tests to evaluate the accuracy of Precision-Recall curves.

run_evalcurve(testset, toolset, auto_combo = TRUE)

Arguments

testset

A character vector to specify a test set generated by create_testset.

toolset

A character vector to specify a tool set generated by create_toolset.

auto_combo

A Boolean value to specify whether a combination of test and tool sets is automatically created.

Value

A data frame with validation results.

See also

create_testset to generate a test dataset. create_toolset to generate a tool set.

Examples

## Evaluate curves for c1, c2, c3 test sets and crv5 tool set
testset <- create_testset("curve", c("c1", "c2", "c3"))
toolset <- create_toolset(set_names = "crv5")
res1 <- run_evalcurve(testset, toolset)
res1
#>    testset toolset  toolname score
#> 1       c1    crv5     PRROC   8/8
#> 2       c1    crv5      ROCR   5/8
#> 3       c1    crv5   precrec   8/8
#> 4       c1    crv5   sklearn   7/8
#> 5       c1    crv5 yardstick   7/8
#> 6       c2    crv5     PRROC   8/8
#> 7       c2    crv5      ROCR   5/8
#> 8       c2    crv5   precrec   8/8
#> 9       c2    crv5   sklearn   5/8
#> 10      c2    crv5 yardstick   5/8
#> 11      c3    crv5     PRROC   7/7
#> 12      c3    crv5      ROCR   4/7
#> 13      c3    crv5   precrec   7/7
#> 14      c3    crv5   sklearn   4/7
#> 15      c3    crv5 yardstick   4/7