Toolsklearn

R6 class of the scikit-learn tool

Format

An R6 class object.

Details

Toolsklearn is a wrapper class for the precision-recall curve calculation of scikit-learn, which is a machine learning library for Python.

The calculation is performed by a standalone Python module that is bundled with prcbench and derived from the scikit-learn source code. As a result, scikit-learn itself is not required, but reticulate, a working Python installation, and numpy are. The tool can be created without them, whereas the actual calculation cannot be performed. In that case the tool returns a flat dummy curve instead of raising an error, in the same way as ToolAUCCalculator does without rJava, so that the predefined tool sets keep working on a machine without Python.

Two AUC calculation methods are available. aucType = 1 uses average precision, which is the summary scikit-learn recommends for precision-recall curves, whereas aucType = 2 uses the trapezoidal rule. The scikit-learn documentation discourages the use of the trapezoidal rule for precision-recall curves.

Timings of this tool are not comparable with those of the tools written in R. Every call crosses the R/Python boundary and converts the input and output vectors, and run_benchmark counts that overhead as part of the measurement. On a small test set it often dominates the curve calculation itself. The accuracy evaluation of run_evalcurve is unaffected.

See also

This class is derived from ToolIFBase. create_toolset for creating a list of tools.

Super class

ToolIFBase -> Toolsklearn

Methods

Public methods

• Toolsklearn$new()

• Toolsklearn$set_drop_intermediate()

• Toolsklearn$set_aucType()

• Toolsklearn$clone()

Inherited methods

• ToolIFBase$call()
• ToolIFBase$get_auc()
• ToolIFBase$get_result()
• ToolIFBase$get_setname()
• ToolIFBase$get_toolname()
• ToolIFBase$get_x()
• ToolIFBase$get_y()
• ToolIFBase$print()
• ToolIFBase$set_setname()
• ToolIFBase$set_toolname()

---

Toolsklearn$new()

Default class initialization method.

Usage

Toolsklearn$new(...)

Arguments

...

set value for drop_intermediate, aucType.

---

Toolsklearn$set_drop_intermediate()

A Boolean value to specify whether suboptimal thresholds are dropped.

Usage

Toolsklearn$set_drop_intermediate(val)

Arguments

val

TRUE: drop, FALSE: keep.

---

Toolsklearn$set_aucType()

Set the AUC calculation method

Usage

Toolsklearn$set_aucType(val)

Arguments

val

1: average precision, 2: trapezoidal rule

---

Toolsklearn$clone()

The objects of this class are cloneable with this method.

Usage

Toolsklearn$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.

Examples

## Initialization
toolsklearn <- Toolsklearn$new()
#> Downloading uv...
#> Done!

## Show object info
toolsklearn
#> 
#>     === Tool interface ===
#> 
#>     Tool name:            sklearn 
#>     Calculate AUC score:  Yes
#>     Store results:        Yes
#>     Prediction performed: No
#>     Available methods:    call(testset, calc_auc, store_res)
#>                           get_toolname()
#>                           set_toolname(toolname)
#>                           get_setname()
#>                           set_setname(setname)
#>                           get_result()
#>                           get_x()
#>                           get_y()
#>                           get_auc()
#>                           set_drop_intermediate(val)
#>                           set_aucType(val)
#>     Help file:            help("Toolsklearn")

## create_toolset should be used for benchmarking and curve evaluation
toolsklearn2 <- create_toolset("sklearn")