The run_benchmark function runs
microbenchmark for specified tools
and test datasets
run_benchmark(testset, toolset, times = 5, unit = "ms", use_sys_time = FALSE)A character vector to specify a test set generated by
create_testset.
A character vector to specify a tool set generated by
create_toolset.
The number of iteration used in
microbenchmark.
A single string to specify the unit used in
summary.microbenchmark.
A Boolean value to specify
system.time is used instead of
summary.microbenchmark.
A data frame of microbenchmark results with additional columns.
The timing of the sklearn tool is not comparable with the timings of
the tools written in R. Every call crosses the R/Python boundary and
converts the input and output vectors, and that overhead is counted as
part of the measurement. On a small test set it often dominates the
curve calculation itself, so the sklearn row measures the cost of
the round trip to Python rather than the speed of the scikit-learn
algorithm. See Toolsklearn for the tool itself, and
run_evalcurve for an evaluation that this does not
affect.
create_testset to generate a test dataset.
create_toolset to generate a tool set.
microbenchmark for benchmarking
details.
if (FALSE) { # \dontrun{
## Benchmarking for b10 and i10 test sets and crv5, auc5, and def5 tool sets
testset <- create_testset("bench", c("b10", "i10"))
toolset <- create_toolset(set_names = "def5")
res1 <- run_benchmark(testset, toolset)
res1
} # }