czbenchmarks.cli.utils

Attributes

log

Functions

get_version(→ str)	Get the current version of the cz-benchmarks library
aggregate_task_results(...)	Aggregate the task results by task_name, model (with args), and set(datasets).

Module Contents

czbenchmarks.cli.utils.log

czbenchmarks.cli.utils.get_version() → str

Get the current version of the cz-benchmarks library

czbenchmarks.cli.utils.aggregate_task_results(results: Iterable[czbenchmarks.cli.types.TaskResult]) → list[czbenchmarks.cli.types.TaskResult]

Aggregate the task results by task_name, model (with args), and set(datasets). Each new result will have a new set of metrics, created by aggregating together metrics of the same type.