czbenchmarks.metrics.types

Classes

MetricType

Enumeration of all supported metric types.

MetricInfo

Stores metadata about a metric.

MetricRegistry

Central registry for all available metrics.

MetricResult

!!! abstract "Usage Documentation"

Module Contents

class czbenchmarks.metrics.types.MetricType(*args, **kwds)[source]

Bases: enum.Enum

Enumeration of all supported metric types.

Each metric type corresponds to a specific evaluation metric that can be computed. The value is the string identifier used in results dictionaries.

ADJUSTED_RAND_INDEX = 'adjusted_rand_index'
NORMALIZED_MUTUAL_INFO = 'normalized_mutual_info'
SILHOUETTE_SCORE = 'silhouette_score'
ENTROPY_PER_CELL = 'entropy_per_cell'
BATCH_SILHOUETTE = 'batch_silhouette'
MEAN_FOLD_ACCURACY = 'mean_fold_accuracy'
MEAN_FOLD_F1_SCORE = 'mean_fold_f1'
MEAN_FOLD_PRECISION = 'mean_fold_precision'
MEAN_FOLD_RECALL = 'mean_fold_recall'
MEAN_FOLD_AUROC = 'mean_fold_auroc'
MEAN_SQUARED_ERROR = 'mean_squared_error'
PEARSON_CORRELATION = 'PEARSON_CORRELATION'
JACCARD = 'jaccard'
class czbenchmarks.metrics.types.MetricInfo(/, **data: Any)[source]

Bases: pydantic.BaseModel

Stores metadata about a metric.

func: Callable

The function that computes the metric

required_args: Set[str]

Set of required argument names

default_params: Dict[str, Any]

Default parameters for the metric function

description: str | None = None

Optional documentation string for custom metrics

tags: Set[str] = None

Set of tags for grouping related metrics

class czbenchmarks.metrics.types.MetricRegistry[source]

Central registry for all available metrics.

Handles registration and computation of metrics with proper validation.

register(metric_type: MetricType, func: Callable, required_args: Set[str] | None = None, default_params: Dict[str, Any] | None = None, description: str = '', tags: Set[str] | None = None) None[source]

Register a new metric.

Parameters:
  • metric_type – Type of metric to register

  • func – Function that computes the metric

  • required_args – Set of required argument names

  • default_params – Default parameters for the metric function

  • description – Documentation string

  • tags – Set of tags for grouping metrics

compute(metric_type: MetricType, **kwargs) float[source]

Compute a metric with the given parameters.

Parameters:
  • metric_type – Type of metric to compute

  • **kwargs – Arguments to pass to metric function

Returns:

Computed metric value

Raises:

ValueError – If metric type unknown or missing required args

get_info(metric_type: MetricType) MetricInfo[source]

Get metadata about a metric.

Parameters:

metric_type – Type of metric

Returns:

MetricInfo object with metric metadata

Raises:

ValueError – If metric type unknown

list_metrics(tags: Set[str] | None = None) Set[MetricType][source]

List available metrics, optionally filtered by tags.

Parameters:

tags – If provided, only return metrics with all these tags

Returns:

Set of matching MetricType values

class czbenchmarks.metrics.types.MetricResult(/, **data: Any)[source]

Bases: pydantic.BaseModel

!!! abstract “Usage Documentation”

[Models](../concepts/models.md)

A base class for creating Pydantic models.

__class_vars__

The names of the class variables defined on the model.

__private_attributes__

Metadata about the private attributes of the model.

__signature__

The synthesized __init__ [Signature][inspect.Signature] of the model.

__pydantic_complete__

Whether model building is completed, or if there are still undefined fields.

__pydantic_core_schema__

The core schema of the model.

__pydantic_custom_init__

Whether the model has a custom __init__ function.

__pydantic_decorators__

Metadata containing the decorators defined on the model. This replaces Model.__validators__ and Model.__root_validators__ from Pydantic V1.

__pydantic_generic_metadata__

Metadata for generic models; contains data used for a similar purpose to __args__, __origin__, __parameters__ in typing-module generics. May eventually be replaced by these.

__pydantic_parent_namespace__

Parent namespace of the model, used for automatic rebuilding of models.

__pydantic_post_init__

The name of the post-init method for the model, if defined.

__pydantic_root_model__

Whether the model is a [RootModel][pydantic.root_model.RootModel].

__pydantic_serializer__

The pydantic-core SchemaSerializer used to dump instances of the model.

__pydantic_validator__

The pydantic-core SchemaValidator used to validate instances of the model.

__pydantic_fields__

A dictionary of field names and their corresponding [FieldInfo][pydantic.fields.FieldInfo] objects.

__pydantic_computed_fields__

A dictionary of computed field names and their corresponding [ComputedFieldInfo][pydantic.fields.ComputedFieldInfo] objects.

__pydantic_extra__

A dictionary containing extra values, if [extra][pydantic.config.ConfigDict.extra] is set to ‘allow’.

__pydantic_fields_set__

The names of fields explicitly set during instantiation.

__pydantic_private__

Values of private attributes set on the model instance.

metric_type: MetricType
value: float
params: Dict[str, Any] | None = None