czbenchmarks.tasks.embedding

Attributes

logger

Classes

EmbeddingTask

Task for evaluating embedding quality using labeled data.

Module Contents

czbenchmarks.tasks.embedding.logger

class czbenchmarks.tasks.embedding.EmbeddingTask(label_key: str, *, random_seed: int = RANDOM_SEED)

Bases: czbenchmarks.tasks.base.BaseTask

Task for evaluating embedding quality using labeled data.

This task computes quality metrics for embeddings using ground truth labels. Currently supports silhouette score evaluation.

Parameters:

• label_key (str) – Key to access ground truth labels in metadata

• random_seed (int) – Random seed for reproducibility

label_key

property display_name: str

A pretty name to use when displaying task results

property required_inputs: Set[czbenchmarks.datasets.DataType]

Required input data types.

Returns:

Set of required input DataTypes (metadata with labels)

property required_outputs: Set[czbenchmarks.datasets.DataType]

Required output data types.

Returns:

required output types from models this task to run (embedding coordinates)