czbenchmarks.cli.types

Attributes

`TaskType`
`ModelArgsDict`
`RuntimeMetricsDict`

Classes

`ModelArgs`	!!! abstract "Usage Documentation"
`TaskArgs`	!!! abstract "Usage Documentation"
`DatasetDetail`	!!! abstract "Usage Documentation"
`ModelDetail`	!!! abstract "Usage Documentation"
`TaskResult`	!!! abstract "Usage Documentation"
`CacheOptions`	!!! abstract "Usage Documentation"

Module Contents

czbenchmarks.cli.types.TaskType

czbenchmarks.cli.types.ModelArgsDict

czbenchmarks.cli.types.RuntimeMetricsDict

class czbenchmarks.cli.types.ModelArgs(/, **data: Any)[source]

Bases: pydantic.BaseModel

!!! abstract “Usage Documentation”: [Models](../concepts/models.md)

A base class for creating Pydantic models.

__class_vars__: The names of the class variables defined on the model.

__private_attributes__: Metadata about the private attributes of the model.

__signature__: The synthesized __init__ [Signature][inspect.Signature] of the model.

__pydantic_complete__: Whether model building is completed, or if there are still undefined fields.

__pydantic_core_schema__: The core schema of the model.

__pydantic_custom_init__: Whether the model has a custom __init__ function.

__pydantic_decorators__: Metadata containing the decorators defined on the model. This replaces Model.__validators__ and Model.__root_validators__ from Pydantic V1.

__pydantic_generic_metadata__: Metadata for generic models; contains data used for a similar purpose to __args__, __origin__, __parameters__ in typing-module generics. May eventually be replaced by these.

__pydantic_parent_namespace__: Parent namespace of the model, used for automatic rebuilding of models.

__pydantic_post_init__: The name of the post-init method for the model, if defined.

__pydantic_root_model__: Whether the model is a [RootModel][pydantic.root_model.RootModel].

__pydantic_serializer__: The pydantic-core SchemaSerializer used to dump instances of the model.

__pydantic_validator__: The pydantic-core SchemaValidator used to validate instances of the model.

__pydantic_fields__: A dictionary of field names and their corresponding [FieldInfo][pydantic.fields.FieldInfo] objects.

__pydantic_computed_fields__: A dictionary of computed field names and their corresponding [ComputedFieldInfo][pydantic.fields.ComputedFieldInfo] objects.

__pydantic_extra__: A dictionary containing extra values, if [extra][pydantic.config.ConfigDict.extra] is set to ‘allow’.

__pydantic_fields_set__: The names of fields explicitly set during instantiation.

__pydantic_private__: Values of private attributes set on the model instance.

name: str

args: dict[str, list[str | int]]

class czbenchmarks.cli.types.TaskArgs(/, **data: Any)[source]

Bases: pydantic.BaseModel, Generic[TaskType]

!!! abstract “Usage Documentation”: [Models](../concepts/models.md)

A base class for creating Pydantic models.

__class_vars__: The names of the class variables defined on the model.

__private_attributes__: Metadata about the private attributes of the model.

__signature__: The synthesized __init__ [Signature][inspect.Signature] of the model.

__pydantic_complete__: Whether model building is completed, or if there are still undefined fields.

__pydantic_core_schema__: The core schema of the model.

__pydantic_custom_init__: Whether the model has a custom __init__ function.

__pydantic_decorators__: Metadata containing the decorators defined on the model. This replaces Model.__validators__ and Model.__root_validators__ from Pydantic V1.

__pydantic_generic_metadata__: Metadata for generic models; contains data used for a similar purpose to __args__, __origin__, __parameters__ in typing-module generics. May eventually be replaced by these.

__pydantic_parent_namespace__: Parent namespace of the model, used for automatic rebuilding of models.

__pydantic_post_init__: The name of the post-init method for the model, if defined.

__pydantic_root_model__: Whether the model is a [RootModel][pydantic.root_model.RootModel].

__pydantic_serializer__: The pydantic-core SchemaSerializer used to dump instances of the model.

__pydantic_validator__: The pydantic-core SchemaValidator used to validate instances of the model.

__pydantic_fields__: A dictionary of field names and their corresponding [FieldInfo][pydantic.fields.FieldInfo] objects.

__pydantic_computed_fields__: A dictionary of computed field names and their corresponding [ComputedFieldInfo][pydantic.fields.ComputedFieldInfo] objects.

__pydantic_extra__: A dictionary containing extra values, if [extra][pydantic.config.ConfigDict.extra] is set to ‘allow’.

__pydantic_fields_set__: The names of fields explicitly set during instantiation.

__pydantic_private__: Values of private attributes set on the model instance.

model_config: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

name: str

task: TaskType

set_baseline: bool

baseline_args: dict[str, Any]

class czbenchmarks.cli.types.DatasetDetail(/, **data: Any)[source]

Bases: pydantic.BaseModel

!!! abstract “Usage Documentation”: [Models](../concepts/models.md)

A base class for creating Pydantic models.

__class_vars__: The names of the class variables defined on the model.

__private_attributes__: Metadata about the private attributes of the model.

__signature__: The synthesized __init__ [Signature][inspect.Signature] of the model.

__pydantic_complete__: Whether model building is completed, or if there are still undefined fields.

__pydantic_core_schema__: The core schema of the model.

__pydantic_custom_init__: Whether the model has a custom __init__ function.

__pydantic_decorators__: Metadata containing the decorators defined on the model. This replaces Model.__validators__ and Model.__root_validators__ from Pydantic V1.

__pydantic_generic_metadata__: Metadata for generic models; contains data used for a similar purpose to __args__, __origin__, __parameters__ in typing-module generics. May eventually be replaced by these.

__pydantic_parent_namespace__: Parent namespace of the model, used for automatic rebuilding of models.

__pydantic_post_init__: The name of the post-init method for the model, if defined.

__pydantic_root_model__: Whether the model is a [RootModel][pydantic.root_model.RootModel].

__pydantic_serializer__: The pydantic-core SchemaSerializer used to dump instances of the model.

__pydantic_validator__: The pydantic-core SchemaValidator used to validate instances of the model.

__pydantic_fields__: A dictionary of field names and their corresponding [FieldInfo][pydantic.fields.FieldInfo] objects.

__pydantic_computed_fields__: A dictionary of computed field names and their corresponding [ComputedFieldInfo][pydantic.fields.ComputedFieldInfo] objects.

__pydantic_extra__: A dictionary containing extra values, if [extra][pydantic.config.ConfigDict.extra] is set to ‘allow’.

__pydantic_fields_set__: The names of fields explicitly set during instantiation.

__pydantic_private__: Values of private attributes set on the model instance.

name: str

organism: str

property name_display: str

property subset_display: str

class czbenchmarks.cli.types.ModelDetail(/, **data: Any)[source]

Bases: pydantic.BaseModel

!!! abstract “Usage Documentation”: [Models](../concepts/models.md)

A base class for creating Pydantic models.

__class_vars__: The names of the class variables defined on the model.

__private_attributes__: Metadata about the private attributes of the model.

__signature__: The synthesized __init__ [Signature][inspect.Signature] of the model.

__pydantic_complete__: Whether model building is completed, or if there are still undefined fields.

__pydantic_core_schema__: The core schema of the model.

__pydantic_custom_init__: Whether the model has a custom __init__ function.

__pydantic_decorators__: Metadata containing the decorators defined on the model. This replaces Model.__validators__ and Model.__root_validators__ from Pydantic V1.

__pydantic_generic_metadata__: Metadata for generic models; contains data used for a similar purpose to __args__, __origin__, __parameters__ in typing-module generics. May eventually be replaced by these.

__pydantic_parent_namespace__: Parent namespace of the model, used for automatic rebuilding of models.

__pydantic_post_init__: The name of the post-init method for the model, if defined.

__pydantic_root_model__: Whether the model is a [RootModel][pydantic.root_model.RootModel].

__pydantic_serializer__: The pydantic-core SchemaSerializer used to dump instances of the model.

__pydantic_validator__: The pydantic-core SchemaValidator used to validate instances of the model.

__pydantic_fields__: A dictionary of field names and their corresponding [FieldInfo][pydantic.fields.FieldInfo] objects.

__pydantic_computed_fields__: A dictionary of computed field names and their corresponding [ComputedFieldInfo][pydantic.fields.ComputedFieldInfo] objects.

__pydantic_extra__: A dictionary containing extra values, if [extra][pydantic.config.ConfigDict.extra] is set to ‘allow’.

__pydantic_fields_set__: The names of fields explicitly set during instantiation.

__pydantic_private__: Values of private attributes set on the model instance.

type: czbenchmarks.models.types.ModelType

args: ModelArgsDict

property name_display: str

property variant_display: str

class czbenchmarks.cli.types.TaskResult(/, **data: Any)[source]

Bases: pydantic.BaseModel

!!! abstract “Usage Documentation”: [Models](../concepts/models.md)

A base class for creating Pydantic models.

__class_vars__: The names of the class variables defined on the model.

__private_attributes__: Metadata about the private attributes of the model.

__signature__: The synthesized __init__ [Signature][inspect.Signature] of the model.

__pydantic_complete__: Whether model building is completed, or if there are still undefined fields.

__pydantic_core_schema__: The core schema of the model.

__pydantic_custom_init__: Whether the model has a custom __init__ function.

__pydantic_decorators__: Metadata containing the decorators defined on the model. This replaces Model.__validators__ and Model.__root_validators__ from Pydantic V1.

__pydantic_generic_metadata__: Metadata for generic models; contains data used for a similar purpose to __args__, __origin__, __parameters__ in typing-module generics. May eventually be replaced by these.

__pydantic_parent_namespace__: Parent namespace of the model, used for automatic rebuilding of models.

__pydantic_post_init__: The name of the post-init method for the model, if defined.

__pydantic_root_model__: Whether the model is a [RootModel][pydantic.root_model.RootModel].

__pydantic_serializer__: The pydantic-core SchemaSerializer used to dump instances of the model.

__pydantic_validator__: The pydantic-core SchemaValidator used to validate instances of the model.

__pydantic_fields__: A dictionary of field names and their corresponding [FieldInfo][pydantic.fields.FieldInfo] objects.

__pydantic_computed_fields__: A dictionary of computed field names and their corresponding [ComputedFieldInfo][pydantic.fields.ComputedFieldInfo] objects.

__pydantic_extra__: A dictionary containing extra values, if [extra][pydantic.config.ConfigDict.extra] is set to ‘allow’.

__pydantic_fields_set__: The names of fields explicitly set during instantiation.

__pydantic_private__: Values of private attributes set on the model instance.

task_name: str

task_name_display: str

model: ModelDetail

datasets: list[DatasetDetail]

metrics: list[czbenchmarks.metrics.types.MetricResult | czbenchmarks.metrics.types.AggregatedMetricResult]

runtime_metrics: RuntimeMetricsDict

property aggregation_key: str: return a key based on the task, model, and datasets in order to aggregate the same results together

class czbenchmarks.cli.types.CacheOptions(/, **data: Any)[source]

Bases: pydantic.BaseModel

!!! abstract “Usage Documentation”: [Models](../concepts/models.md)

A base class for creating Pydantic models.

__class_vars__: The names of the class variables defined on the model.

__private_attributes__: Metadata about the private attributes of the model.

__signature__: The synthesized __init__ [Signature][inspect.Signature] of the model.

__pydantic_complete__: Whether model building is completed, or if there are still undefined fields.

__pydantic_core_schema__: The core schema of the model.

__pydantic_custom_init__: Whether the model has a custom __init__ function.

__pydantic_decorators__: Metadata containing the decorators defined on the model. This replaces Model.__validators__ and Model.__root_validators__ from Pydantic V1.

__pydantic_generic_metadata__: Metadata for generic models; contains data used for a similar purpose to __args__, __origin__, __parameters__ in typing-module generics. May eventually be replaced by these.

__pydantic_parent_namespace__: Parent namespace of the model, used for automatic rebuilding of models.

__pydantic_post_init__: The name of the post-init method for the model, if defined.

__pydantic_root_model__: Whether the model is a [RootModel][pydantic.root_model.RootModel].

__pydantic_serializer__: The pydantic-core SchemaSerializer used to dump instances of the model.

__pydantic_validator__: The pydantic-core SchemaValidator used to validate instances of the model.

__pydantic_fields__: A dictionary of field names and their corresponding [FieldInfo][pydantic.fields.FieldInfo] objects.

__pydantic_computed_fields__: A dictionary of computed field names and their corresponding [ComputedFieldInfo][pydantic.fields.ComputedFieldInfo] objects.

__pydantic_extra__: A dictionary containing extra values, if [extra][pydantic.config.ConfigDict.extra] is set to ‘allow’.

__pydantic_fields_set__: The names of fields explicitly set during instantiation.

__pydantic_private__: Values of private attributes set on the model instance.

download_embeddings: bool

upload_embeddings: bool

upload_results: bool

remote_cache_url: str

classmethod from_args(args: argparse.Namespace) → CacheOptions[source]