czbenchmarks.tasks.single_cell

Submodules

• czbenchmarks.tasks.single_cell.cross_species_integration
• czbenchmarks.tasks.single_cell.cross_species_label_prediction
• czbenchmarks.tasks.single_cell.perturbation_expression_prediction

Classes

CrossSpeciesIntegrationOutput	Output for cross-species integration task.
CrossSpeciesIntegrationTask	Task for evaluating cross-species integration quality.
CrossSpeciesIntegrationTaskInput	Pydantic model for CrossSpeciesIntegrationTask inputs.
CrossSpeciesLabelPredictionTaskInput	Pydantic model for CrossSpeciesLabelPredictionTask inputs.
CrossSpeciesLabelPredictionOutput	Base class for task outputs.
CrossSpeciesLabelPredictionTask	Task for cross-species label prediction evaluation.
PerturbationExpressionPredictionOutput	Output for perturbation task.
PerturbationExpressionPredictionTask	Task for evaluating perturbation-induced expression predictions against
PerturbationExpressionPredictionTaskInput	Pydantic model for Perturbation task inputs.

Package Contents

class czbenchmarks.tasks.single_cell.CrossSpeciesIntegrationOutput(/, **data: Any)

Bases: czbenchmarks.tasks.task.TaskOutput

Output for cross-species integration task.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

cell_representation: czbenchmarks.tasks.types.CellRepresentation

labels: czbenchmarks.types.ListLike

species: czbenchmarks.types.ListLike

class czbenchmarks.tasks.single_cell.CrossSpeciesIntegrationTask(*, random_seed: int = RANDOM_SEED)

Bases: czbenchmarks.tasks.task.Task

Task for evaluating cross-species integration quality.

This task computes metrics to assess how well different species’ data are integrated in the embedding space while preserving biological signals. It operates on multiple datasets from different species.

display_name = 'Cross-species Integration'

description = 'Evaluate cross-species integration quality using various integration metrics.'

input_model

baseline_model

requires_multiple_datasets = True

abstract compute_baseline(expression_data: czbenchmarks.tasks.types.CellRepresentation, baseline_input: czbenchmarks.tasks.task.NoBaselineInput = None)

Set a baseline embedding for cross-species integration.

Not implemented as standard preprocessing is not applicable across species.

class czbenchmarks.tasks.single_cell.CrossSpeciesIntegrationTaskInput(/, **data: Any)

Bases: czbenchmarks.tasks.task.TaskInput

Pydantic model for CrossSpeciesIntegrationTask inputs.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

labels: Annotated[List[czbenchmarks.types.ListLike], Field(description='List of ground truth labels for each species dataset (e.g., cell types).')]

organism_list: Annotated[List[czbenchmarks.datasets.types.Organism], Field(description='List of organisms corresponding to each dataset for cross-species evaluation.')]

class czbenchmarks.tasks.single_cell.CrossSpeciesLabelPredictionTaskInput(/, **data: Any)

Bases: czbenchmarks.tasks.task.TaskInput

Pydantic model for CrossSpeciesLabelPredictionTask inputs.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

labels: Annotated[List[czbenchmarks.types.ListLike], Field(description='List of ground truth labels for each species dataset (e.g., cell types).')]

organisms: Annotated[List[czbenchmarks.datasets.types.Organism], Field(description='List of organisms corresponding to each dataset for cross-species evaluation.')]

sample_ids: Annotated[List[czbenchmarks.types.ListLike] | None, Field(description='Optional list of sample/donor IDs for aggregation, one per dataset.')] = None

aggregation_method: Annotated[Literal['none', 'mean', 'median'], Field(description="Method to aggregate cells with the same sample_id ('none', 'mean', or 'median').")] = 'mean'

n_folds: Annotated[int, Field(description='Number of cross-validation folds for intra-species evaluation.')] = 5

class czbenchmarks.tasks.single_cell.CrossSpeciesLabelPredictionOutput(/, **data: Any)

Bases: czbenchmarks.tasks.task.TaskOutput

Base class for task outputs.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

results: List[Dict[str, Any]]

class czbenchmarks.tasks.single_cell.CrossSpeciesLabelPredictionTask(*, random_seed: int = RANDOM_SEED)

Bases: czbenchmarks.tasks.task.Task

Task for cross-species label prediction evaluation.

This task evaluates cross-species transfer by training classifiers on one species and testing on another species. It computes accuracy, F1, precision, recall, and AUROC for multiple classifiers (Logistic Regression, KNN, Random Forest).

The task can optionally aggregate cell-level embeddings to sample/donor level before running classification.

display_name = 'cross-species label prediction'

description = 'Evaluate cross-species label prediction performance using multiple classifiers.'

input_model

baseline_model

requires_multiple_datasets = True

abstract compute_baseline(expression_data: czbenchmarks.tasks.types.CellRepresentation, baseline_input: czbenchmarks.tasks.task.NoBaselineInput = None)

Set a baseline for cross-species label prediction.

Not implemented as standard preprocessing needs to be applied per species.

class czbenchmarks.tasks.single_cell.PerturbationExpressionPredictionOutput(/, **data: Any)

Bases: czbenchmarks.tasks.task.TaskOutput

Output for perturbation task.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

pred_mean_change_dict: Dict[str, numpy.ndarray]

true_mean_change_dict: Dict[str, numpy.ndarray]

class czbenchmarks.tasks.single_cell.PerturbationExpressionPredictionTask(*, random_seed: int = RANDOM_SEED)

Bases: czbenchmarks.tasks.task.Task

Task for evaluating perturbation-induced expression predictions against their ground truth values. This is done by calculating metrics derived from predicted and ground truth log fold change values for each condition. Currently, Spearman rank correlation is supported.

The following arguments are required and must be supplied by the task input class (PerturbationExpressionPredictionTaskInput) when running the task. These parameters are described below for documentation purposes:

• predictions_adata (ad.AnnData):

  The anndata containing model predictions

• dataset_adata (ad.AnnData):

  The anndata object from SingleCellPerturbationDataset.

• pred_effect_operation (Literal[“difference”, “ratio”]):

  How to compute predicted effect between treated and control mean predictions over genes.

  • “ratio” uses \(\log\left(\frac{\text{mean}(\text{treated}) + \varepsilon}{\text{mean}(\text{control}) + \varepsilon}\right)\) when means are positive.

  • “difference” uses \(\text{mean}(\text{treated}) - \text{mean}(\text{control})\) and is generally safe across scales (probabilities, z-scores, raw expression).

  Default is “ratio”.

• gene_index (Optional[pd.Index]):

  The index of the genes in the predictions AnnData.

• cell_index (Optional[pd.Index]):

  The index of the cells in the predictions AnnData.

display_name = 'Perturbation Expression Prediction'

description = 'Evaluate the quality of predicted changes in expression levels for genes that are...

input_model

baseline_model

condition_key = None

abstract compute_baseline(expression_data: czbenchmarks.tasks.types.CellRepresentation, baseline_input: czbenchmarks.tasks.task.NoBaselineInput = None)

Set a baseline embedding for perturbation expression prediction.

Not implemented as this task evaluates expression matrices, not embeddings.

class czbenchmarks.tasks.single_cell.PerturbationExpressionPredictionTaskInput(/, **data: Any)

Bases: czbenchmarks.tasks.task.TaskInput

Pydantic model for Perturbation task inputs.

Dataclass to contain input parameters for the PerturbationExpressionPredictionTask. The row and column ordering of the model predictions can optionallybe provided as cell_index and gene_index, respectively, so the task can align a model matrix that is a subset of or re-ordered relative to the dataset adata.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

adata: Annotated[anndata.AnnData, Field(description='AnnData object from SingleCellPerturbationDataset containing perturbation data and metadata.')]

pred_effect_operation: Annotated[Literal['difference', 'ratio'], Field(description="Method to compute predicted effect: 'difference' (mean(treated) - mean(control)) or 'ratio' (log ratio of means).")] = 'ratio'

gene_index: Annotated[pandas.Index | None, Field(description='Optional gene index for predictions to align model predictions with dataset genes.')] = None

cell_index: Annotated[pandas.Index | None, Field(description='Optional cell index for predictions to align model predictions with dataset cells.')] = None