"NCBITaxon:2697049" for SARS-CoV-2
"NCBITaxon:32630" for ERCC Spike-Ins
| Value | for Organism |
|---|---|
"NCBITaxon:9483"
|
Callithrix jacchus |
"NCBITaxon:9606"
|
Homo sapiens |
"NCBITaxon:9544"
|
Macaca mulatta |
"NCBITaxon:10090"
|
Mus musculus |
"NCBITaxon:9598"
|
Pan troglodytes |
| Value | for Organism |
|---|---|
"NCBITaxon:2697049"
|
SARS-CoV-2 |
"NCBITaxon:32630"
|
ERCC Spike-Ins |
"EFO:0022857" for Visium Spatial Gene Expression V1, then [is_single](https://github.com/chanzuckerberg/single-cell-curation/blob/main/schema/7.0.0/schema.md#is_single) MUST be True indicating that the dataset represents one Space Ranger output for a single tissue section.
* If [`assay_ontology_term_id`](https://github.com/chanzuckerberg/single-cell-curation/blob/main/schema/7.0.0/schema.md#assay_ontology_term_id) is "EFO:0030062" for Slide-seqV2, then [is_single](https://github.com/chanzuckerberg/single-cell-curation/blob/main/schema/7.0.0/schema.md#is_single) MUST be True, indicating that the dataset represents the output for a single array on a puck.
Otherwise, Census MUST NOT include the corresponding observations (cells).
##### Full-gene sequencing assays
Full-gene sequencing assays represent the subset of accepted assays that perform full-gene sequencing when used at the single-cell level. Such data MUST be normalized by gene length for downstream analysis.
| Full-gene sequencing assays | | |
|--------|--------|--------|
| [EFO:0003755](https://www.ebi.ac.uk/ols4/ontologies/efo/classes?obo_id=EFO%3A0003755)| Column | Encoding | Description |
|---|---|---|
| label | string | Human readable label of metadata variable |
| value | string | Value associated to metadata variable |
| label | value |
|---|---|
| census_schema_version | 2.0.0 |
| census_build_date | 2024-04-01 |
| dataset_schema_version | 5.1.0 |
| total_cell_count | 10000 |
| unique_cell_count | 1000 |
| Column | Encoding | Description |
|---|---|---|
| citation | string | As defined in the CELLxGENE schema. |
| collection_id | string | As defined in CELLxGENE Discover data schema (see "Schemas" section for field definitions)". |
| collection_name | string | |
| collection_doi | string | |
| collection_doi_label | string | |
| dataset_id | string | |
| dataset_title | string | |
| dataset_h5ad_path | string | Relative path to the source h5ad file in the Census storage bucket. |
| dataset_total_cell_count | int | Total number of cells from the dataset included in the Census. |
| dataset_version_id | string | As defined in CELLxGENE Discover data schema (see "Schemas" section for field definitions)". |
| Column | Encoding | Description |
|---|---|---|
| organism | string | The value of an organism_label defined in Census table of organisms – `census_obj["census_info"]["organisms"]` – `SOMADataframe` |
| category | string | Cell metadata used for grouping, one of the following:
|
| label | string | Label associated with an instance of metadata (e.g. "lung" if category is "tissue"). "na" if none. |
| ontology_term_id | string | ID associated with an instance of metadata (e.g. "UBERON:0002048" if category is "tissue"). "na" if none. |
| total_cell_count | int | Total number of cell counts for the combination of values of all other fields above. |
| unique_cell_count | int | Unique number of cells for the combination of values of all other fields above. Unique number of cells refers to the cell count, for this group, when is_primary_data == True |
| organism | category | label | ontology_term_id | total_cell_count | unique_cell_count |
|---|---|---|---|---|---|
| Homo sapiens | all | na | na | x | x |
| Homo sapiens | cell_type | cell_type_a | CL:XXXXX | x | x |
| Homo sapiens | cell_type | … | … | x | x |
| Homo sapiens | cell_type | cell_type_N | CL:XXXXX | x | x |
| Homo sapiens | assay | assay_a | EFO:XXXXX | x | x |
| Homo sapiens | assay | … | … | x | x |
| Homo sapiens | assay | assay_N | EFO:XXXXX | x | x |
| Homo sapiens | tissue | tissue_a | UBERON:XXXXX | x | x |
| Homo sapiens | tissue | … | … | x | x |
| Homo sapiens | tissue | tissue_N | UBERON:XXXXX | x | x |
| Homo sapiens | tissue_general | tissue_general_a | UBERON:XXXXX | x | x |
| Homo sapiens | tissue_general | … | … | x | x |
| Homo sapiens | tissue_general | tissue_general_N | UBERON:XXXXX | x | x |
| Homo sapiens | disease | disease_a | MONDO:XXXXX | x | x |
| Homo sapiens | disease | … | … | x | x |
| Homo sapiens | disease | disease_N | MONDO:XXXXX | x | x |
| Homo sapiens | self_reported_ethnicity | self_reported_ethnicity_a | HANCESTRO:XXXXX | x | x |
| Homo sapiens | self_reported_ethnicity | … | … | x | x |
| Homo sapiens | self_reported_ethnicity | self_reported_ethnicity_N | HANCESTRO:XXXXX | x | x |
| Homo sapiens | sex | sex_a | PATO:XXXXX | x | x |
| Homo sapiens | sex | … | … | x | x |
| Homo sapiens | sex | sex_N | PATO:XXXXX | x | x |
| Homo sapiens | suspension_type | suspension_type_a | na | x | x |
| Homo sapiens | suspension_type | … | … | x | x |
| Homo sapiens | suspension_type | suspension_type_N | na | x | x |
| Column | Encoding | Description |
|---|---|---|
| organism_ontology_term_id | string | As defined in the CELLxGENE dataset schema. |
| organism_label | string | As defined in the CELLxGENE dataset schema. |
| organism | string | Machine-friendly name for Single Cell Census Data – `census_obj["census_data"][organism]` – `SOMAExperiment`. Its value MUST be the result of:
|
| organism_ontology_term_id | organism_label | organism |
|---|---|---|
| NCBITaxon:9483 | Callithrix jacchus | callithrix_jacchus |
| NCBITaxon:9606 | Homo sapiens | homo_sapiens |
| NCBITaxon:10090 | Mus musculus | mus_musculus |
organism matches an organism defined in Census table of organisms – `census_obj["census_info"]["organisms"]` – `SOMADataframe`.
For example, non-spatial data for _Homo sapiens_ MUST be stored as a `SOMAExperiment` in `census_obj["census_data"]["homo_sapiens"]`.
For each organism with qualifying data, the `SOMAExperiment` MUST contain the following:
* Cell metadata – `census_obj["census_data"][organism].obs` – `SOMADataFrame`
* Data – `census_obj["census_data"][organism].ms` – `SOMACollection`. This `SOMACollection` MUST only contain one `SOMAMeasurement` in `census_obj["census_data"][organism].ms["RNA"]` with the following:
* Matrix data – `census_obj["census_data"][organism].ms["RNA"].X` – `SOMACollection`. It MUST contain exactly one layer:
* Count matrix – `census_obj["census_data"][organism].ms["RNA"].X["raw"]` – `SOMASparseNDArray`
* Normalized count matrix – `census_obj["census_data"][organism].ms["RNA"].X["normalized"]` – `SOMASparseNDArray`
* Feature metadata – `census_obj["census_data"][organism].ms["RNA"].var` – `SOMAIndexedDataFrame`
* Feature dataset presence matrix – `census_obj["census_data"][organism].ms["RNA"]["feature_dataset_presence_matrix"]` – `SOMASparseNDArray`
#### Matrix Data, count (raw) matrix – `census_obj["census_data"][organism].ms["RNA"].X["raw"]` – `SOMASparseNDArray`
Per the CELLxGENE dataset schema, [all RNA assays MUST include UMI or read counts](https://github.com/chanzuckerberg/single-cell-curation/blob/main/schema/7.0.0/schema.md#x-matrix-layers). These counts MUST be encoded as `float32` in this `SOMASparseNDArray` with a fill value of zero (0), and no explicitly stored zero values.
#### Matrix Data, normalized count matrix – `census_obj["census_data"][organism].ms["RNA"].X["normalized"]` – `SOMASparseNDArray`
This is an experimental data artifact - it may be removed at any time.
A library-sized normalized layer, containing a normalized variant of the count (raw) matrix.
For [full-gene sequencing assays](#full-gene-sequencing-assays), given a value `X[i,j]` in the counts (raw) matrix, library-size normalized values are defined
as `normalized[i,j] = (X[i,j] / var[j].feature_length) / sum(X[i, ] / var.feature_length[j])`.
For all other assays, for a value `X[i,j]` in the counts (raw) matrix, library-size normalized values are defined
as `normalized[i,j] = X[i,j] / sum(X[i, ])`.
#### Feature metadata – `census_obj["census_data"][organism].ms["RNA"].var` – `SOMADataFrame`
The Census MUST only contain features with a [`feature_biotype`](https://github.com/chanzuckerberg/single-cell-curation/blob/main/schema/7.0.0/schema.md#feature_biotype) value of "gene".
The [gene references are pinned](https://github.com/chanzuckerberg/single-cell-curation/blob/main/schema/7.0.0/schema.md#required-gene-annotations) as defined in the CELLxGENE dataset schema.
The following columns MUST be included:
| Column | Encoding | Description |
|---|---|---|
| feature_id | str | Index of adata.var as defined in CELLxGENE dataset schema |
| feature_name | str | As defined in CELLxGENE dataset schema |
| feature_length | int | As defined in CELLxGENE dataset schema. |
| nnz | int64 | For this feature, the number of non-zero values present in the `X['raw']` counts (raw) matrix. |
| n_measured_obs | int64 | For this feature, the number of observations present in the source H5AD (sum of feature presence matrix). |
| Feature_1 | … | Feature_M | |
|---|---|---|---|
| Dataset_soma_joinid_1 | [True|False] |
… | [True|False] |
| … | … | … | … |
| Dataset_soma_joinid_N | [True|False] |
… | [True|False] |
| Column | Encoding | Description |
|---|---|---|
| dataset_id | string | CELLxGENE dataset ID |
| tissue_general_ontology_term_id | string | UBERON ontology term identifier for the high-level tissue mapping assigned by the TissueMapper. |
| tissue_general | string | UBERON ontology label for the high-level tissue mapping assigned by the TissueMapper. |
| assay_ontology_term_id | As defined in CELLxGENE dataset schema | |
| assay | ||
| cell_type_ontology_term_id | ||
| cell_type | ||
| development_stage_ontology_term_id | ||
| development_stage | ||
| disease_ontology_term_id | ||
| disease | ||
| donor_id | ||
| is_primary_data | ||
| observation_joinid | ||
| self_reported_ethnicity_ontology_term_id | ||
| self_reported_ethnicity | ||
| sex_ontology_term_id | ||
| sex | ||
| suspension_type | ||
| tissue_ontology_term_id | ||
| tissue | ||
| tissue_type | ||
| nnz | int64 | For this observation, the number of non-zero measurements in the `X['raw']` counts (raw) matrix. |
| n_measured_vars | int64 | For this observation, the number of features present in the source H5AD (sum of feature presence matrix). |
| raw_sum | float32 | For this observation, the sum of the `X['raw']` counts (raw) matrix values. |
| raw_mean_nnz | float32 | For this observation, the mean of the `X['raw']` counts (raw) matrix values. Zeroes are excluded from the calculation. |
| raw_variance_nnz | float32 | For this observation, the variance of the `X['raw']` counts (raw) matrix values. Zeroes are excluded from the calculation. |
organism matches an organism defined in Census table of organisms – `census_obj["census_info"]["organisms"]` – `SOMADataframe`.
For example, spatial data for _Homo sapiens_ MUST be stored as a `SOMAExperiment` in `census_obj["census_spatial_sequencing"]["homo_sapiens"]`.
**EDITORIAL NOTE: The assay references/restatement will be addressed when this section is rewritten**.
Only Visium Spatial Gene Expression V1 ("EFO:0022857") and Slide-seqV2 ("EFO:0030062") are supported for spatial data. [See the "assays included" section above](assays).
For each organism the `SOMAExperiment` MUST contain the following:
* Cell metadata – `census_obj["census_spatial_sequencing"][organism].obs` – `SOMADataFrame`
* Non-spatial data – `census_obj["census_spatial_sequencing"][organism].ms` – `SOMACollection`. This `SOMACollection` MUST only contain one `SOMAMeasurement` in `census_obj["census_spatial_sequencing"][organism].ms["RNA"]` with the following:
* Matrix data – `census_obj["census_spatial_sequencing"][organism].ms["RNA"].X` – `SOMACollection`. It MUST contain exactly one layer:
* Count matrix – `census_obj["census_spatial_sequencing"][organism].ms["RNA"].X["raw"]` – `SOMASparseNDArray`
* Feature metadata – `census_obj["census_spatial_sequencing"][organism].ms["RNA"].var` – `SOMAIndexedDataFrame`
* Feature dataset presence matrix – `census_obj["census_spatial_sequencing"][organism].ms["RNA"]["feature_dataset_presence_matrix"]` – `SOMASparseNDArray`
* Obs to spatial data mapping:
* Obs to spatial data – `census_obj["census_spatial_sequencing"][organism].obs_spatial_presence` – `SOMADataFrame`
* Spatial data – `census_obj["census_spatial_sequencing"][organism].spatial` – `SOMACollection`.
* Spatial Scenes with spatial data – `census_obj["census_spatial_sequencing"][organism].spatial[scene_id]` – `SOMAScene`. There will be as many Spatial Scenes as spatial datasets. Each`SOMAScene` MUST contain the following:
* Positions array – `census_obj["census_spatial_sequencing"][organism].spatial[scene_id].obsl["loc"]` – `SOMAPointCloudDataFrame`.
* High resolution image – `census_obj["census_spatial_sequencing"][organism].spatial[scene_id].img[library_id]["highres_image"]` – `MultiscaleImage`.
#### Matrix Data, count (raw) matrix – `census_obj["census_spatial_sequencing"][organism].ms["RNA"].X["raw"]` – `SOMASparseNDArray`
Spatial and non-spatial data share the [same requirements](#matrix-data-count-raw-matrix--census_objcensus_dataorganismmsrnaxraw--somasparsendarray).
#### Feature metadata – `census_obj["census_spatial_sequencing"][organism].ms["RNA"].var` – `SOMADataFrame`
Spatial and non-spatial data share the [same requirements](#feature-metadata--census_objcensus_dataorganismmsrnavar--somadataframe).
#### Feature dataset presence matrix – `census_obj["census_spatial_sequencing"][organism].ms["RNA"]["feature_dataset_presence_matrix"]` – `SOMASparseNDArray`
Spatial and non-spatial data share the [same requirements](#feature-dataset-presence-matrix--census_objcensus_dataorganismmsrnafeature_dataset_presence_matrix--somasparsendarray).
#### Cell metadata – `census_obj["census_spatial_sequencing"][organism].obs` – `SOMADataFrame`
Spatial and non-spatial data share the [same requirements](#cell-metadata--census_objcensus_dataorganismobs--somadataframe).
**Important note:** In addition, the following spatial `obs` columns from the CELLxGENE dataset schema MUST be included in this `SOMADataFrame`
| Column | Encoding | Description |
|---|---|---|
| array_col | As defined in CELLxGENE dataset schema | |
| array_row | ||
| in_tissue | ||
| Column | Encoding | Description |
|---|---|---|
| soma_joinid | int | It MUST be valid soma_joinid from census_obj["census_spatial_sequencing"][organism].obs. |
scene_id | string | It MUST be valid scene_id from census_obj["census_spatial_sequencing"][organism].spatial. |
| data | bool | It MUST be True if the scene contains spatial information about the observation, otherwise it MUST be False. |
| Column | Encoding | Description |
|---|---|---|
| x | float | It MUST be the corresponding value in the first column of obsm["spatial"]. As defined in the CELLxGENE dataset schema. |
y | float | It MUST be the corresponding value in the second column of obsm["spatial"]. As defined in the CELLxGENE dataset schema. |
| soma_joinid | integer | It MUST be valid soma_joinid from census_obj["census_spatial_sequencing"][organism].obs. |
| Key | Encoding | Description |
|---|---|---|
| soma_geometry_type | str | MUST be "radius" | soma_geometry | float | Radius of points: diameter/2. If Visium Spatial Gene Expression V1 ("EFO:0022857") diameter MUST be .uns["spatial"][library_id]['spot_diameter_fullres'], as defined in the CELLxGENE dataset schema. For Slide-seqV2 ("EFO:0030062") the radius is a small constant. |
EFO:0008703 for CytoSeq
* Added EFO:0009999 for G&T-Seq
* Added EFO:0010005 for DR-Seq
* Added EFO:0010006 for scM&T-seq
* Added EFO:0010007 for scTrio-seq
* Added EFO:0022604 for 10x 3' v4
* Added EFO:0022605 for 10x 5' v3
* Added EFO:0022857 for Visium Spatial Gene Expression V1
* Added EFO:0022962 for SHARE-seq
* Added EFO:0030001 for sci-CAR
* Added EFO:0030026 for sci-Plex
* Added EFO:0030059 for 10x multiome
* Added EFO:0030060 for mCT-seq
* Added EFO:0900000 for particle-templated instant partition sequencing
* Added EFO:0900001 for Asteria scRNA-seq kit
* Added EFO:0900002 for HIVE CLX Single-Cell RNAseq Solution
* Removed EFO:0008908 for SCI-seq
* Removed EFO:0010713 for 10x immune profiling
* Removed EFO:0010714 for 10x TCR enrichment
* Removed EFO:0010715 for 10x Ig enrichment
* Removed EFO:0010961 for Visium Spatial Gene Expression
* Removed EFO:0010964 for barcoded plate-based single cell RNA-seq
* Removed EFO:0030078 for droplet-based single-cell RNA library preparation
* Inlined and updated the Census Accepted Assays Full Gene (CSV) file
* Added EFO:0009999 for G&T-Seq
* Added EFO:0010006 for scM&T-seq
* Added EFO:0030060 for mCT-seq
* Removed EFO:0008441 for full length single cell RNA sequencing
* Removed EFO:0009810 for full length single nucleus RNA sequencing
* Census metadata – `census_obj["census_info"]["summary"]`
* Deprecated `"number_donors_homo_sapiens"`
* Deprecated `"number_donors_mus_musculus"`
* Census table of organisms – `census_obj["census_info"]["organisms"]`
* Replaced the code reference that documented the value of organism with its requirements
* Cell metadata – `census_obj["census_data"][organism].obs`
* Corrected code references for tissue_general_ontology_term_id and tissue_general
### Version 2.3.0
* Added new top level object for holding spatial information, `census_spatial_sequencing`.
### Version 2.2.0
* Allow organoid data, i.e. `tissue_type` can now be `organoid` other than `tissue`.
### Version 2.1.0
* Update to require [CELLxGENE schema version 5.2.0](https://github.com/chanzuckerberg/single-cell-curation/blob/main/schema/5.2.0/schema.md)
* Adds `collection_doi_label` to "Census table of CELLxGENE Discover datasets – `census_obj["census_info"]["datasets"]`"
### Version 2.0.1
* Update accepted assays for Census based on guidance from curators.
### Version 2.0.0
* Update to require [CELLxGENE schema version 5.0.0](https://github.com/chanzuckerberg/single-cell-curation/blob/main/schema/5.0.0/schema.md)
* Expanded list of assays included in the Census.
* Expanded the list of assays defined as full-gene sequencing assays, which have special `normalized` layer handling.
* Clarified handling of datasets which are multi-species on the obs or var axis.
### Version 1.3.0
* Update to require [CELLxGENE schema version 4.0.0](https://github.com/chanzuckerberg/single-cell-curation/blob/main/schema/4.0.0/schema.md)
* Adds `citation` to "Census table of CELLxGENE Discover datasets – `census_obj["census_info"]["datasets"]`"
* Adds `observation_joinid` and `tissue_type` to `obs` dataframe
### Version 1.2.0
* Update to require [CELLxGENE schema version 3.1.0](https://github.com/chanzuckerberg/single-cell-curation/blob/main/schema/3.1.0/schema.md)
### Version 1.1.0
* Adds `dataset_version_id` to "Census table of CELLxGENE Discover datasets – `census_obj["census_info"]["datasets"]`"
* Add `X["normalized"]` layer
* Add `nnz` and `n_measured_obs` columns to `ms["RNA"].var` dataframe
* Add `nnz`, `n_measured_vars`, `raw_sum`, `raw_mean_nnz` and `raw_variance_nnz` columns to `obs` dataframe
### Version 1.0.0
* Updates text to reflect official name: CZ CELLxGENE Discover Census.
* Updates `census["census_info"]["summary"]` to reflect official name in the column `label`:
* From `"cell_census_build_date"` to `"census_build_date"`.
* From `"cell_census_schema_version"` to `"census_schema_version"`.
* Adds the following row to `census["census_info"]["summary"]`:
* `"dataset_schema_version"`
### Version 0.1.1
* Adds clarifying text for "Feature Dataset Presence Matrix"
### Version 0.1.0
* The "Dataset Presence Matrix" was renamed to "Feature Dataset Presence Matrix" and moved from `census_obj["census_data"][organism].ms["RNA"].varp["dataset_presence_matrix"]` to `census_obj["census_data"][organism].ms["RNA"]["feature_dataset_presence_matrix"]`.
* Editorial: changes all double quotes in the schema to ASCII quotes 0x22.
### Version 0.0.1
* Initial Census schema is published.