{ "cells": [ { "attachments": {}, "cell_type": "markdown", "metadata": { "id": "YkHzXhTHvXUc" }, "source": [ "# Querying data using the gget cellxgene module\n", "\n", "*By Laura Luebbert, lauraluebbert@caltech.edu.*\n", "\n", "[gget](https://github.com/pachterlab/gget) is a free, open-source command-line tool and Python package that enables efficient querying of genomic databases. gget consists of a collection of separate but interoperable modules, each designed to facilitate one type of database querying in a single line of code.\n", "\n", "The [gget cellxgene](https://pachterlab.github.io/gget/cellxgene.html) module builds on the [CZ CELLxGENE Discover Census](https://chanzuckerberg.github.io/cellxgene-census/) to query data from [CZ CELLxGENE Discover](https://cellxgene.cziscience.com/). This notebook briefly introduces the [gget cellxgene](https://pachterlab.github.io/gget/cellxgene.html) module by providing one simple example for each supported query type.\n", "\n", "If you use gget cellxgene in a publication, please [cite gget](https://pachterlab.github.io/gget/cite.html) in addition to [citing CZ CELLxGENE](https://cellxgene.cziscience.com/docs/08__Cite%20cellxgene%20in%20your%20publications).\n", "\n", "You can also [open this notebook in Google Colab](https://colab.research.google.com/github/chanzuckerberg/cellxgene-census/blob/main/api/python/notebooks/api_demo/census_gget_demo.ipynb).\n", "\n", "**Contents** \n", "\n", "1. Install gget.\n", "2. Fetch an [AnnData](https://anndata.readthedocs.io/en/latest/) object by selecting gene(s), tissue(s) and cell type(s).\n", "3. Plot a dot plot similar to those shown on the CZ CELLxGENE Discover [Gene Expression](https://cellxgene.cziscience.com/gene-expression).\n", "4. Fetch only cell metadata (corresponds to AnnData.obs).\n", "5. Use [gget cellxgene](https://pachterlab.github.io/gget/cellxgene.html) from the command line." ] }, { "attachments": {}, "cell_type": "markdown", "metadata": { "id": "gaPshWPxwzo9" }, "source": [ "## Install gget and set up cellxgene module" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "execution": { "iopub.execute_input": "2023-07-28T16:16:11.920035Z", "iopub.status.busy": "2023-07-28T16:16:11.919631Z", "iopub.status.idle": "2023-07-28T16:16:22.202695Z", "shell.execute_reply": "2023-07-28T16:16:22.202036Z" }, "id": "bKTgv7hCQxS1" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\r\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip available: \u001b[0m\u001b[31;49m22.3.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.2.1\u001b[0m\r\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\r\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "Fri Jul 28 16:16:17 2023 INFO Installing cellxgene-census package (requires pip).\n", "Fri Jul 28 16:16:22 2023 INFO cellxgene_census installed succesfully.\n" ] } ], "source": [ "# The cellxgene module was added to gget in version 0.25.7\n", "!pip install -q gget >=0.25.7\n", "\n", "import gget\n", "\n", "gget.setup(\"cellxgene\")" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "execution": { "iopub.execute_input": "2023-07-28T16:16:22.205662Z", "iopub.status.busy": "2023-07-28T16:16:22.205208Z", "iopub.status.idle": "2023-07-28T16:16:22.209709Z", "shell.execute_reply": "2023-07-28T16:16:22.209150Z" }, "id": "f4hLtaBPToWG", "outputId": "e81057ba-578a-464b-caa5-32edcdd206b4" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Help on function cellxgene in module gget.gget_cellxgene:\n", "\n", "cellxgene(species='homo_sapiens', gene=None, ensembl=False, column_names=['dataset_id', 'assay', 'suspension_type', 'sex', 'tissue_general', 'tissue', 'cell_type'], meta_only=False, tissue=None, cell_type=None, development_stage=None, disease=None, sex=None, is_primary_data=True, dataset_id=None, tissue_general_ontology_term_id=None, tissue_general=None, assay_ontology_term_id=None, assay=None, cell_type_ontology_term_id=None, development_stage_ontology_term_id=None, disease_ontology_term_id=None, donor_id=None, self_reported_ethnicity_ontology_term_id=None, self_reported_ethnicity=None, sex_ontology_term_id=None, suspension_type=None, tissue_ontology_term_id=None, census_version='stable', verbose=True, out=None)\n", " Query data from CZ CELLxGENE Discover (https://cellxgene.cziscience.com/) using the\n", " CZ CELLxGENE Discover Census (https://github.com/chanzuckerberg/cellxgene-census).\n", " \n", " NOTE: Querying large datasets requires a large amount of RAM. Use the cell metadata attributes\n", " to define the (sub)dataset of interest.\n", " The CZ CELLxGENE Discover Census recommends >16 GB of memory and a >5 Mbps internet connection.\n", " \n", " General args:\n", " - species Choice of 'homo_sapiens' or 'mus_musculus'. Default: 'homo_sapiens'.\n", " - gene Str or list of gene name(s) or Ensembl ID(s), e.g. ['ACE2', 'SLC5A1'] or ['ENSG00000130234', 'ENSG00000100170']. Default: None.\n", " NOTE: Set ensembl=True when providing Ensembl ID(s) instead of gene name(s).\n", " See https://cellxgene.cziscience.com/gene-expression for examples of available genes.\n", " - ensembl True/False (default: False). Set to True when genes are provided as Ensembl IDs.\n", " - column_names List of metadata columns to return (stored in AnnData.obs when meta_only=False).\n", " Default: [\"dataset_id\", \"assay\", \"suspension_type\", \"sex\", \"tissue_general\", \"tissue\", \"cell_type\"]\n", " For more options see: https://api.cellxgene.cziscience.com/curation/ui/#/ -> Schemas -> dataset\n", " - meta_only True/False (default: False). If True, returns only metadata dataframe (corresponds to AnnData.obs).\n", " - census_version Str defining version of Census, e.g. \"2023-05-15\" or \"latest\" or \"stable\". Default: \"stable\".\n", " - verbose True/False whether to print progress information. Default True.\n", " - out If provided, saves the generated AnnData h5ad (or csv when meta_only=True) file with the specified path. Default: None.\n", " \n", " Cell metadata attributes:\n", " - tissue Str or list of tissue(s), e.g. ['lung', 'blood']. Default: None.\n", " See https://cellxgene.cziscience.com/gene-expression for examples of available tissues.\n", " - cell_type Str or list of celltype(s), e.g. ['mucus secreting cell', 'neuroendocrine cell']. Default: None.\n", " See https://cellxgene.cziscience.com/gene-expression and select a tissue to see examples of available celltypes.\n", " - development_stage Str or list of development stage(s). Default: None.\n", " - disease Str or list of disease(s). Default: None.\n", " - sex Str or list of sex(es), e.g. 'female'. Default: None.\n", " - is_primary_data True/False (default: True). If True, returns only the canonical instance of the cellular observation.\n", " This is commonly set to False for meta-analyses reusing data or for secondary views of data.\n", " - dataset_id Str or list of CELLxGENE dataset ID(s). Default: None.\n", " - tissue_general_ontology_term_id Str or list of high-level tissue UBERON ID(s). Default: None.\n", " Also see: https://github.com/chanzuckerberg/single-cell-data-portal/blob/9b94ccb0a2e0a8f6182b213aa4852c491f6f6aff/backend/wmg/data/tissue_mapper.py\n", " - tissue_general Str or list of high-level tissue label(s). Default: None.\n", " Also see: https://github.com/chanzuckerberg/single-cell-data-portal/blob/9b94ccb0a2e0a8f6182b213aa4852c491f6f6aff/backend/wmg/data/tissue_mapper.py\n", " - tissue_ontology_term_id Str or list of tissue ontology term ID(s) as defined in the CELLxGENE dataset schema. Default: None.\n", " - assay_ontology_term_id Str or list of assay ontology term ID(s) as defined in the CELLxGENE dataset schema. Default: None.\n", " - assay Str or list of assay(s) as defined in the CELLxGENE dataset schema. Default: None.\n", " - cell_type_ontology_term_id Str or list of celltype ontology term ID(s) as defined in the CELLxGENE dataset schema. Default: None.\n", " - development_stage_ontology_term_id Str or list of development stage ontology term ID(s) as defined in the CELLxGENE dataset schema. Default: None.\n", " - disease_ontology_term_id Str or list of disease ontology term ID(s) as defined in the CELLxGENE dataset schema. Default: None.\n", " - donor_id Str or list of donor ID(s) as defined in the CELLxGENE dataset schema. Default: None.\n", " - self_reported_ethnicity_ontology_term_id Str or list of self reported ethnicity ontology ID(s) as defined in the CELLxGENE dataset schema. Default: None.\n", " - self_reported_ethnicity Str or list of self reported ethnicity as defined in the CELLxGENE dataset schema. Default: None.\n", " - sex_ontology_term_id Str or list of sex ontology ID(s) as defined in the CELLxGENE dataset schema. Default: None.\n", " - suspension_type Str or list of suspension type(s) as defined in the CELLxGENE dataset schema. Default: None.\n", " \n", " Returns AnnData object (when meta_only=False) or dataframe (when meta_only=True).\n", "\n" ] } ], "source": [ "# Display all options of the cellxgene gget module\n", "help(gget.cellxgene)" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": { "id": "6j6vqbOXw9X3" }, "source": [ "## Fetch an [AnnData](https://anndata.readthedocs.io/en/latest/) object by selecting gene(s), tissue(s) and cell type(s)\n", "You can use all of the options listed above to filter for data of interest. Here, we will demonstrate the module by fetching a small dataset containing only three genes and two lung cell types:" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "execution": { "iopub.execute_input": "2023-07-28T16:16:22.212324Z", "iopub.status.busy": "2023-07-28T16:16:22.211965Z", "iopub.status.idle": "2023-07-28T16:16:43.724382Z", "shell.execute_reply": "2023-07-28T16:16:43.723763Z" }, "id": "OnDHwjjSQ2uD", "outputId": "9cdb068d-300a-4cdc-f0cb-cf62d0eb3097" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Fri Jul 28 16:16:22 2023 INFO Fetching AnnData object from CZ CELLxGENE Discover. This might take a few minutes...\n", "The \"stable\" release is currently 2023-07-25. Specify 'census_version=\"2023-07-25\"' in future calls to open_soma() to ensure data consistency.\n", "Fri Jul 28 16:16:22 2023 INFO The \"stable\" release is currently 2023-07-25. Specify 'census_version=\"2023-07-25\"' in future calls to open_soma() to ensure data consistency.\n" ] } ], "source": [ "# Fetch AnnData object based on specified genes, tissue and cell types\n", "adata = gget.cellxgene(\n", " gene=[\"ACE2\", \"ABCA1\", \"SLC5A1\"], tissue=\"lung\", cell_type=[\"mucus secreting cell\", \"neuroendocrine cell\"]\n", ")" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": { "id": "6krDnFMLyeRl" }, "source": [ "Let's look at some of the features of the AnnData object we just fetched:" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "execution": { "iopub.execute_input": "2023-07-28T16:16:43.727629Z", "iopub.status.busy": "2023-07-28T16:16:43.727173Z", "iopub.status.idle": "2023-07-28T16:16:43.733167Z", "shell.execute_reply": "2023-07-28T16:16:43.732546Z" }, "id": "MqZM-2uNTt1L", "outputId": "7b14f3e9-8264-428d-b6d4-3320e17776d5" }, "outputs": [ { "data": { "text/plain": [ "AnnData object with n_obs × n_vars = 3679 × 3\n", " obs: 'dataset_id', 'assay', 'suspension_type', 'sex', 'tissue_general', 'tissue', 'cell_type', 'is_primary_data'\n", " var: 'soma_joinid', 'feature_id', 'feature_name', 'feature_length'" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "adata" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": { "id": "Yl34ulk7ziH8" }, "source": [ "A few thousand cells from CZ CELLxGENE Discover matched the filters specified above and their ACE2, ABCA1, and SLC5A1 expression matrix in lung mucus secreting and neuroendocrine cells was fetched. The `.var` and `.obs` layers contain additional information about each gene and cell, respectively:" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 143 }, "execution": { "iopub.execute_input": "2023-07-28T16:16:43.735745Z", "iopub.status.busy": "2023-07-28T16:16:43.735429Z", "iopub.status.idle": "2023-07-28T16:16:43.742939Z", "shell.execute_reply": "2023-07-28T16:16:43.742344Z" }, "id": "qzdc41PdTwDN", "outputId": "a1a48a86-14bc-4559-ab3b-19d7f29fa153" }, "outputs": [ { "data": { "text/html": [ "
\n", " | soma_joinid | \n", "feature_id | \n", "feature_name | \n", "feature_length | \n", "
---|---|---|---|---|
0 | \n", "38 | \n", "ENSG00000165029 | \n", "ABCA1 | \n", "11343 | \n", "
1 | \n", "5332 | \n", "ENSG00000130234 | \n", "ACE2 | \n", "9739 | \n", "
2 | \n", "24539 | \n", "ENSG00000100170 | \n", "SLC5A1 | \n", "5081 | \n", "
\n", " | dataset_id | \n", "assay | \n", "suspension_type | \n", "sex | \n", "tissue_general | \n", "tissue | \n", "cell_type | \n", "is_primary_data | \n", "
---|---|---|---|---|---|---|---|---|
0 | \n", "9f222629-9e39-47d0-b83f-e08d610c7479 | \n", "10x 3' v2 | \n", "cell | \n", "unknown | \n", "lung | \n", "lung | \n", "mucus secreting cell | \n", "True | \n", "
1 | \n", "9f222629-9e39-47d0-b83f-e08d610c7479 | \n", "10x 3' v2 | \n", "cell | \n", "unknown | \n", "lung | \n", "lung | \n", "mucus secreting cell | \n", "True | \n", "
2 | \n", "9f222629-9e39-47d0-b83f-e08d610c7479 | \n", "10x 3' v2 | \n", "cell | \n", "unknown | \n", "lung | \n", "lung | \n", "mucus secreting cell | \n", "True | \n", "
3 | \n", "9f222629-9e39-47d0-b83f-e08d610c7479 | \n", "10x 3' v2 | \n", "cell | \n", "unknown | \n", "lung | \n", "lung | \n", "mucus secreting cell | \n", "True | \n", "
4 | \n", "9f222629-9e39-47d0-b83f-e08d610c7479 | \n", "10x 3' v2 | \n", "cell | \n", "unknown | \n", "lung | \n", "lung | \n", "mucus secreting cell | \n", "True | \n", "
... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "
3674 | \n", "8c42cfd0-0b0a-46d5-910c-fc833d83c45e | \n", "10x 3' v2 | \n", "cell | \n", "female | \n", "lung | \n", "lung | \n", "mucus secreting cell | \n", "True | \n", "
3675 | \n", "8c42cfd0-0b0a-46d5-910c-fc833d83c45e | \n", "10x 3' v2 | \n", "cell | \n", "female | \n", "lung | \n", "lung | \n", "mucus secreting cell | \n", "True | \n", "
3676 | \n", "8c42cfd0-0b0a-46d5-910c-fc833d83c45e | \n", "10x 3' v2 | \n", "cell | \n", "female | \n", "lung | \n", "lung | \n", "mucus secreting cell | \n", "True | \n", "
3677 | \n", "8c42cfd0-0b0a-46d5-910c-fc833d83c45e | \n", "10x 3' v2 | \n", "cell | \n", "female | \n", "lung | \n", "lung | \n", "mucus secreting cell | \n", "True | \n", "
3678 | \n", "8c42cfd0-0b0a-46d5-910c-fc833d83c45e | \n", "10x 3' v2 | \n", "cell | \n", "female | \n", "lung | \n", "lung | \n", "mucus secreting cell | \n", "True | \n", "
3679 rows × 8 columns
\n", "\n", " | dataset_id | \n", "assay | \n", "suspension_type | \n", "sex | \n", "tissue_general | \n", "tissue | \n", "cell_type | \n", "is_primary_data | \n", "
---|---|---|---|---|---|---|---|---|
0 | \n", "047d57f2-4d14-45de-aa98-336c6f583750 | \n", "10x 3' v2 | \n", "cell | \n", "unknown | \n", "lung | \n", "lung | \n", "mesenchymal stem cell | \n", "True | \n", "
1 | \n", "047d57f2-4d14-45de-aa98-336c6f583750 | \n", "10x 3' v2 | \n", "cell | \n", "unknown | \n", "lung | \n", "lung | \n", "progenitor cell | \n", "True | \n", "
2 | \n", "047d57f2-4d14-45de-aa98-336c6f583750 | \n", "10x 3' v2 | \n", "cell | \n", "unknown | \n", "lung | \n", "lung | \n", "mesenchymal cell | \n", "True | \n", "
3 | \n", "047d57f2-4d14-45de-aa98-336c6f583750 | \n", "10x 3' v2 | \n", "cell | \n", "unknown | \n", "lung | \n", "lung | \n", "mesenchymal stem cell | \n", "True | \n", "
4 | \n", "047d57f2-4d14-45de-aa98-336c6f583750 | \n", "10x 3' v2 | \n", "cell | \n", "unknown | \n", "lung | \n", "lung | \n", "mesenchymal cell | \n", "True | \n", "
... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "
97547 | \n", "48b37086-25f7-4ecd-be66-f5bb378e3aea | \n", "10x 3' v2 | \n", "cell | \n", "male | \n", "lung | \n", "lung | \n", "fibroblast of lung | \n", "True | \n", "
97548 | \n", "48b37086-25f7-4ecd-be66-f5bb378e3aea | \n", "10x 3' v2 | \n", "cell | \n", "male | \n", "lung | \n", "lung | \n", "natural killer cell | \n", "True | \n", "
97549 | \n", "48b37086-25f7-4ecd-be66-f5bb378e3aea | \n", "10x 3' v2 | \n", "cell | \n", "male | \n", "lung | \n", "lung | \n", "pulmonary interstitial fibroblast | \n", "True | \n", "
97550 | \n", "48b37086-25f7-4ecd-be66-f5bb378e3aea | \n", "10x 3' v2 | \n", "cell | \n", "male | \n", "lung | \n", "lung | \n", "adventitial cell | \n", "True | \n", "
97551 | \n", "48b37086-25f7-4ecd-be66-f5bb378e3aea | \n", "10x 3' v2 | \n", "cell | \n", "male | \n", "lung | \n", "lung | \n", "fibroblast of lung | \n", "True | \n", "
97552 rows × 8 columns
\n", "