cellxgene_census.experimental.ml.pytorch.experiment_dataloader

cellxgene_census.experimental.ml.pytorch.experiment_dataloader(datapipe: IterDataPipe, num_workers: int = 0, **dataloader_kwargs: Any) → DataLoader

Factory method for torch.utils.data.DataLoader. This method can be used to safely instantiate a torch.utils.data.DataLoader that works with cellxgene_census.experimental.ml.pytorch.ExperimentDataPipe, since some of the torch.utils.data.DataLoader constructor parameters are not applicable when using a torchdata.datapipes.iter.IterDataPipe (shuffle, batch_size, sampler, batch_sampler, collate_fn).

Parameters:

• datapipe – An torchdata.datapipes.iter.IterDataPipe, which can be an cellxgene_census.experimental.ml.pytorch.ExperimentDataPipe or any other torchdata.datapipes.iter.IterDataPipe that has been chained to the cellxgene_census.experimental.ml.pytorch.ExperimentDataPipe.

• num_workers – Number of worker processes to use for data loading. If 0, data will be loaded in the main process.

• **dataloader_kwargs – Additional keyword arguments to pass to the torch.utils.data.DataLoader constructor, except for shuffle, batch_size, sampler, batch_sampler, and collate_fn, which are not supported when using cellxgene_census.experimental.ml.pytorch.ExperimentDataPipe.

Returns:

A torch.utils.data.DataLoader.

Raises:

ValueError – if any of the shuffle, batch_size, sampler, batch_sampler, or collate_fn params are passed as keyword arguments.

Lifecycle

experimental