cellxgene_census.experimental.ml.pytorch.experiment_dataloader

cellxgene_census.experimental.ml.pytorch.experiment_dataloader(datapipe: IterDataPipe, num_workers: int = 0, **dataloader_kwargs: Any) DataLoader

Factory method for PyTorch DataLoader. This method can be used to safely instantiate a DataLoader that works with the ExperimentDataPipe, since some of the DataLoader constructor params are not applicable when using a IterDataPipe (shuffle, batch_size, sampler, batch_sampler, collate_fn).

Parameters:
  • datapipe – A PyTorch IterDataPipe, which can be an ExperimentDataPipe or any other IterDataPipe that has been chained to the ExperimentDataPipe.

  • num_workers – Number of worker processes to use for data loading. If 0, data will be loaded in the main process.

  • **dataloader_kwargs – Additional keyword arguments to pass to the torch.utils.data.DataLoader constructor, except for shuffle, batch_size, sampler, batch_sampler, and collate_fn, which are not supported when using ExperimentDataPipe. See https://pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader.

Returns:

A torch.utils.data.DataLoader.

Raises:
  • ValueError – if any of the shuffle, batch_size, sampler, batch_sampler, or collate_fn params

  • are passed as keyword arguments.

Lifecycle

experimental