get_data_pl

DataModule

DataModule(name: str = 'bookcorpus', split: str = 'train', batch_size: int = 4, num_workers: int = 16, pin_memory: bool = True)

Bases: LightningDataModule

Initializes a DataLoader object for "bookcorpus". Support for more datasets coming soon.

Examples:

>>> data  = DataModule()

setup

setup(stage) -> None

Initializes a huggingface dataset: bookcorpus.

train_dataloader

train_dataloader() -> DataLoader

Creates instance of DataLoader.

Returns:

• DataLoader –

  A DataLoader for a specified dataset.