o }oi +@svddlmZmZddlmZddlZddlmZm Z ddl m Z m Z ddl mZGdddejZGdd d e ZdS) )ListOptionalN)EVAL_DATALOADERSTRAIN_DATALOADERS) DataLoaderDataset)MegatronDataSamplercseZdZdZ             d#d ed ed ed ed eeededededededeffdd Zd$de ddfddZ de fddZ de fddZde fdd Zdefd!d"ZZS)%MockDataModulea A PyTorch Lightning DataModule for creating mock datasets for training, validation, and testing. Args: image_h (int): Height of the images in the dataset. Default is 1024. image_w (int): Width of the images in the dataset. Default is 1024. micro_batch_size (int): Micro batch size for the data sampler. Default is 4. global_batch_size (int): Global batch size for the data sampler. Default is 8. rampup_batch_size (Optional[List[int]]): Ramp-up batch size for the data sampler. Default is None. num_train_samples (int): Number of training samples. Default is 10,000. num_val_samples (int): Number of validation samples. Default is 10,000. num_test_samples (int): Number of testing samples. Default is 10,000. num_workers (int): Number of worker threads for data loading. Default is 8. pin_memory (bool): Whether to use pinned memory for data loading. Default is True. persistent_workers (bool): Whether to use persistent workers for data loading. Default is False. image_precached (bool): Whether the images are pre-cached. Default is False. text_precached (bool): Whether the text data is pre-cached. Default is False. N'TFimage_himage_wmicro_batch_sizeglobal_batch_sizerampup_batch_sizenum_train_samplesnum_val_samplesnum_test_samples num_workers pin_memorypersistent_workerscsvt||_||_||_||_||_| |_| |_| |_ | |_ | |_ ||_ ||_ d|_d|_t|j|||d|_dS)N )seq_lenrrr)super__init__rrrrrrrrimage_precachedtext_precachedrr tokenizer seq_lengthr data_sampler)selfrrrrrrrrrrrrr __class__m/home/ubuntu/.local/lib/python3.10/site-packages/nemo/collections/diffusion/data/diffusion_mock_datamodule.pyr-s*  zMockDataModule.__init__stagereturncCsRtdd|j|j|jd|_tdd|j|j|jd|_tdd|j|j|jd|_dS)z Sets up datasets for training, validation, and testing. Args: stage (str): The stage of the process (e.g., 'fit', 'test'). Default is an empty string. r )image_Himage_WlengthrrN) _MockT2IDatasetrrr _train_dsr_validation_dsr_test_ds)r"r(r%r%r&setupUs* zMockDataModule.setupcCt|ds |||jS)z Returns the training DataLoader. Returns: TRAIN_DATALOADERS: DataLoader for the training dataset. r.)hasattrr1_create_dataloaderr.r"r%r%r&train_dataloaderr  zMockDataModule.train_dataloadercCr2)z Returns the validation DataLoader. Returns: EVAL_DATALOADERS: DataLoader for the validation dataset. r/)r3r1r4r/r5r%r%r&val_dataloader}r7zMockDataModule.val_dataloadercCr2)z Returns the testing DataLoader. Returns: EVAL_DATALOADERS: DataLoader for the testing dataset. r0)r3r1r4r0r5r%r%r&test_dataloaderr7zMockDataModule.test_dataloadercKst|f|j|j|jd|S)a Creates a DataLoader for the given dataset. Args: dataset: The dataset to load. **kwargs: Additional arguments for the DataLoader. Returns: DataLoader: Configured DataLoader for the dataset. )rrr)rrrr)r"datasetkwargsr%r%r&r4s z!MockDataModule._create_dataloader) r r r r Nr r r r TFFF)r')__name__ __module__ __qualname____doc__intrrboolrstrr1rr6rr8r9rr4 __classcell__r%r%r#r&r sV     (   r csHeZdZdZ         dfd d ZddZddZZS)r-aM A mock dataset class for text-to-image tasks, simulating data samples for training and testing. This dataset generates synthetic data for both image and text inputs, with options to use pre-cached latent representations or raw data. The class is designed for use in testing and prototyping machine learning models. Attributes: image_H (int): Height of the generated images. image_W (int): Width of the generated images. length (int): Total number of samples in the dataset. image_key (str): Key for accessing image data in the output dictionary. txt_key (str): Key for accessing text data in the output dictionary. hint_key (str): Key for accessing hint data in the output dictionary. image_precached (bool): Whether to use pre-cached latent representations for images. text_precached (bool): Whether to use pre-cached embeddings for text. prompt_seq_len (int): Sequence length for text prompts. pooled_prompt_dim (int): Dimensionality of pooled text embeddings. context_dim (int): Dimensionality of the text embedding context. vae_scale_factor (int): Scaling factor for the VAE latent representation. vae_channels (int): Number of channels in the VAE latent representation. latent_shape (tuple): Shape of the latent representation for images (if pre-cached). prompt_embeds_shape (tuple): Shape of the text prompt embeddings (if pre-cached). pooped_prompt_embeds_shape (tuple): Shape of pooled text embeddings (if pre-cached). text_ids_shape (tuple): Shape of the text token IDs (if pre-cached). Methods: __getitem__(index): Retrieves a single sample from the dataset based on the specified index. __len__(): Returns the total number of samples in the dataset. 順imagestxthintFr cst||_||_||_||_||_||_||_||_ |jr.| t || t || f|_ |j rA| | f|_ | f|_ | df|_dSdS)N)rrr,HW image_keytxt_keyhint_keyrrr@ latent_shapeprompt_embeds_shapepooped_prompt_embeds_shapetext_ids_shape)r"r*r+r,rOrPrQrrprompt_seq_lenpooled_prompt_dim context_dimvae_scale_factor vae_channelsr#r%r&rs  z_MockT2IDataset.__init__cCsi}|jrt|j|d<t|j|d<ntd|j|j||j<td|j|j||j<|jrKt|j |d<t|j |d<t|j |d<|Sd||j <|S)a Retrieves a single sample from the dataset. The sample can include raw image and text data or pre-cached latent representations, depending on the configuration. Args: index (int): Index of the sample to retrieve. Returns: dict: A dictionary containing the generated data sample. The keys and values depend on whether `image_precached` and `text_precached` are set. Possible keys include: - 'latents': Pre-cached latent representation of the image. - 'control_latents': Pre-cached control latent representation. - 'images': Raw image tensor. - 'hint': Hint tensor for the image. - 'prompt_embeds': Pre-cached text prompt embeddings. - 'pooled_prompt_embeds': Pooled text prompt embeddings. - 'text_ids': Text token IDs. - 'txt': Text input string (if text is not pre-cached). latentscontrol_latentsrL prompt_embedspooled_prompt_embedstext_idszThis is a sample caption input) rtorchrandnrRrMrNrOrQrrSrTrUrP)r"indexitemr%r%r& __getitem__s z_MockT2IDataset.__getitem__cCs|jS)z Returns the total number of samples in the dataset. Returns: int: Total number of samples (`length` attribute). )r,r5r%r%r&__len__sz_MockT2IDataset.__len__) rDrErFrGFFrHrIrJr rK)r<r=r>r?rrdrerCr%r%r#r&r-s % (r-)typingrrlightning.pytorchpytorchplr`!lightning.pytorch.utilities.typesrrtorch.utils.datarrnemo.lightning.pytorch.pluginsrLightningDataModuler r-r%r%r%r&s