o }oi"@sddlmZmZmZmZddlmZddlZ ddl Z ddl m Z m Z ddlmZddlmZmZddlmZddlmZGdd d ejZGd d d eZdS) )AnyDictListOptionalN)EVAL_DATALOADERSTRAIN_DATALOADERS)data) DataLoaderDataset)MegatronDataSampler)loggingcseZdZdZ              d(d ed eed ed ed ededeeedededededededeeffdd Z d)de ddfddZ de fddZ defd d!Zdefd"d#Zdefd$d%Zdee effd&d'ZZS)*MockDataModulezO Mock data module with data sampling and preprocessing configurations. MN逖TF seq_lengthdecoder_seq_length tokenizerimage_processormicro_batch_sizeglobal_batch_sizerampup_batch_sizenum_train_samplesnum_val_samplesnum_test_samples num_workers pin_memorypersistent_workers task_encodercst||_||_||_||_||_| |_| |_| |_ | |_ | |_ ||_ ||_ ||_|dus4|durWtdddlm}ddlm}|d}|pO|d|_ |pU|j|_t|j|j|||d|_dS)a Initializes the mock data module with data sampling and preprocessing configurations. task_encoder: This Mock data module uses Energon Task encoder if provided. Args: seq_length (int): Maximum sequence length for tokens. decoder_seq_length (Optional[int]): Sequence length for the decoder. Used by Megatron Sampler. tokenizer: Tokenizer for text processing. image_processor: Processor for image preprocessing. micro_batch_size (int): Batch size for training and validation. global_batch_size (int): Total batch size across GPUs. rampup_batch_size (Optional[List[int]]): Batch size ramp-up schedule. Used by Megatron Sampler. num_train_samples (int): Number of training samples. num_val_samples (int): Number of validation samples. num_test_samples (int): Number of testing samples. num_workers (int): Number of workers for data loading. pin_memory (bool): Whether to pin memory for data loading. persistent_workers (bool): Whether workers should remain persistent. task_encoder: Task encoder for Energon tasks. NzVProcessor or tokenizer are not provided! Fall back to `openai/clip-vit-large-patch14`.r) AutoProcessor) AutoTokenizerzopenai/clip-vit-large-patch14)seq_lendecoder_seq_lenrrr)super__init__rrrrrrrrrrrrrr warning transformersr =nemo.collections.common.tokenizers.huggingface.auto_tokenizerr!from_pretrainedr data_sampler)selfrrrrrrrrrrrrrrr r! processor __class__W/home/ubuntu/.local/lib/python3.10/site-packages/nemo/collections/vlm/clip/data/mock.pyr%!s8 %      zMockDataModule.__init__stagereturncCsXt|j|jd|j|j|jd|_t|j|jd|j|j|_t|j|jd|j |j|_ dS)Ntrain)rvalidtest) _MockClipDatasetrrrrr _train_dsr_validation_dsr_test_ds)r+r2r/r/r0setupfs zMockDataModule.setupcCt|ds |||jS)Nr8)hasattrr;_create_dataloaderr8r+r/r/r0train_dataloaderw  zMockDataModule.train_dataloadercCr<)Nr9)r=r;r>r9r?r/r/r0val_dataloader}rAzMockDataModule.val_dataloadercCr<)Nr:)r=r;r>r:r?r/r/r0test_dataloaderrAzMockDataModule.test_dataloadercKs&t|f|j|j|j|j|jd|S)N)rrr batch_size collate_fn)r rrrrrE)r+datasetkwargsr/r/r0r>sz!MockDataModule._create_dataloadercCstdiS)aT Save the state of the data module. This method is called when saving a checkpoint. It generates and saves the state of the data module, including the state of the dataloader and the number of consumed samples. Returns: Dict[str, Any]: A dictionary containing the state of the data module. zHtrainer object not connected to data module object returning empty state)r r&r?r/r/r0 state_dicts zMockDataModule.state_dict)rNNNrrNrrrrTFN)r1)__name__ __module__ __qualname____doc__intrrboolrr%strr;rr@rrBrCr r>rrH __classcell__r/r/r-r0r sf      E r c sneZdZ  ddededededdf fdd Zdefd d Zdeeej ffd d Z ddZ ddZ Z S)r7*Nname num_samplesrseedr3c sPt||_||_|j|_|j}|d|d|_|_||_||_ ||_ dS)Nheightwidth) r$r%rRr vocab_size crop_size image_height image_widthlengthrTr) r+rrrRrSrrTrrXr-r/r0r%s  z_MockClipDataset.__init__cCs|jS)N)r[r?r/r/r0__len__sz_MockClipDataset.__len__cCsxtjj|j|d}t|j|j|jgtj d}t|jd|j |j gtj d}|j dur7|j |ddS||dS)N)rT)sizedtypezThis is Random Mock Text)imagetxt)imagescaptions)nprandom default_rngrTtorch from_numpyintegersrWrint64rYrZfloat32r encode_sample)r+idxnp_gentokensrbr/r/r0 __getitem__s  z_MockClipDataset.__getitem__cCstj|}d|d<|S)z A default implementation of a collation function. Users should override this method to define custom data loaders. Nattention_mask)r dataloaderdefault_collate)r+batchcollated_batchr/r/r0 _collate_fns z_MockClipDataset._collate_fncCs ||S)aMethod that user pass as functor to DataLoader. The method optionally performs neural type checking and add types to the outputs. Please note, subclasses of Dataset should not implement `input_types`. # Usage: dataloader = torch.utils.data.DataLoader( ...., collate_fn=dataset.collate_fn, .... ) Returns ------- Collated batch, with or without types. )rv)r+rtr/r/r0rEs z_MockClipDataset.collate_fn)rQN)rIrJrKrOrMr%r\rrgTensorrprvrErPr/r/r-r0r7s$  r7)typingrrrrlightning.pytorchpytorchplnumpyrdrg!lightning.pytorch.utilities.typesrr torch.utilsrtorch.utils.datar r nemo.lightning.pytorch.pluginsr nemo.utilsr LightningDataModuler r7r/r/r/r0s