o }oi"@sddlmZmZmZmZddlmZddlZ ddl Z ddl m Z m Z ddlmZddlmZmZddlmZGdddejZGd d d eZdS) )DictListOptionalTupleN)EVAL_DATALOADERSTRAIN_DATALOADERS)data) DataLoaderDataset)MegatronDataSamplercseZdZdZ              d)d ed ed edeeefdededeeededededededededeffdd Z d*de ddfdd Z de fd!d"Z defd#d$Zdefd%d&Zdefd'd(ZZS)+MockDataModulea8 Mock DataModule for testing and development. Generates synthetic data for training, validation, and testing purposes. Args: seq_length (int): Sequence length for the generated data. decoder_seq_length (Optional[int]): Decoder sequence length if applicable, used in pp. vocab_size (int): Size of the vocabulary of tokenizer. crop_size (Tuple[int, int]): Image crop size (height, width). micro_batch_size (int): Micro batch size for data loading. global_batch_size (int): Global batch size across all processes. rampup_batch_size (Optional[List[int]]): Batch size ramp-up configuration. num_train_samples (int): Number of training samples to generate. num_val_samples (int): Number of validation samples to generate. num_test_samples (int): Number of test samples to generate. num_workers (int): Number of workers for data loading. pin_memory (bool): Whether to pin memory for data loading. persistent_workers (bool): Whether workers should remain persistent. N0r'TF seq_lengthdecoder_seq_length vocab_size crop_sizemicro_batch_sizeglobal_batch_sizerampup_batch_size tokenizerimage_processornum_train_samplesnum_val_samplesnum_test_samples num_workers pin_memorypersistent_workerscszt||_||_||_||_| |_| |_| |_| |_ ||_ ||_ ||_ ||_ ||_| |_t|j|j|||d|_dS)N)seq_lendecoder_seq_lenrrr)super__init__rrrrrrrr r!r"rrrrr data_sampler)selfrrrrrrrrrrrrr r!r" __class__Y/home/ubuntu/.local/lib/python3.10/site-packages/nemo/collections/vlm/mllama/data/mock.pyr&0s,  zMockDataModule.__init__stagereturncCsRt|j|jd|j|j|_t|j|jd|j|j|_t|j|jd|j|j|_ dS)z(Set up datasets for the specified stage.trainvalidtestN) _MockMLlamaDatasetrrrr _train_dsr_validation_dsr_test_ds)r(r.r+r+r,setupZs zMockDataModule.setupcCt|ds |||jS)z$Returns the DataLoader for training.r4)hasattrr7_create_dataloaderr4r(r+r+r,train_dataloaderf  zMockDataModule.train_dataloadercCr8)z&Returns the DataLoader for validation.r5)r9r7r:r5r;r+r+r,val_dataloaderlr=zMockDataModule.val_dataloadercCr8)z#Returns the DataLoader for testing.r6)r9r7r:r6r;r+r+r,test_dataloaderrr=zMockDataModule.test_dataloadercKs"t|f|j|j|j|jd|S)z/Creates a DataLoader for the specified dataset.)r r!r" collate_fn)r r r!r"r@)r(datasetkwargsr+r+r,r:xsz!MockDataModule._create_dataloader)r NrrrrNNNrrrrTF)r-)__name__ __module__ __qualname____doc__intrrrboolr&strr7rr<rr>r?r r: __classcell__r+r+r)r,r sj       * r c seZdZdZ ddededededdf fd d Zdefd d Zd edej fddZ de ee j ffddZddZddZZS)r3a Mock dataset for generating synthetic data with text and image components. Args: vocab_size (int): Vocabulary size for text data. crop_size (Tuple[int, int]): Image crop size (height, width). name (str): Name of the dataset split ('train', 'valid', 'test'). num_samples (int): Number of samples in the dataset. seq_length (int): Sequence length for the text data. seed (int): Seed for random number generation. *name num_samplesrseedr/Ncs`t||_||_||_|\|_|_||_||_t j |jt j d|_ t j |jt jd|_dS)N)dtype)r%r&rLrr image_height image_widthlengthrNtorchonesfloat loss_maskarangeint64 position_ids)r(rrrLrMrrNr)r+r,r&s  z_MockMLlamaDataset.__init__cCs|jS)z-Returns the number of samples in the dataset.)rRr;r+r+r,__len__sz_MockMLlamaDataset.__len__idxcCs,tjj|j|d}|j|j|jgtjdS)zAGenerates a random sequence of integers representing text tokens.rNsizerO)nprandom default_rngrNintegersrrrX)r(r[np_genr+r+r, _get_textsz_MockMLlamaDataset._get_textc Cstjj|j|d}t|j|j|jdgtj d}t| ddd|j |j f}t|jddgtj dd}| }|dd}|dd}|td d ggtdg|||j|j|d S) zGenerates a single data sample.r\r]rrNi)imagesmasks num_chunkstokensaspect_ratio_idsrVrYlabels)r_r`rarNrS from_numpyrbrrrXstandard_normalrPrQclonetensorrVrY)r(r[rcrlrirmrnr+r+r, __getitem__s "   z_MockMLlamaDataset.__getitem__cCsBi}dd|D|d<d|d<|tj||d|d<|S)z A default implementation of a collation function. Users should override this method to define custom data loaders. cSsg|]}|dqS)rj)pop).0sampler+r+r, sz2_MockMLlamaDataset._collate_fn.. batch_masksNattention_maskri batch_images)updater dataloaderdefault_collatert)r(batchcollated_batchr+r+r, _collate_fns z_MockMLlamaDataset._collate_fncCs ||S)aMethod that user pass as functor to DataLoader. The method optionally performs neural type checking and add types to the outputs. Please note, subclasses of Dataset should not implement `input_types`. # Usage: dataloader = torch.utils.data.DataLoader( ...., collate_fn=dataset.collate_fn, .... ) Returns ------- Collated batch, with or without types. )r)r(r~r+r+r,r@s z_MockMLlamaDataset.collate_fn)rK)rCrDrErFrIrGr&rZr_ndarrayrdrrSTensorrsrr@rJr+r+r)r,r3s& r3)typingrrrrlightning.pytorchpytorchplnumpyr_rS!lightning.pytorch.utilities.typesrr torch.utilsrtorch.utils.datar r nemo.lightning.pytorch.pluginsr LightningDataModuler r3r+r+r+r,s   i