o Si "@sddlmZmZmZmZddlZddlmZddlm Z ddl m Z m Z ddl mZddlmZmZddlmZGd d d ejjjZdS) )CallableDictListUnionN)default_collate)CutSet)BatchIOPrecomputedFeatures)validate_for_asr)compute_num_framesifnone)Hdf5MemoryIssueFixc seZdZdZdddefdedeeegefdeee j ge j fde ffdd Z d ed e eee j eefffd d ZZS)K2Speech2TextTranslationDataseta The PyTorch Dataset for the speech translation task using k2 library. This dataset expects to be queried with lists of cut IDs, for which it loads features and automatically collates/batches them. To use it with a PyTorch DataLoader, set ``batch_size=None`` and provide a :class:`SimpleCutSampler` sampler. Each item in this dataset is a dict of: .. code-block:: { 'inputs': float tensor with shape determined by :attr:`input_strategy`: - single-channel: - features: (B, T, F) - audio: (B, T) - multi-channel: currently not supported 'supervisions': [ { 'sequence_idx': Tensor[int] of shape (S,) 'src_text': List[str] of len S 'tgt_text': List[str] of len S # For feature input strategies 'start_frame': Tensor[int] of shape (S,) 'num_frames': Tensor[int] of shape (S,) # For audio input strategies 'start_sample': Tensor[int] of shape (S,) 'num_samples': Tensor[int] of shape (S,) # Optionally, when return_cuts=True 'cut': List[AnyCut] of len S } ] } Dimension symbols legend: * ``B`` - batch size (number of Cuts) * ``S`` - number of supervision segments (greater or equal to B, as each Cut may have multiple supervisions) * ``T`` - number of frames of the longest Cut * ``F`` - number of features The 'sequence_idx' field is the index of the Cut used to create the example in the Dataset. FN return_cutscut_transformsinput_transformsinput_strategycs>t||_t|g|_t|g|_||_tdd|_dS)a2 K2 Speech2TextTranslation IterableDataset constructor. :param return_cuts: When ``True``, will additionally return a "cut" field in each batch with the Cut objects used to create that batch. :param cut_transforms: A list of transforms to be applied on each sampled batch, before converting cuts to an input representation (audio/features). Examples: cut concatenation, noise cuts mixing, etc. :param input_transforms: A list of transforms to be applied on each sampled batch, after the cuts are converted to audio/features. Examples: normalization, SpecAugment, etc. :param input_strategy: Converts cuts into a collated batch of audio/features. By default, reads pre-computed features from disk. d)reset_intervalN) super__init__rr rrrr hdf5_fix)selfrrrr __class__U/home/ubuntu/.local/lib/python3.10/site-packages/lhotse/dataset/speech_translation.pyr@s   z(K2Speech2TextTranslationDataset.__init__cutsreturncst||j|jdd}|jD]}||}q|jdd}||}t|dkr0|\}}}n|\}}|j|}tj t | dd}|j D]}|||d}qH|t ddt|Dd }|d ||jrtd d|D|d d <td d|D} | rggg} } } |dj|djdurz|jjjWn tytdw|D]5} | jD]/}| dd|jdD| fdd|jdD| fdd|jdDqq| |d d<| |d d<| |d d<|S)z Return a new batch, with the batch size automatically determined using the constraints of max_duration and max_cuts. F) ascending)dim)supervision_segmentscSs.g|]\}}|jD] }|j|jddq qS)translated_text)texttgt_text) supervisionsr%custom).0 sequence_idxcut supervisionrrr sz?K2Speech2TextTranslationDataset.__getitem__..)inputsr'r'cSsg|] }|jD]}|qqSr)r')r)r+suprrrr-s r+css0|]}|jD] }|jduod|jvVqqdS)Nword)r' alignment)r)csrrr s z>K2Speech2TextTranslationDataset.__getitem__..rNz[Can't determine the frame_shift -- it is not present either in cuts or the input_strategy. cSsg|]}|jqSr)symbolr)aliwordrrrr-sr0cg|] }t|jdqS frame_shift sampling_rate)r startr6r:rrr-cr8r9)r endr6r:rrr-r> word_startword_end)r rupdatesort_by_durationrrlensupervision_intervalstorchstacklistvaluesrr enumeraterallr;r< extractorAttributeError ValueErrorr'appendr1)rrtnfm input_tplr._rEsegmentsbatchhas_word_alignmentswordsstartsendsr2r3rr:r __getitem__asv                   z+K2Speech2TextTranslationDataset.__getitem__)__name__ __module__ __qualname____doc__r boolrrrrFTensorrrrstrrrY __classcell__rrrrrs 20!r)typingrrrrrFtorch.utils.data.dataloaderr lhotse.cutrlhotse.dataset.input_strategiesrr !lhotse.dataset.speech_recognitionr lhotse.utilsr r lhotse.workaroundsr utilsdataDatasetrrrrrs