o SiB$@sddlmZmZmZmZddlZddlmZmZddl m Z ddl m Z ddl mZmZddlmZmZddlmZGd d d ejjjZd e d dfd dZdS))CallableDictListUnionN) DataLoaderdefault_collate)validate)CutSet)BatchIOPrecomputedFeatures)compute_num_framesifnone)Hdf5MemoryIssueFixc seZdZdZdddefdedeeegefdeee j ge j fde ffdd Z d ed e eee j eefffd d ZZS)K2SpeechRecognitionDataseta The PyTorch Dataset for the speech recognition task using k2 library. This dataset expects to be queried with lists of cut IDs, for which it loads features and automatically collates/batches them. To use it with a PyTorch DataLoader, set ``batch_size=None`` and provide a :class:`SimpleCutSampler` sampler. Each item in this dataset is a dict of: .. code-block:: { 'inputs': float tensor with shape determined by :attr:`input_strategy`: - single-channel: - features: (B, T, F) - audio: (B, T) - multi-channel: currently not supported 'supervisions': [ { 'sequence_idx': Tensor[int] of shape (S,) 'text': List[str] of len S # For feature input strategies 'start_frame': Tensor[int] of shape (S,) 'num_frames': Tensor[int] of shape (S,) # For audio input strategies 'start_sample': Tensor[int] of shape (S,) 'num_samples': Tensor[int] of shape (S,) # Optionally, when return_cuts=True 'cut': List[AnyCut] of len S } ] } Dimension symbols legend: * ``B`` - batch size (number of Cuts) * ``S`` - number of supervision segments (greater or equal to B, as each Cut may have multiple supervisions) * ``T`` - number of frames of the longest Cut * ``F`` - number of features The 'sequence_idx' field is the index of the Cut used to create the example in the Dataset. FN return_cutscut_transformsinput_transformsinput_strategycs>t||_t|g|_t|g|_||_tdd|_dS)a k2 ASR IterableDataset constructor. :param return_cuts: When ``True``, will additionally return a "cut" field in each batch with the Cut objects used to create that batch. :param cut_transforms: A list of transforms to be applied on each sampled batch, before converting cuts to an input representation (audio/features). Examples: cut concatenation, noise cuts mixing, etc. :param input_transforms: A list of transforms to be applied on each sampled batch, after the cuts are converted to audio/features. Examples: normalization, SpecAugment, etc. :param input_strategy: Converts cuts into a collated batch of audio/features. By default, reads pre-computed features from disk. d)reset_intervalN) super__init__rr rrrrhdf5_fix)selfrrrr __class__U/home/ubuntu/.local/lib/python3.10/site-packages/lhotse/dataset/speech_recognition.pyr=s   z#K2SpeechRecognitionDataset.__init__cutsreturncst||j|jdd}|jD]}||}q|jdd}||}t|dkr0|\}}}n|\}}|j|}tj t | dd}|j D]}|||d}qH|t ddt|Dd }|d ||jrtd d|D|d d <td d|D} | rggg} } } |dj|djdurz|jjjWn tytdw|D]5} | jD]/}| dd|jdD| fdd|jdD| fdd|jdDqq| |d d<| |d d<| |d d<|S)z Return a new batch, with the batch size automatically determined using the constraints of max_duration and max_cuts. F) ascending)dim)supervision_segmentscSs&g|]\}}|jD]}d|jiq qS)text) supervisionsr%).0 sequence_idxcut supervisionrrr sz:K2SpeechRecognitionDataset.__getitem__..)inputsr&r&cSsg|] }|jD]}|qqSr)r&)r'r)suprrrr+s r)css0|]}|jD] }|jduod|jvVqqdS)Nword)r& alignment)r'csrrr s z9K2SpeechRecognitionDataset.__getitem__..rNz[Can't determine the frame_shift -- it is not present either in cuts or the input_strategy. cSsg|]}|jqSr)symbolr'aliwordrrrr+sr.cg|] }t|jdqS frame_shift sampling_rate)r startr4r8rrr+cr6r7)r endr4r8rrr+r< word_startword_end)validate_for_asrrupdatesort_by_durationrrlensupervision_intervalstorchstacklistvaluesrr enumeraterallr9r: extractorAttributeError ValueErrorr&appendr/)rrtnfm input_tplr,_rDsegmentsbatchhas_word_alignmentswordsstartsendsr0r1rr8r __getitem__^sv                    z&K2SpeechRecognitionDataset.__getitem__)__name__ __module__ __qualname____doc__r boolrrr rETensorr rrstrrrX __classcell__rrrrr s 10!rrrcCsvt|d}|D]0}|jD]*}|j| ks"Jd|jd|jd|j|j|ks7Jd|jd|jdq qdS)NgMb`?zHSupervisions starting before the cut are not supported for ASR (sup id: z , cut id: )zESupervisions ending after the cut are not supported for ASR (sup id: )rr&r;idr=duration)rtolr)r*rrrr@s(  r@)typingrrrrrEtorch.utils.data.dataloaderrrlhotser lhotse.cutr lhotse.dataset.input_strategiesr r lhotse.utilsr r lhotse.workaroundsrutilsdataDatasetrr@rrrrs   @