o Sij @sddlZddlmZmZmZmZddlZddlZddl m Z ddl m Z m Z mZmZddlmZddlmZddlmZddlmZdd lmZmZmZdd lmZdd lmZGd d d ej j!j"Z#Gddde#Z$Gddde#Z%Gddde Z&deefddZ'ddZ(dS)N)AnyDictListOptional)IterableDataset) RecordingSetSecondscompute_num_samplesvalidate)torchaudio_supports_ffmpeg)suppress_audio_loading_errors) AugmentFn)CutSet) collate_audiocollate_featurescollate_matrices)FeatureExtractor)_get_strided_batch_streamingcsNeZdZdZd fdd ZdedeeeffddZ deddfd d Z Z S) UnsupervisedDatasetz Dataset that contains no supervision - it only provides the features extracted from recordings. .. code-block:: { 'features': (B x T x F) tensor 'features_lens': (B, ) tensor } returnNcstdSN)super__init__self __class__O/home/ubuntu/.local/lib/python3.10/site-packages/lhotse/dataset/unsupervised.pyrszUnsupervisedDataset.__init__cutscCs"||t|\}}|||dS)N)rfeatures features_lens) _validater)rrr r!rrr __getitem__!s  zUnsupervisedDataset.__getitem__cC"t|tdd|DsJdS)Ncs|]}|jVqdSr) has_features.0cutrrr ,z0UnsupervisedDataset._validate..r allrrrrrr"*zUnsupervisedDataset._validaterN) __name__ __module__ __qualname____doc__rrrstrrr#r" __classcell__rrrrrs   rcsXeZdZdZd deddffdd Zdedeee ffd d Z deddfd d Z Z S)UnsupervisedWaveformDataseta A variant of UnsupervisedDataset that provides waveform samples instead of features. The output is a tensor of shape (C, T), with C being the number of channels and T the number of audio samples. In this implementation, there will always be a single channel. Returns: .. code-block:: { 'audio': (B x NumSamples) float tensor 'audio_lens': (B, ) int tensor } TcollaterNcst||_dSr)rrr8)rr8rrrr?s  z$UnsupervisedWaveformDataset.__init__rc Cs|||jrt|\}}|||dSg}g}|D]!}t||||Wdn1s6wYqt||dS)N)raudio audio_lens)rr9)r"r8rr append load_audior from_cuts)rrr9r: remain_cuts remain_audioscrrrr#Cs   z'UnsupervisedWaveformDataset.__getitem__cCr$)Ncsr%r has_recordingr'rrrr*Xr+z8UnsupervisedWaveformDataset._validate..r,r.rrrr"Vr/z%UnsupervisedWaveformDataset._validate)T) r1r2r3r4boolrrrr5rr#r"r6rrrrr7/s r7csXeZdZdZ d dedeeffdd Zdede j fd d Z deddfd d Z Z S)DynamicUnsupervisedDataseta( An example dataset that shows how to use on-the-fly feature extraction in Lhotse. It accepts two additional inputs - a FeatureExtractor and an optional WavAugmenter for time-domain data augmentation.. The output is approximately the same as that of the ``UnsupervisedDataset`` - there might be slight differences for ``MixedCut``s, because this dataset mixes them in the time domain, and ``UnsupervisedDataset`` does that in the feature domain. Cuts that are not mixed will yield identical results in both dataset classes. Nfeature_extractor augment_fncst||_||_dSr)rrrErF)rrErFrrrres  z#DynamicUnsupervisedDataset.__init__rrcs,|dtffdd }t||}|S)Nrc 3sJ|D]}t|jjjdVWdn1swYqdS)N) extractorrF)r compute_featuresrErF)rr)rrr generate_cutqs z.generate_cut)r"rr)rrrIr rrrr#ns  z&DynamicUnsupervisedDataset.__getitem__cCr$)Ncsr%rrAr'rrrr*~r+z7DynamicUnsupervisedDataset._validate..r,r.rrrr"|r/z$DynamicUnsupervisedDataset._validater)r1r2r3r4rrr rrtorchTensorr#r"r6rrrrrD[s  rDc@s<eZdZdZdedededdfddZd d d Zd d ZdS)RecordingChunkIterableDataseta  This dataset iterates over chunks of a recording, for each recording provided. It supports setting a chunk_shift < chunk_size to run model predictions on overlapping audio chunks. The format of yielded items is the following:: { "recording_id": str "begin_time": tensor with dtype=float32 shape=(1,) "end_time": tensor with dtype=float32 shape=(1,) "audio": tensor with dtype=float32 shape=(chunk_size_in_samples,) } Unlike most other datasets in Lhotse, this dataset does not yield batched items, and should be used like the following:: >>> recordings = RecordingSet.from_file("my-recordings.jsonl.gz") ... dataset = RecordingChunkIterableDataset(recordings, chunk_size=30.0, chunk_shift=25.0) ... dloader = torch.utils.data.DataLoader( ... dataset, ... batch_size=32, ... collate_fn=audio_chunk_collate, ... worker_init_fn=audio_chunk_worker_init_fn, ... ) recordings chunk_size chunk_shiftrNcCs4t||_||_||_d|_t|j|_|dS)Nr)listrMrNrOstartlenendr )rrMrNrOrrrrs   z&RecordingChunkIterableDataset.__init__cCs|jD]B}t|jdksJdt|jd|jd|jdjdks3Jd|jdjd|jd|jdksEJd |jd |jdqdS) NzCWe currently don't support multi-source audio in this dataset (got z sources in recording z).rfilezHWe currently only support 'file' AudioSource type in this dataset (got: z in recording zDWe currently only support single-channel audio in this dataset (got z channels in recording )rMrRsourcesidtype num_channels)rrrrrr s  z&RecordingChunkIterableDataset.validatec csddl}|j|j|jD]b}t|j|j}t|j|j|j}d}|j}||j dj d8}|j ||t j dD]&}t|}|jtj|tj dtj|tj d|dV||j7}||j}q:Wdn1skwYqdS)Nrrb)overlapdtype)r]) recording_id begin_timeend_timer9) soundfilerMrQrSr rN sampling_raterO SoundFilerVsourceblocksnpfloat32rJ as_tensorrW) rsfrZrN chunk_overlapr_r`streamchunkrrr__iter__s2   z&RecordingChunkIterableDataset.__iter__r0) r1r2r3r4rrrr rmrrrrrLs rLbatchcCsddlm}dd|D}||}tdd|D}tt||f}t|D]\}}t|||d|jdf<q'||d<|S)Nr)default_collatecSsg|]}|dqS)r9)pop)r(drrr sz'audio_chunk_collate..css|]}|jdVqdS)rN)shape)r(arrrr*sz&audio_chunk_collate..r9) torch.utils.dataromaxrJzerosrR enumeraterhrs)rnroaudiosoutmaxlenr9irtrrraudio_chunk_collates r}cCsdtjj}|j}|j}|j}tt ||t |j }|j }||||_t |j|||_dSr)rJutilsdataget_worker_infodatasetrQrSintmathceilfloat num_workersrWmin) worker_id worker_infor overall_start overall_end per_workerrrraudio_chunk_worker_init_fns r))rtypingrrrrnumpyrfrJrurlhotserrr r lhotse.audio.backendr lhotse.audio.utilsr lhotse.augmentationr lhotse.cutrlhotse.dataset.collationrrrlhotse.featuresrlhotse.features.kaldi.layersrr~rDatasetrr7rDrLr}rrrrrs&       ,&N