o wi^E@sxddlZddlZddlmZddlmZddlmZmZm Z ddl Z ddl Z ddl ZddlmZddlmZddlmZddlmZdd lmZmZmZmZmZdd lmZm Z dd l!m"Z"dd l!m#Z$dd l%m&Z&ddl'm(Z(d)gddde*DZ+eGdddZ,eGdddZ-dee.fddZ/de0de,de1de1fddZ2e&Gd d!d!eZ3Gd"d#d#e Z4dS)$N) dataclass)Path)DictListOptional)expand_sharded_filepaths)available_formats) read_manifest)FeatureProcessor)filter_dataset_by_durationget_weighted_sampler load_audio sample_audio stack_tensors)DatasetIterableDataset)logging) webdataset) experimental)webdataset_split_by_workers;)wavmp3flacopuscCsg|]}|qS)lower).0fmtrrf/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/nemo/collections/tts/data/vocoder_dataset.py *sr c@s>eZdZUeed<eed<dZeed<dZee e ed<dS) DatasetMeta manifest_path audio_dirg? sample_weightNaudio_tar_filepaths) __name__ __module__ __qualname__r__annotations__r$floatr%rrstrrrrrr!-s  r!c@s&eZdZUeed<eed<eed<dS) DatasetSample dataset_namemanifest_entryr#N)r&r'r(r+r)dictrrrrrr,5s  r,batchc Csg}g}g}g}|D]}||d||d||d||dq t|}t|}t||gd}||||d} | S)Nr-audio_filepathaudio audio_len)max_lens) dataset_namesaudio_filepathsr2 audio_lens)appendtorch IntTensorintmaxitemr) r0dataset_name_listaudio_filepath_list audio_listaudio_len_listexamplebatch_audio_len audio_max_len batch_audio batch_dictrrraudio_collate_fn<s$ rGr-dataset min_duration max_durationc Cst|j}t|||d\}}}t|tdt|tdt|td|ddtd|ddg}g} |D]} t|| t|jd} | | | |j qB|| fS) N)entriesrIrJzOriginal # of files: zFiltered # of files: zOriginal duration: z.2fz hourszFiltered duration: )r-r.r#) r r"r rinfolenr,rr#r8r$) r-rHrIrJrKfiltered_entries total_hoursfiltered_hourssamplessample_weightsentrysamplerrrpreprocess_manifestWs   rUcseZdZdZ       ddededeedeedeeeefd ee d ee d ee d e ffd d Z dededee j jjfddZddZddZddZZS)VocoderDataseta Class for processing and loading Vocoder training examples. Args: dataset_meta: Dict of dataset names (string) to dataset metadata. sample_rate: Sample rate to load audio as. If the audio is stored at a different sample rate, then it will be resampled. n_samples: Optional int, if provided then n_samples samples will be randomly sampled from the full audio file. weighted_sampling_steps_per_epoch: Optional int, If provided, then data will be sampled (with replacement) based on the sample weights provided in the dataset metadata. If None, then sample weights will be ignored. feature_processors: Optional, list of feature processors to run on training examples. min_duration: Optional float, if provided audio files in the training manifest shorter than 'min_duration' will be ignored. max_duration: Optional float, if provided audio files in the training manifest longer than 'max_duration' will be ignored. trunc_duration: Optional int, if provided audio will be truncated to at most 'trunc_duration' seconds. volume_norm: Whether to apply volume normalization to loaded audio. NF dataset_meta sample_rate n_samples!weighted_sampling_steps_per_epochfeature_processorsrIrJtrunc_duration volume_normc st||_||_||_| |_||_d|_|r+t d| t | |_ ng|_ g|_g|_|D]#\} } tdi| } t| | ||d\} }|j| 7_|j|7_q8dS)NFFound feature processors r-rHrIrJr)super__init__rXrYr\r]rZload_precomputed_melrrLkeyslistvaluesr[ data_samplesrRitemsr!rU)selfrWrXrYrZr[rIrJr\r]r- dataset_inforHrQweights __class__rrras0  zVocoderDataset.__init__ batch_size world_sizereturncCs"|jsdSt|j|||jd}|S)N)rRrmrn num_steps)rZr rR)rhrmrnsamplerrrr get_samplerszVocoderDataset.get_samplercC t|jSN)rMrfrhrrr__len__ zVocoderDataset.__len__c Cs|j|}|jrt|j|j|j|j|jd\}}}nt|j|j|j|j|jd\}}}t j |t j d}|j d}|j |||d}|jD]} | |qC|S)N)r.r#rXrYr])r.r#rXrJr])dtyperr-r1r2r3)rfrYrr.r#rXr]r r\r9tensorfloat32shaper-r[process) rhindexdata audio_array_audio_filepath_relr2r3rB processorrrr __getitem__s4     zVocoderDataset.__getitem__cCt|SrtrGrhr0rrr collate_fnzVocoderDataset.collate_fn)NNNNNNF)r&r'r(__doc__rr;rr+r r*boolrar9utilsrSamplerrrrvrr __classcell__rrrkrrVrs@   ( "rVcseZdZdZ         d!deded eed ed ed eed eedeeee fdededeffdd Z ddZ ddZ dedefddZ ddZddZdd ZZS)"TarredVocoderDatasetaH A similar Dataset to the VocoderDataset, but loads tarred audio files. Accepts a single comma-separated JSON manifest file (in the same style as for the VocoderDataset), as well as the path(s) to the tarball(s) containing the wav files. Each line of the manifest should contain the information for one audio file, and duration of audio. Valid formats for the audio_tar_filepaths argument include: (1) a single string that can be brace-expanded, e.g. 'path/to/audio.tar' or 'path/to/audio_{1..100}.tar.gz', or (2) a list of file paths that will not be brace-expanded, e.g. ['audio_1.tar', 'audio_2.tar', ...]. See the WebDataset documentation for more information about accepted data and input formats. If using multiple processes the number of shards should be divisible by the number of workers to ensure an even split among workers. If it is not divisible, logging will give a warning but training will proceed. In addition, if using mutiprocessing, each shard MUST HAVE THE SAME NUMBER OF ENTRIES after filtering is applied. We currently do not check for this, but your program may hang if the shards are uneven! Additionally, please note that the len() of this DataLayer is assumed to be the length of the manifest after filtering. An incorrect manifest length may lead to some DataLoader issues down the line. Args: dataset_meta: Dict of dataset names (string) to dataset metadata. audio_tar_filepaths: Either a list of audio tarball filepaths, or a string (can be brace-expandable). sample_rate: Sample rate to load audio as. If the audio is stored at a different sample rate, then it will be resampled. n_samples: Optional int, if provided then n_samples samples will be randomly sampled from the full audio file. shuffle_n (int): How many samples to look ahead and load to be shuffled. See WebDataset documentation for more details. Defaults to 0. min_duration: Optional float, if provided audio files in the training manifest shorter than 'min_duration' will be ignored. max_duration: Optional float, if provided audio files in the training manifest longer than 'max_duration' will be ignored. trunc_duration: Optional int, if provided audio will be truncated to at most 'trunc_duration' seconds. feature_processors: Optional, list of feature processors to run on training examples. shard_strategy (str): Tarred dataset shard distribution strategy chosen as a str value during ddp. - `scatter`: The default shard strategy applied by WebDataset, where each node gets a unique set of shards, which are permanently pre-allocated and never changed at runtime. - `replicate`: Optional shard strategy, where each node gets all of the set of shards available in the tarred dataset, which are permanently pre-allocated and never changed at runtime. The benefit of replication is that it allows each node to sample data points from the entire dataset independently of other nodes, and reduces dependence on value of `shuffle_n`. .. warning:: Replicated strategy allows every node to sample the entire set of available tarfiles, and therefore more than one node may sample the same tarfile, and even sample the same data points! As such, there is no assured guarantee that all samples in the dataset will be sampled at least once during 1 epoch. Scattered strategy, on the other hand, on specific occasions (when the number of shards is not divisible with ``world_size``), will not sample the entire dataset. For these reasons it is not advisable to use tarred datasets as validation or test datasets. global_rank (int): Worker rank, used for partitioning shards. Defaults to 0. world_size (int): Total number of processes, used for partitioning shards. Defaults to 0. Nr皙?scatterrWrXrY shuffle_nrIrJr\r[shard_strategy global_rankrnc  stt| dkrtd| d||_||_|r't||j|_ nd|_ |r>t d|t | |_ ng|_ g|_g|_|D]'\} }|j}|j|g7_tdi|}t| |||d\}}|j|7_qKi|_|jD](}tjtj|jdd}||jvr||j|<qytd|d|jdt d | t|| | | d }ttj|d tt|t tj!t"d d t#dd|j$t%|j&|_'dS)Nrz Arguments zA does not support for TarredVocoderDataset, they will be ignored.r^r_r1zDuplicate file_id z found in manifest z world size: )sharded_filepathsrrnr)urls__key__)r2keyr2rr)(r`rarMrwarningrcrXrYr; trunc_samplesrLrdrer[rfr%rgr!rUfile_id_to_sample_mapospathsplitextbasenamer. ValueErrorrwds DataPipelineSimpleShardListrshuffletarfile_to_samplesrenameVALID_FILE_FORMATSto_tuple_filtermap _build_sample_dataset)rhrWrXrYrrIrJr\r[rrrnkwargsr-rir%rHrQrrTfile_idrkrrra sh           zTarredVocoderDataset.__init__csGfddd}||jS)Ncs(eZdZfddZddZddZdS)z6TarredVocoderDataset._filter..FilteredIteratorcs|_||_dSrt)iteratorr)rhrrrrraqs z?TarredVocoderDataset._filter..FilteredIterator.__init__cSs|Srtrrurrr__iter__usz?TarredVocoderDataset._filter..FilteredIterator.__iter__cSs< t|j\}}tjtj|d}||jvr||fSq)NTr)nextrrrrrr)rh audio_bytesaudio_filenamerrrr__next__xs  z?TarredVocoderDataset._filter..FilteredIterator.__next__N)r&r'r(rarrrrrrFilteredIteratorps  r)r)rhrrrrrros zTarredVocoderDataset._filterc Cs"|\}}tjtj|d}|j|}tjt|dd\}}||j kr=t d|d|j dt j j|||j d}t|}|jrj|jd}||jkrctd||jd} || | |j}n|d|j}|jrt|d|j}t|jd} |j||| d } |jD]} | | q| S) Nrr{)filerxzSample rate of z& does not match target sample rate of z. Resampling audio.)orig_sr target_sr)ry)rrrrrsfreadioBytesIOrXrrlibrosacoreresampler9 from_numpyrYr|randintrrzr-r[r}) rhtuprrrrrsr len_audiostartr3rBrrrrrs6       z"TarredVocoderDataset._build_samplermcCsdS)zH Currently sampler is not supported for tarred dataset. Nr)rhrmrnrrrrrsz TarredVocoderDataset.get_samplercCrrtrrrrrrrzTarredVocoderDataset.collate_fncCs |jSrt)rrrurrrrrwzTarredVocoderDataset.__iter__cCrsrt)rMrrurrrrvrwzTarredVocoderDataset.__len__) NrrNNNrrr)r&r'r(rrr;rr*r+r rarrrrrrrvrrrrkrrsP>    O&r)5rr dataclassesrpathlibrtypingrrrr soundfilertorch.utils.datar9'nemo.collections.asr.data.audio_to_textr0nemo.collections.asr.parts.preprocessing.segmentrvalid_sf_formats/nemo.collections.asr.parts.utils.manifest_utilsr ;nemo.collections.tts.parts.preprocessing.feature_processorsr 2nemo.collections.tts.parts.utils.tts_dataset_utilsr r r rrnemo.core.classesrr nemo.utilsrrrnemo.utils.decoratorsrnemo.utils.distributedrjoinrcrr!r,r/rGr+r*rUrVrrrrrsH            r