o }oi; @s`ddlZddlmZmZddlmZmZddlmZm Z ddl Z ddl m Z ddl mZddlmZddlmZdd lmZdd lmZmZdd lmZdd lmZdd lmZmZdZzddlZWn e e!fyodZYnwGdddZ"Gddde"eeZ#GdddeeZ$Gddde$Z%GdddeeZ&Gddde"eeZ'GdddeeZ(dS)N)ABCabstractmethod) ExitStackcontextmanager)ListOptional) instantiate) DictConfig)tqdm) OperationMode)ModelPT)PretrainedModelInfo typecheck) AudioSignal) NeuralType)logging model_utilsTFc@seZdZdZddZdS)NeedsNormalizerz?Base class for all TTS models that needs text normalization(TN)cCsd|vr|j |_ dSdSdS)Ntext_normalizerzj`nemo_text_processing` not installed, see https://github.com/NVIDIA/NeMo-text-processing for more details.z The normalizer will be disabled. whitelistztext_normalizer.whitelisttext_normalizer_call_kwargs) PYNINI_AVAILABLErerrorrregister_artifactrr normalizer normalizetext_normalizer_callr)selfcfgnormalizer_kwargsr T/home/ubuntu/.local/lib/python3.10/site-packages/nemo/collections/tts/models/base.py_setup_normalizer*s$    z!NeedsNormalizer._setup_normalizerN)__name__ __module__ __qualname____doc__r"r r r r!r's rcsReZdZdZededdfddZeddd Zedd d Z fd dZ Z S)SpectrogramGeneratorz?Base class for all TTS models that turn text into a spectrogram str_inputreturn torch.tensorcKdS)a A helper function that accepts raw python strings and turns them into a tensor. The tensor should have 2 dimensions. The first is the batch, which should be of size 1. The second should represent time. The tensor should represent either tokenized or embedded text, depending on the model. Note that some models have `normalize` parameter in this function which will apply normalizer if it is available. Nr rr(kwargsr r r!parseBzSpectrogramGenerator.parsetokenscKr+)z Accepts a batch of text or text_tokens and returns a batch of spectrograms Args: tokens: A torch tensor representing the text to be generated Returns: spectrograms Nr rr0r-r r r!generate_spectrogramLr/z)SpectrogramGenerator.generate_spectrogramList[PretrainedModelInfo]cC<g}|D]}|}|durt|dkr||q|S This method returns a list of pre-trained model which can be instantiated directly from NVIDIA's NGC cloud. Returns: List of available pre-trained models. Nr__subclasses__list_available_modelslenextendclslist_of_modelssubclasssubclass_modelsr r r!r9X  z*SpectrogramGenerator.list_available_modelscsrdD]}||vrt|||j|<||qd|vr)t|d|jd<|dd|vr1tdt|dS)N) enable_volumeenable_ragged_batches num_speakers emb_rangez$embedding range is not user-settable)bool export_configpopint Exceptionsuperset_export_config)rargsk __class__r r!rLfs  z&SpectrogramGenerator.set_export_config)r0r*r)r*r)r3) r#r$r%r&rstrr.r2 classmethodr9rL __classcell__r r rOr!r'?s   r'c@,eZdZdZed ddZed dd Zd S) Vocoderz A base class for models that convert spectrograms to audios. Note that this class takes as input either linear or mel spectrograms. specr*r)cKr+)z Accepts a batch of spectrograms and returns a batch of audio. Args: spec: ['B', 'n_freqs', 'T'], A torch tensor representing the spectrograms to be vocoded. Returns: audio Nr )rrWr-r r r!convert_spectrogram_to_audioyr/z$Vocoder.convert_spectrogram_to_audior3cCr4r5r7r<r r r!r9rAzVocoder.list_available_modelsN)rWr*r)r*rQ)r#r$r%r&rrXrSr9r r r r!rVss   rVcseZdZdZfddZeddZeddZedd Z d d Z d d Z e e dee dddde deidddddefddZZS) GlowVocoderzBase class for all Vocoders that use a Glow or reversible Flow-based setup. All child class are expected to have a parameter called audio_to_melspec_precessor that is an instance of nemo.collections.asr.parts.FilterbankFeaturescs6tj|i|tj|_d|_d|_d|_d|_dSN) rK__init__r infer_modestftistftn_mel bias_spect)rrMr-rOr r!r[s  zGlowVocoder.__init__cCs|jSrZ)r])rr r r!modeszGlowVocoder.modeccs*|j}||_z dVW||_dS||_wrZ)rb)rrbold_moder r r! temp_modes zGlowVocoder.temp_modeccsTt}||tj|tdVWddS1s#wYdSrZ)r enter_contextrdr r\torchno_grad)rstackr r r! nemo_infers "zGlowVocoder.nemo_inferc s|jdurIz|jj|jj|jj|jj|jWnty/}zt|d|d}~wwddfdd|_fdd|_ |j durkz|jj |_ WdStyj}zt|d|d}~wwdS)Nz could not find a valid audio_to_melspec_precessor. GlowVocoder requires child class to have audio_to_melspec_precessor defined to obtain stft parameters. audio_to_melspec_precessor requires n_fft, hop_length, win_length, window, and nfilt to be defined.cSsJtj|||||dd}t|}t|ddt|d|dfS)NT)n_fft hop_length win_lengthwindowreturn_complex).rp.r)rfr^ view_as_realsqrtpowsumatan2)audiorjrkrlrmrWr r r!yet_another_patchs *z@GlowVocoder.check_children_attributes..yet_another_patchcs|dSN)rjrkrlrmr )xrkrjrlrmrxr r!sz7GlowVocoder.check_children_attributes..cs0tjt|t||t|dSry)rfr_complexcossin)rzy)rkrjrlrmr r!r|sz could not find a valid audio_to_melspec_precessor. GlowVocoder requires child class to have audio_to_melspec_precessor defined to obtain stft parameters. audio_to_melspec_precessor requires nfilt to be defined.) r^audio_to_melspec_precessorrjrkrlrmtodeviceAttributeErrorr_r`nfilt)rer r{r!check_children_attributess:   z%GlowVocoder.check_children_attributescCs|||,td|jdf|j}|j|ddd}||\}}|dd|_ WddS1s7wYdS)NXF)rWsigmadenoiserq).N) rrirfzerosr`rrrXr^ra)rspect bias_audiora_r r r!update_bias_spects "zGlowVocoder.update_bias_spect)BTT)optional)rwstrengthrw) input_types output_types{Gz?r*rcCsZ||jdur |||\}}||j|j|}t|d}|||}|S)Nr) rrarr^rrrfclampr_)rrwr audio_spect audio_anglesaudio_spect_denoisedaudio_denoisedr r r!rs   zGlowVocoder.denoise)r)r#r$r%r&r[propertyrbrrdrirrrrrfloatrrTr r rOr!rYs     4 rYc@rU) MelToSpeczb A base class for models that convert mel spectrograms to linear (magnitude) spectrograms melr*r)cKr+)a3 Accepts a batch of spectrograms and returns a batch of linear spectrograms Args: mel: A torch tensor representing the mel spectrograms ['B', 'mel_freqs', 'T'] Returns: spec: A torch tensor representing the linear spectrograms ['B', 'n_freqs', 'T'] Nr )rrr-r r r!!convert_mel_spectrogram_to_linearr/z+MelToSpec.convert_mel_spectrogram_to_linearr3cCr4r5r7r<r r r!r9rAzMelToSpec.list_available_modelsN)rr*r)r*rQ)r#r$r%r&rrrSr9r r r r!rs   rc@sBeZdZdZededdfddZedd d Zedd d Z dS)TextToWaveformzKBase class for all end-to-end TTS models that generate a waveform from textr(r)r*cKr+)aC A helper function that accepts a raw python string and turns it into a tensor. The tensor should have 2 dimensions. The first is the batch, which should be of size 1. The second should represent time. The tensor should represent either tokenized or embedded text, depending on the model. Nr r,r r r!r.%r/zTextToWaveform.parser0List[torch.tensor]cKr+)a3 Accepts a batch of text and returns a list containing a batch of audio Args: tokens: A torch tensor representing the text to be converted to speech Returns: audio: A list of length batch_size containing torch tensors representing the waveform output Nr r1r r r!convert_text_to_waveform-r/z'TextToWaveform.convert_text_to_waveformr3cCr4r5r7r<r r r!r97rAz$TextToWaveform.list_available_modelsN)r0r*r)rrQ) r#r$r%r&rrRr.rrSr9r r r r!r"s  rc@sZeZdZe    ddedededed ed eed eefd d Z e dddZ dS)G2PModeltext_graphemes r pred_textmanifest_filepathoutput_manifest_filepathgrapheme_field batch_size num_workers pred_fieldr)c Cs||dd||d}|t|}t|dA} t|ddd*} tt| D]\} } t| } || | |<| tj| dddq$Wd n1sJwYWd n1sYwYt d |d |S) a Main function for Inference. Converts grapheme entries from the manifest "graheme_field" to phonemes Args: manifest_filepath: Path to .json manifest file output_manifest_filepath: Path to .json manifest file to save predictions, will be saved in "target_field" grapheme_field: name of the field in manifest_filepath for input grapheme text pred_field: name of the field in the output_file to save predictions batch_size: int = 32 # Batch size to use for inference num_workers: int = 0 # Number of workers to use for DataLoader during inference Returns: Predictions generated by the model F)rr drop_lastshufflerrrwzutf-8)encoding) ensure_ascii NzPredictions saved to .) _inferr openr enumeratejsonloadswritedumpsrinfo) rrrrrrrconfig all_predsf_inf_outiliner r r!convert_graphemes_to_phonemesGs(   z&G2PModel.convert_graphemes_to_phonemesr3cCst|}|S)r6)r&resolve_subclass_pretrained_model_info)r=r>r r r!r9qs zG2PModel.list_available_modelsN)rrrrrQ) r#r$r%rfrgrRrIrrrrSr9r r r r!rFs. )r))rabcrr contextlibrrtypingrrrf hydra.utilsr omegaconfr r (nemo.collections.tts.parts.utils.helpersr nemo.core.classesr nemo.core.classes.commonr rnemo.core.neural_types.elementsr"nemo.core.neural_types.neural_typer nemo.utilsrrrnemo_text_processing ImportErrorModuleNotFoundErrorrr'rVrYrrrr r r r!s6        4!n $