o %ݫiG2@sldZddlZddlmZddlmZddlmZeeZ GdddeZ Gdd d eZ Gd d d eZ dS) aR Specifies the inference interfaces for Text-To-Speech (TTS) modules. Authors: * Aku Rouhe 2021 * Peter Plantinga 2021 * Loren Lugosch 2020 * Mirco Ravanelli 2020 * Titouan Parcollet 2021 * Abdel Heba 2021 * Andreas Nautsch 2022, 2023 * Pooneh Mousavi 2023 * Sylvain de Langen 2023 * Adel Moumen 2023 * Pradnya Kandarkar 2023 N)length_to_mask) Pretrained) get_loggercsHeZdZdZdgZfddZdddZdd Zd d Zd d Z Z S)HIFIGANa A ready-to-use wrapper for HiFiGAN (mel_spec -> waveform). Arguments --------- *args : tuple **kwargs : dict Arguments are forwarded to ``Pretrained`` parent class. Example ------- >>> tmpdir_vocoder = getfixture('tmpdir') / "vocoder" >>> hifi_gan = HIFIGAN.from_hparams(source="speechbrain/tts-hifigan-ljspeech", savedir=tmpdir_vocoder) >>> mel_specs = torch.rand(2, 80,298) >>> waveforms = hifi_gan.decode_batch(mel_specs) >>> # You can use the vocoder coupled with a TTS system >>> # Initialize TTS (tacotron2) >>> tmpdir_tts = getfixture('tmpdir') / "tts" >>> from speechbrain.inference.TTS import Tacotron2 >>> tacotron2 = Tacotron2.from_hparams(source="speechbrain/tts-tacotron2-ljspeech", savedir=tmpdir_tts) >>> # Running the TTS >>> mel_output, mel_length, alignment = tacotron2.encode_text("Mary had a little lamb") >>> # Running Vocoder (spectrogram-to-waveform) >>> waveforms = hifi_gan.decode_batch(mel_output) generatorcs(tj|i||jjj|_d|_dSNT)super__init__hparamsr inferenceinfer first_callselfargskwargs __class__R/home/ubuntu/.local/lib/python3.10/site-packages/speechbrain/inference/vocoders.pyr 7s  zHIFIGAN.__init__NcCst|jr |jjd|_t|||j}Wdn1s$wY|dur8|dur8| |||}|S)a^Computes waveforms from a batch of mel-spectrograms Arguments --------- spectrogram: torch.Tensor Batch of mel-spectrograms [batch, mels, time] mel_lens: torch.tensor A list of lengths of mel-spectrograms for the batch Can be obtained from the output of Tacotron/FastSpeech hop_len: int hop length used for mel-spectrogram extraction should be the same value as in the .yaml file Returns ------- waveforms: torch.Tensor Batch of mel-waveforms [batch, 1, time] FN) r r rremove_weight_normtorchno_gradr todevice mask_noise)r spectrogrammel_lenshop_lenwaveformrrr decode_batch<s  zHIFIGAN.decode_batchcC@|d}t|||jd|jd}||d|dSaaMask the noise caused by padding during batch inference Arguments --------- waveform: torch.tensor Batch of generated waveforms [batch, 1, time] mel_lens: torch.tensor A list of lengths of mel-spectrograms for the batch Can be obtained from the output of Tacotron/FastSpeech hop_len: int hop length used for mel-spectrogram extraction same value as in the .yaml file Returns ------- waveform: torch.tensor Batch of waveforms without padded noise [batch, 1, time] )rgsqueezershaperbool masked_fill_ unsqueezerrrrmaskrrrr\  zHIFIGAN.mask_noisecCsb|jr |jjd|_t||d|j }Wdn1s'wY| dS)aComputes waveforms from a single mel-spectrogram Arguments --------- spectrogram: torch.Tensor mel-spectrogram [mels, time] Returns ------- waveform: torch.Tensor waveform [1, time] audio can be saved by: >>> import torchaudio >>> waveform = torch.rand(1, 666666) >>> sample_rate = 22050 >>> torchaudio.save(str(getfixture('tmpdir') / "test.wav"), waveform, sample_rate) FrN) r r rrrrr r)rrr%)rrrrrrdecode_spectrogramws   zHIFIGAN.decode_spectrogramcC ||SzDecodes the input spectrogramsr rrrrrforward zHIFIGAN.forward)NN __name__ __module__ __qualname____doc__HPARAMS_NEEDEDr r rr-r2 __classcell__rrrrrs   rcsTeZdZdZdgZfddZ   dddZd d Z  dd d Zd dZ Z S)DiffWaveVocodera7 A ready-to-use inference wrapper for DiffWave as vocoder. The wrapper allows to perform generative tasks: locally-conditional generation: mel_spec -> waveform Arguments --------- *args : tuple **kwargs : dict Arguments are forwarded to ``Pretrained`` parent class. diffusioncs2tj|i|t|jdr|jjj|_dSt)Ndiffwave)rr hasattrr r<r r NotImplementedErrorrrrrr s zDiffWaveVocoder.__init__NFcCsft|jd|||j||d}Wdn1swY|dur1|dur1||||}|S)aGenerate waveforms from spectrograms Arguments --------- mel: torch.tensor spectrogram [batch, mels, time] hop_len: int Hop length during mel-spectrogram extraction Should be the same value as in the .yaml file Used to determine the output wave length Also used to mask the noise for vocoding task mel_lens: torch.tensor Used to mask the noise caused by padding A list of lengths of mel-spectrograms for the batch Can be obtained from the output of Tacotron/FastSpeech fast_sampling: bool whether to do fast sampling fast_sampling_noise_schedule: list the noise schedules used for fast sampling Returns ------- waveforms: torch.tensor Batch of mel-waveforms [batch, 1, time] F unconditionalscale condition fast_samplingfast_sampling_noise_scheduleN)rrr rrr)rmelrrrDrErrrrr s !  zDiffWaveVocoder.decode_batchcCr!r"r$r*rrrrr,zDiffWaveVocoder.mask_noisecCsTt|jd||d|j||d}Wdn1s wY|dS)aComputes waveforms from a single mel-spectrogram Arguments --------- spectrogram: torch.tensor mel-spectrogram [mels, time] hop_len: int hop length used for mel-spectrogram extraction same value as in the .yaml file fast_sampling: bool whether to do fast sampling fast_sampling_noise_schedule: list the noise schedules used for fast sampling Returns ------- waveform: torch.tensor waveform [1, time] audio can be saved by: >>> import torchaudio >>> waveform = torch.rand(1, 666666) >>> sample_rate = 22050 >>> torchaudio.save(str(getfixture('tmpdir') / "test.wav"), waveform, sample_rate) Frr@N)rrr r)rrr%)rrrrDrErrrrr-s  z"DiffWaveVocoder.decode_spectrogramcCr.r/r0r1rrrr2r3zDiffWaveVocoder.forward)NFN)FNr4rrrrr;s   / *r;csDeZdZdZdgZfddZd ddZd dd Zd d d ZZ S) UnitHIFIGANa A ready-to-use wrapper for Unit HiFiGAN (discrete units -> waveform). Arguments --------- *args : tuple See `Pretrained` **kwargs : dict See `Pretrained` Example ------- >>> tmpdir_vocoder = getfixture('tmpdir') / "vocoder" >>> hifi_gan = UnitHIFIGAN.from_hparams(source="speechbrain/hifigan-hubert-l1-3-7-12-18-23-k1000-LibriTTS", savedir=tmpdir_vocoder) >>> codes = torch.randint(0, 99, (100, 1)) >>> waveform = hifi_gan.decode_unit(codes) rcs.tj|i||jjj|_d|_d|_dSr)rr r rr r r tokenizerrrrr 9s  zUnitHIFIGAN.__init__NcCs|jr |jjd|_|ddkrtd|jr|d7}|dur(||j}t |j ||j|d}Wd|S1sCwY|S)avComputes waveforms from a batch of discrete units Arguments --------- units: torch.tensor Batch of discrete units [batch, codes] spk: torch.tensor Batch of speaker embeddings [batch, spk_dim] Returns ------- waveforms: torch.tensor Batch of mel-waveforms [batch, 1, time] Fr#zPThe 'units' argument should have a length of at least 3 because of padding size.Nspk) r r rrsize ValueErrorrHrrrrr runitsrKrrrrr @s"    zUnitHIFIGAN.decode_batchcCs|jr |jjd|_|ddkrtd|jr|d}|dur+|d|j }t |j |d|j |d}Wdn1sHwY| dS)a1Computes waveforms from a single sequence of discrete units Arguments --------- units: torch.tensor codes: [time] spk: torch.tensor spk: [spk_dim] Returns ------- waveform: torch.tensor waveform [1, time] FrzPThe 'units' argument should have a length of at least 4 because of padding size.r#NrJ)r r rrrLrMrHr)rrrrr r%rNrrr decode_unitcs   zUnitHIFIGAN.decode_unitcCs|j||dS)zDecodes the input unitsrJr0)rrOrKrrrr2szUnitHIFIGAN.forward)N) r5r6r7r8r9r r rQr2r:rrrrrG$s   #!rG) r8rspeechbrain.dataio.dataior speechbrain.inference.interfacesrspeechbrain.utils.loggerrr5loggerrr;rGrrrrs   {