o %ݫi! @s<dZddlZddlmZGdddeZGdddeZdS)aO Specifies the inference interfaces for speech and audio encoders. Authors: * Aku Rouhe 2021 * Peter Plantinga 2021 * Loren Lugosch 2020 * Mirco Ravanelli 2020 * Titouan Parcollet 2021 * Abdel Heba 2021 * Andreas Nautsch 2022, 2023 * Pooneh Mousavi 2023 * Sylvain de Langen 2023 * Adel Moumen 2023 * Pradnya Kandarkar 2023 N) Pretrainedc@s.eZdZdZdgZddZddZddZd S) WaveformEncoderaVA ready-to-use waveformEncoder model It can be used to wrap different embedding models such as SSL ones (wav2vec2) or speaker ones (Xvector) etc. Two functions are available: encode_batch and encode_file. They can be used to obtain the embeddings directly from an audio file or from a batch of audio tensors respectively. The given YAML must contain the fields specified in the *_NEEDED[] lists. Arguments --------- See ``Pretrained`` Example ------- >>> from speechbrain.inference.encoders import WaveformEncoder >>> tmpdir = getfixture("tmpdir") >>> ssl_model = WaveformEncoder.from_hparams( ... source="speechbrain/ssl-wav2vec2-base-libri", ... savedir=tmpdir, ... ) # doctest: +SKIP >>> ssl_model.encode_file("samples/audio_samples/example_fr.wav") # doctest: +SKIP encodercKs<|j|fi|}|d}tdg}|||}|dS)ahEncode the given audiofile into a sequence of embeddings. Arguments --------- path : str Path to audio file which to encode. **kwargs : dict Arguments forwarded to ``load_audio`` Returns ------- torch.Tensor The audiofile embeddings produced by this system. r? embeddings) load_audio unsqueezetorchtensor encode_batch)selfpathkwargswaveformbatch rel_lengthresultsrR/home/ubuntu/.local/lib/python3.10/site-packages/speechbrain/inference/encoders.py encode_file1s    zWaveformEncoder.encode_filecCs4|}||j||j}}|j||}|S)aEncodes the input audio into a sequence of hidden states The waveforms should already be in the model's desired format. Arguments --------- wavs : torch.Tensor Batch of waveforms [batch, time, channels] or [batch, time] depending on the model. wav_lens : torch.Tensor Lengths of the waveforms relative to the longest one in the batch, tensor of shape [batch]. The longest one should have relative length 1.0 and others len(waveform) / max_length. Used for ignoring padding. Returns ------- torch.Tensor The encoded batch )floattodevicemodsr)r wavswav_lens encoder_outrrrr GszWaveformEncoder.encode_batchcC |||SzRuns the encoderr )r rrrrrforwarda zWaveformEncoder.forwardN)__name__ __module__ __qualname____doc__MODULES_NEEDEDrr r rrrrrs  rc@sLeZdZdZddgZdddZdd Zd d Zd d ZdddZ ddZ dS)MelSpectrogramEncoderaA MelSpectrogramEncoder class created for the Zero-Shot Multi-Speaker TTS models. This is for speaker encoder models using the PyTorch MelSpectrogram transform for compatibility with the current TTS pipeline. This class can be used to encode a single waveform, a single mel-spectrogram, or a batch of mel-spectrograms. Arguments --------- See ``Pretrained`` Example ------- >>> import torchaudio >>> from speechbrain.inference.encoders import MelSpectrogramEncoder >>> # Model is downloaded from the speechbrain HuggingFace repo >>> tmpdir = getfixture("tmpdir") >>> encoder = MelSpectrogramEncoder.from_hparams( ... source="speechbrain/tts-ecapa-voxceleb", ... savedir=tmpdir, ... ) # doctest: +SKIP >>> # Compute embedding from a waveform (sample_rate must match the sample rate of the encoder) >>> signal, fs = torchaudio.load("tests/samples/single-mic/example1.wav") # doctest: +SKIP >>> spk_emb = encoder.encode_waveform(signal) # doctest: +SKIP >>> # Compute embedding from a mel-spectrogram (sample_rate must match the sample rate of the ecoder) >>> mel_spec = encoder.mel_spectogram(audio=signal) # doctest: +SKIP >>> spk_emb = encoder.encode_mel_spectrogram(mel_spec) # doctest: +SKIP >>> # Compute embeddings for a batch of mel-spectrograms >>> spk_embs = encoder.encode_mel_spectrogram_batch(mel_spec) # doctest: +SKIP normalizerembedding_modelh㈵>cCsttj||d|S)z+Dynamic range compression for audio signals)min)r logclamp)r xCclip_valrrrdynamic_range_compressionsz/MelSpectrogramEncoder.dynamic_range_compressionc Cs~ddlm}|j|jj|jj|jj|jj|jj|jj |jj |jj |jj |jj |jjd |j}||}|jjr=||}|S)zcalculates MelSpectrogram for a raw audio signal Arguments --------- audio : torch.tensor input audio signal Returns ------- mel : torch.Tensor Mel-spectrogram r) transforms) sample_rate hop_length win_lengthn_fftn_melsf_minf_maxpower normalizednorm mel_scale) torchaudior3MelSpectrogramhparamsr4r5r6r7n_mel_channelsmel_fminmel_fmaxr;mel_normalizedr=r>rrr2)r audior3 audio_to_melmelrrrmel_spectograms(   z$MelSpectrogramEncoder.mel_spectogramcCs"||j}|j|d}||S)z Encodes a single waveform Arguments --------- wav : torch.Tensor waveform Returns ------- encoder_out : torch.Tensor Speaker embedding for the input waveform )rF)rrrIencode_mel_spectrogram)r wavmel_specrrrencode_waveforms   z%MelSpectrogramEncoder.encode_waveformcCs8|}t|jdkr|d}tdg}|||}|S)a Encodes a single mel-spectrograms Arguments --------- mel_spec : torch.Tensor Mel-spectrograms Returns ------- encoder_out : torch.Tensor Speaker embedding for the input mel-spectrogram rr)lenshaperr r encode_mel_spectrogram_batch)r rLrrrrrrrJs    z,MelSpectrogramEncoder.encode_mel_spectrogramNcCsd|durtj|jd|jd}||j||j}}t|dd}|j||}|j|}|S)ap Encodes a batch of mel-spectrograms Arguments --------- mel_specs : torch.Tensor Mel-spectrograms lens : torch.Tensor Relative lengths of the mel-spectrograms Returns ------- encoder_out : torch.Tensor Speaker embedding for the input mel-spectrogram batch Nr)rr*rN) r onesrPrr transposerAr(r))r mel_specslensfeatsrrrrrQs z2MelSpectrogramEncoder.encode_mel_spectrogram_batchcCrrr)r rTrUrrr __forwardr!zMelSpectrogramEncoder.__forward)r*r+)N) r"r#r$r%r&r2rIrMrJrQ_MelSpectrogramEncoder__forwardrrrrr'fs" $  r')r%r speechbrain.inference.interfacesrrr'rrrrs  P