o %ݫi-@shdZddlZddlZddlZddlmZddlmZddlm Z m Z GdddeZ Gdd d eZ dS) aR Specifies the inference interfaces for Audio Classification modules. Authors: * Aku Rouhe 2021 * Peter Plantinga 2021 * Loren Lugosch 2020 * Mirco Ravanelli 2020 * Titouan Parcollet 2021 * Abdel Heba 2021 * Andreas Nautsch 2022, 2023 * Pooneh Mousavi 2023 * Sylvain de Langen 2023 * Adel Moumen 2023 * Pradnya Kandarkar 2023 N) Pretrained) split_path) LocalStrategyfetchc@s>eZdZdZgdZd ddZdddZd d Zdd d ZdS)EncoderClassifiera9A ready-to-use class for utterance-level classification (e.g, speaker-id, language-id, emotion recognition, keyword spotting, etc). The class assumes that an encoder called "embedding_model" and a model called "classifier" are defined in the yaml file. If you want to convert the predicted index into a corresponding text label, please provide the path of the label_encoder in a variable called 'lab_encoder_file' within the yaml. The class can be used either to run only the encoder (encode_batch()) to extract embeddings or to run a classification step (classify_batch()). Arguments --------- See ``Pretrained`` Example ------- >>> import torchaudio >>> from speechbrain.inference.classifiers import EncoderClassifier >>> # Model is downloaded from the speechbrain HuggingFace repo >>> tmpdir = getfixture("tmpdir") >>> classifier = EncoderClassifier.from_hparams( ... source="speechbrain/spkrec-ecapa-voxceleb", ... savedir=tmpdir, ... ) >>> classifier.hparams.label_encoder.ignore_len() >>> # Compute embeddings >>> signal, fs = torchaudio.load("tests/samples/single-mic/example1.wav") >>> embeddings = classifier.encode_batch(signal) >>> # Classification >>> prediction = classifier.classify_batch(signal) )compute_features mean_var_normembedding_model classifierNFcCst|jdkr |d}|durtj|jd|jd}||j||j}}|}|j |}|j ||}|j ||}|rR|j |tj|jd|jd}|S)aEncodes the input audio into a single vector embedding. The waveforms should already be in the model's desired format. You can call: ``normalized = .normalizer(signal, sample_rate)`` to get a correctly converted signal in most cases. Arguments --------- wavs : torch.Tensor Batch of waveforms [batch, time, channels] or [batch, time] depending on the model. Make sure the sample rate is fs=16000 Hz. wav_lens : torch.Tensor Lengths of the waveforms relative to the longest one in the batch, tensor of shape [batch]. The longest one should have relative length 1.0 and others len(waveform) / max_length. Used for ignoring padding. normalize : bool If True, it normalizes the embeddings with the statistics contained in mean_var_norm_emb. Returns ------- torch.Tensor The encoded batch rN)device)lenshape unsqueezetorchonesr tofloatmodsrrr hparamsmean_var_norm_emb)selfwavswav_lens normalizefeats embeddingsrU/home/ubuntu/.local/lib/python3.10/site-packages/speechbrain/inference/classifiers.py encode_batchFs  zEncoderClassifier.encode_batchcCsJ|||}|j|d}tj|dd\}}|jj|}||||fS)aaPerforms classification on the top of the encoded features. It returns the posterior probabilities, the index and, if the label encoder is specified it also the text label. Arguments --------- wavs : torch.Tensor Batch of waveforms [batch, time, channels] or [batch, time] depending on the model. Make sure the sample rate is fs=16000 Hz. wav_lens : torch.Tensor Lengths of the waveforms relative to the longest one in the batch, tensor of shape [batch]. The longest one should have relative length 1.0 and others len(waveform) / max_length. Used for ignoring padding. Returns ------- out_prob The log posterior probabilities of each class ([batch, N_class]) score: It is the value of the log-posterior for the best class ([batch,]) index The indexes of the best class ([batch,]) text_lab: List with the text labels corresponding to the indexes. (label encoder should be provided). r dim) rrr squeezermaxr label_encoder decode_torch)rrrembout_probscoreindextext_labrrrclassify_batchws  z EncoderClassifier.classify_batchc Ksr|j|fi|}|d}tdg}|||}|j|d}tj|dd\}} |j j | } ||| | fS)aClassifies the given audiofile into the given set of labels. Arguments --------- path : str Path to audio file to classify. **kwargs : dict Arguments forwarded to ``load_audio``. Returns ------- out_prob : torch.Tensor The log posterior probabilities of each class ([batch, N_class]) score : torch.Tensor It is the value of the log-posterior for the best class ([batch,]) index : torch.Tensor The indexes of the best class ([batch,]) text_lab : list of str List with the text labels corresponding to the indexes. (label encoder should be provided). rg?r r r!) load_audiorrtensorrrr r#r$rr%r&) rpathkwargswaveformbatch rel_lengthr'r(r)r*r+rrr classify_files    zEncoderClassifier.classify_filecC |||SzRuns the classificationr,rrrrrrforward zEncoderClassifier.forward)NFN) __name__ __module__ __qualname____doc__MODULES_NEEDEDrr,r4r9rrrrrs$  1# rc@s.eZdZdZd ddZd ddZd ddZdS) AudioClassifieraiA ready-to-use class for utterance-level classification (e.g, speaker-id, language-id, emotion recognition, keyword spotting, etc). The class assumes that an encoder called "embedding_model" and a model called "classifier" are defined in the yaml file. If you want to convert the predicted index into a corresponding text label, please provide the path of the label_encoder in a variable called 'lab_encoder_file' within the yaml. The class can be used either to run only the encoder (encode_batch()) to extract embeddings or to run a classification step (classify_batch()). Arguments --------- See ``Pretrained``. Example ------- >>> import torchaudio >>> from speechbrain.inference.classifiers import AudioClassifier >>> tmpdir = getfixture("tmpdir") >>> classifier = AudioClassifier.from_hparams( ... source="speechbrain/cnn14-esc50", ... savedir=tmpdir, ... ) >>> signal = torch.randn(1, 16000) >>> prediction, _, _, text_lab = classifier.classify_batch(signal) >>> print(prediction.shape) torch.Size([1, 1, 50]) Nc Cs||j}|j|}tjjj||jj d}|jj r"|j |}nt |}|j|}|jdkr7|d}|j|}t j|dd\}} |jj| } ||| | fS)aPerforms classification on the top of the encoded features. It returns the posterior probabilities, the index and, if the label encoder is specified it also the text label. Arguments --------- wavs : torch.Tensor Batch of waveforms [batch, time, channels] or [batch, time] depending on the model. Make sure the sample rate is fs=16000 Hz. wav_lens : torch.Tensor Lengths of the waveforms relative to the longest one in the batch, tensor of shape [batch]. The longest one should have relative length 1.0 and others len(waveform) / max_length. Used for ignoring padding. Returns ------- out_prob : torch.Tensor The log posterior probabilities of each class ([batch, N_class]) score : torch.Tensor It is the value of the log-posterior for the best class ([batch,]) index : torch.Tensor The indexes of the best class ([batch,]) text_lab : list of str List with the text labels corresponding to the indexes. (label encoder should be provided). )power)r r r!)rr r compute_stft speechbrain processingfeaturesspectral_magnituderspec_mag_poweruse_melspectra compute_fbankrlog1pr ndimmeanr r$r%r&) rrrX_stft X_stft_power net_inputr out_probsr)r*r+rrrr,s        zAudioClassifier.classify_batchc Cst|\}}t|||tjd}t|\}}||j}|jj }||krCt d ||tj j ||d|j}|jddd}||}||\} } } } | | | | fS)aClassifies the given audiofile into the given set of labels. Arguments --------- path : str Path to audio file to classify. savedir : str Path to folder for caching downloads. Returns ------- out_prob The log posterior probabilities of each class ([batch, N_class]) score: It is the value of the log-posterior for the best class ([batch,]) index The indexes of the best class ([batch,]) text_lab: List with the text labels corresponding to the indexes. (label encoder should be provided). )sourcesavedirlocal_strategyz(Resampling the audio from {} Hz to {} Hz) orig_freqnew_freqrT)r"keepdim)rrrSYMLINK torchaudioloadrr r sample_rateprintformat transformsResamplerOr,) rr/rUrTflr2fs_filefs_modeltfrSr)r*r+rrrr4s2   zAudioClassifier.classify_filecCr5r6r7r8rrrr9Cr:zAudioClassifier.forwardr;)r<r=r>r?r,r4r9rrrrrAs   22rA) r?rr[rF speechbrain.inference.interfacesrspeechbrain.utils.data_utilsrspeechbrain.utils.fetchingrrrrArrrrs  &