o ei@svdZddlZddlmmZddlZddlZddlm Z ddl m Z ddl m Z ddlmZmZGddde ZdS) aN Specifies the inference interfaces for interpretability modules. Authors: * Aku Rouhe 2021 * Peter Plantinga 2021 * Loren Lugosch 2020 * Mirco Ravanelli 2020 * Titouan Parcollet 2021 * Abdel Heba 2021 * Andreas Nautsch 2022, 2023 * Pooneh Mousavi 2023 * Sylvain de Langen 2023 * Adel Moumen 2023 * Pradnya Kandarkar 2023 N) Pretrained)spectral_phase) split_path) LocalStrategyfetchcsTeZdZdZfddZddZddZdd Zd d Zdd dZ dddZ Z S)PIQAudioInterpreterah This class implements the interface for the PIQ posthoc interpreter for an audio classifier. Arguments --------- *args : tuple **kwargs : dict Arguments are forwarded to ``Pretrained`` parent class. Example ------- >>> from speechbrain.inference.interpretability import PIQAudioInterpreter >>> tmpdir = getfixture("tmpdir") >>> interpreter = PIQAudioInterpreter.from_hparams( ... source="speechbrain/PIQ-ESC50", ... savedir=tmpdir, ... ) >>> signal = torch.randn(1, 16000) >>> interpretation, _ = interpreter.interpret_batch(signal) cstj|i|dSN)super__init__)selfargskwargs __class__d/home/ubuntu/transcripts/venv/lib/python3.10/site-packages/speechbrain/inference/interpretability.pyr 2szPIQAudioInterpreter.__init__cCs6|j|}tjjj||jjd}t |}|||fS)z#Pre-process wavs to calculate STFTs)power) mods compute_stft speechbrain processingfeaturesspectral_magnitudehparamsspec_mag_powertorchlog1p)r wavsX_stft X_stft_powerX_stft_logpowerrrr preprocess5s   zPIQAudioInterpreter.preprocesscCs>|j|}|d}|j|d}|d}||||fS)z#the forward pass for the classifier))rembedding_modelmean classifiersqueezeargmax)r r hcat embeddings predictions class_predrrrclassifier_forward?s    z&PIQAudioInterpreter.classifier_forwardcCs|tjt|dt|dfdd}|ddd|jdddddf}|jdkr2|d}||}|j|}|S)z!Inverts STFT spectra given phase.r")dimNr$) rcatcos unsqueezesinshapendimr compute_istft)r X_int X_stft_phaseX_stft_phase_sbX_wpsbx_int_sbrrrinvert_stft_with_phaseGs&   z*PIQAudioInterpreter.invert_stft_with_phasecCs||j}||\}}}t|}||\}}}} |jjr*|j|| \} }} n|jj |} | d} | j d} |jj rTt | } | |ddd| ddf} nt | } | |jj}| |k|ddd| ddf} t| } || |}|jj| d}||fS)a#Classifies the given audio into the given set of labels. It also provides the interpretation in the audio domain. Arguments --------- wavs : torch.Tensor Batch of waveforms [batch, time, channels] or [batch, time] depending on the model. Make sure the sample rate is fs=16000 Hz. Returns ------- x_int_sound_domain : torch.Tensor The interpretation in the waveform domain text_lab : str The text label for the classification r$Nr)todevicer!rr.ruse_vqrpsidecoderr(r5use_mask_outputFsigmoidsoftplusmaxmask_threxpm1r= label_encoder decode_torchr3)r rr rrr9r*r+r,r-xhatz_q_xTmaxr8thx_int_sound_domaintext_labrrrinterpret_batchXs.       "  z#PIQAudioInterpreter.interpret_batchNc Cst|\}}t|||tjd}t|\}}||j}|jj }||krCt d ||tj j ||d|j}|jddd}||}||\} } | | |fS)aQClassifies the given audiofile into the given set of labels. It also provides the interpretation in the audio domain. Arguments --------- path : str Path to audio file to classify. savedir : str Path to cache directory. Returns ------- x_int_sound_domain : torch.Tensor The interpretation in the waveform domain text_lab : str The text label for the classification fs_model : int The sampling frequency of the model. Useful to save the audio. )sourcesavedirlocal_strategyz(Resampling the audio from {} Hz to {} Hz) orig_freqnew_freqrT)r/keepdim)rrrSYMLINK torchaudioloadr>r?r sample_rateprintformat transformsResampler&rR) r pathrTrSflbatchfs_filefs_modeltfrPrQrrrinterpret_files2   z"PIQAudioInterpreter.interpret_filecCs |||S)zRuns the classification)rR)r rwav_lensrrrforwards zPIQAudioInterpreter.forwardr) __name__ __module__ __qualname____doc__r r!r.r=rRrgri __classcell__rrrrrs   /0r)rmrtorch.nn.functionalnn functionalrDrZr speechbrain.inference.interfacesrspeechbrain.processing.NMFrspeechbrain.utils.data_utilsrspeechbrain.utils.fetchingrrrrrrrs