o ei@s8dZddlZddlmZddlmZGdddeZdS)aa Specifies the inference interfaces for Spoken Language Understanding (SLU) modules. Authors: * Aku Rouhe 2021 * Peter Plantinga 2021 * Loren Lugosch 2020 * Mirco Ravanelli 2020 * Titouan Parcollet 2021 * Abdel Heba 2021 * Andreas Nautsch 2022, 2023 * Pooneh Mousavi 2023 * Sylvain de Langen 2023 * Adel Moumen 2023 * Pradnya Kandarkar 2023 N)EncoderDecoderASR) PretrainedcsPeZdZdZddgZddgZfddZdd Zd d Zd d Z ddZ Z S) EndToEndSLUa)An end-to-end SLU model. The class can be used either to run only the encoder (encode()) to extract features or to run the entire model (decode()) to map the speech to its semantics. Arguments --------- *args : tuple **kwargs : dict Arguments are forwarded to ``Pretrained`` parent class. Example ------- >>> from speechbrain.inference.SLU import EndToEndSLU >>> tmpdir = getfixture("tmpdir") >>> slu_model = EndToEndSLU.from_hparams( ... source="speechbrain/slu-timers-and-such-direct-librispeech-asr", ... savedir=tmpdir, ... ) # doctest: +SKIP >>> slu_model.decode_file("tests/samples/single-mic/example6.wav") # doctest: +SKIP "{'intent': 'SimpleMath', 'slots': {'number1': 37.67, 'number2': 75.7, 'op': ' minus '}}" tokenizerasr_model_sourceslu_enc beam_searchercs:tj|i||jj|_tj|jjd|jid|_dS)Ndevice)sourcerun_opts) super__init__hparamsrr from_hparamsrr asr_model)selfargskwargs __class__W/home/ubuntu/transcripts/venv/lib/python3.10/site-packages/speechbrain/inference/SLU.pyr 2s   zEndToEndSLU.__init__cKsL|j|fi|}||j}|d}tdg}|||\}}|dS)amMaps the given audio file to a string representing the semantic dictionary for the utterance. Arguments --------- path : str Path to audio file to decode. **kwargs : dict Arguments forwarded to ``load_audio``. Returns ------- str The predicted semantics. rg?) load_audiotor unsqueezetorchtensor decode_batch)rpathrwaveformbatch rel_lengthpredicted_wordspredicted_tokensrrr decode_file:s    zEndToEndSLU.decode_filecCsD|}||j||j}}|j||}|j|}|S)a`Encodes the input audio into a sequence of hidden states Arguments --------- wavs : torch.Tensor Batch of waveforms [batch, time, channels] or [batch, time] depending on the model. wav_lens : torch.Tensor Lengths of the waveforms relative to the longest one in the batch, tensor of shape [batch]. The longest one should have relative length 1.0 and others len(waveform) / max_length. Used for ignoring padding. Returns ------- torch.Tensor The encoded batch )floatrr r encode_batchdetachmodsr)rwavswav_lensASR_encoder_out encoder_outrrrr&Rs  zEndToEndSLU.encode_batchcst2|j|j}}||}j||\}}}}fdd|D}Wd||fS1s9wY||fS)aMaps the input audio to its semantics Arguments --------- wavs : torch.Tensor Batch of waveforms [batch, time, channels] or [batch, time] depending on the model. wav_lens : torch.Tensor Lengths of the waveforms relative to the longest one in the batch, tensor of shape [batch]. The longest one should have relative length 1.0 and others len(waveform) / max_length. Used for ignoring padding. Returns ------- list Each waveform in the batch decoded. tensor Each predicted token id. csg|]}j|qSr)r decode_ids).0 token_seqrrr s z,EndToEndSLU.decode_batch..N)rno_gradrr r&r(r)rr)r*r,r#scores_r"rr0rrks      zEndToEndSLU.decode_batchcCs |||S)z8Runs full decoding - note: no gradients through decoding)r)rr)r*rrrforwards zEndToEndSLU.forward) __name__ __module__ __qualname____doc__HPARAMS_NEEDEDMODULES_NEEDEDr r$r&rr5 __classcell__rrrrrs !r)r9rspeechbrain.inference.ASRr speechbrain.inference.interfacesrrrrrrs