o }oi4@sddlmZmZddlmZddlmZddlmZddl m Z ddl m Z ddl mZddlmZdd lmZmZdd lmZGd d d e eZGd ddeeZGdddeeZGddde eZdS))ListOptional)Trainer) DictConfig)EncDecCTCModelBPE)EncDecCTCModel)EncDecRNNTBPEModel)EncDecRNNTModel) ASRK2Mixin)PretrainedModelInfo typecheck)loggingcsxeZdZdZddedeffdd Zedee e fdd Z d e e ffd d Z e    dfd d ZZS)EncDecK2SeqModel3Encoder decoder models with various lattice losses.NcfgtrainercR|jdd}|dkr|dkrtd|jjd|tj||d|dSN loss_typectcmmiClass z does not support `loss_type`=rrgraph_module_cfgget ValueError __class____name__super__init___init_k2selfrrrrb/home/ubuntu/.local/lib/python3.10/site-packages/nemo/collections/asr/models/k2_sequence_models.pyr  zEncDecK2SeqModel.__init__returncCdSz This method returns a list of pre-trained model which can be instantiated directly from NVIDIA's NGC cloud. Returns: List of available pre-trained models. Nr%clsr%r%r&list_available_models'z&EncDecK2SeqModel.list_available_modelsnew_vocabularyc>t||jrd|_td|jjd||j dSa Changes vocabulary used during CTC decoding process. Use this method when fine-tuning on from pre-trained model. This method changes only decoder and leaves encoder and pre-processing modules unchanged. For example, you would use it if you want to use pretrained encoder when fine-tuning on a data in another language, or when you'd need model to learn capitalization, punctuation and/or special characters. If new_vocabulary == self.decoder.vocabulary then nothing will be changed. Args: new_vocabulary: list with new vocabulary. Must contain at least 2 elements. Typically, this is target alphabet. Returns: None N@With .change_vocabulary() call for a model with criterion_type=``, a new token_lm has to be set manually: call .update_k2_modules(new_cfg) or update .graph_module_cfg.backend_cfg.token_lm before calling this method. rchange_vocabulary use_graph_lmtoken_lmr warninglosscriterion_typeupdate_k2_modulesrr#r/r$r%r&r51 z"EncDecK2SeqModel.change_vocabularyc*tj||||d\}}}|j|||dSa/ Forward pass of the model. Args: input_signal: Tensor that represents a batch of raw audio signals, of shape [B, T]. T here represents timesteps, with 1 second of audio represented as `self.sample_rate` number of floating point values. input_signal_length: Vector of length B, that contains the individual lengths of the audio sequences. processed_signal: Tensor that represents a batch of processed audio signals, of shape (B, D, T) that has undergone processing via some DALI preprocessor. processed_signal_length: Vector of length B, that contains the individual lengths of the processed audio sequences. Returns: A tuple of 3 elements - 1) The log probabilities tensor of shape [B, T, D]. 2) The lengths of the acoustic sequence after propagation through the encoder, of shape [B]. 3) The greedy token predictions of the model of shape [B, T] (via argmax) ) input_signalinput_signal_lengthprocessed_signalprocessed_signal_length) log_probsencoded_lengthgreedy_predictionsrforward_forward_k2_post_processingr#r@rArBrCrD encoded_lenrFr$r%r&rHM zEncDecK2SeqModel.forwardNNNNNr __module__ __qualname____doc__rrr classmethodrrr r-strr5r rH __classcell__r%r%r$r&r rcsxeZdZdZddedeffdd Zedee e fdd Z d e d e ffd d Z e    dfdd ZZS)EncDecK2SeqModelBPEJEncoder decoder models with Byte Pair Encoding and various lattice losses.Nrrcrrrr"r$r%r&r wr'zEncDecK2SeqModelBPE.__init__r(cCr)r*r%r+r%r%r&r-~r.z)EncDecK2SeqModelBPE.list_available_modelsnew_tokenizer_dirnew_tokenizer_typec@t|||jrd|_td|jjd||j dSa Changes vocabulary of the tokenizer used during CTC decoding process. Use this method when fine-tuning on from pre-trained model. This method changes only decoder and leaves encoder and pre-processing modules unchanged. For example, you would use it if you want to use pretrained encoder when fine-tuning on a data in another language, or when you'd need model to learn capitalization, punctuation and/or special characters. Args: new_tokenizer_dir: Path to the new tokenizer directory. new_tokenizer_type: Either `bpe` or `wpe`. `bpe` is used for SentencePiece tokenizers, whereas `wpe` is used for `BertTokenizer`. Returns: None Nr2r3r4r#rYrZr$r%r&r5z%EncDecK2SeqModelBPE.change_vocabularycr>r?rGrJr$r%r&rHrLzEncDecK2SeqModelBPE.forwardrMrNrOr%r%r$r&rWtrVrWcsXeZdZdZd dedeffdd Zedee fdd Z d e e ffd d Z ZS)EncDecK2RnntSeqModelrNrrcf|jdd}|jdd}|dks|dkr%td|jjd|d|tj||d|dS Nrrnntr:mlrz0 does not support `criterion_type`=z with `loss_type`=rrr#rrrr:r$r%r&r   zEncDecK2RnntSeqModel.__init__r(cCr)r*r%r+r%r%r&r-r.z*EncDecK2RnntSeqModel.list_available_modelsr/cr0r1r4r<r$r%r&r5r=z&EncDecK2RnntSeqModel.change_vocabularyrM)rrPrQrRrrr rSrr r-rrTr5rUr%r%r$r&r_   r_csXeZdZdZddedeffdd Zedee fdd Z d e d e ffd d Z Z S)EncDecK2RnntSeqModelBPErXNrrcr`rarrdr$r%r&r rez EncDecK2RnntSeqModelBPE.__init__r(cCr)r*r%r+r%r%r&r-r.z-EncDecK2RnntSeqModelBPE.list_available_modelsrYrZcr[r\r4r]r$r%r&r5r^z)EncDecK2RnntSeqModelBPE.change_vocabularyrM)rrPrQrRrrr rSrr r-rTr5rUr%r%r$r&rgrfrgN)typingrrlightning.pytorchr omegaconfr*nemo.collections.asr.models.ctc_bpe_modelsr&nemo.collections.asr.models.ctc_modelsr+nemo.collections.asr.models.rnnt_bpe_modelsr'nemo.collections.asr.models.rnnt_modelsr %nemo.collections.asr.parts.k2.classesr nemo.core.classes.commonr r nemo.utilsr rrWr_rgr%r%r%r&s        WW5