o i @sdddlZddlmZmZmZddlmZddlmZddl m Z e ddGdd d ej j ZdS) N)ListOptionalTuple)tables)SpecAug) Hypothesisdecoder_classes rnnt_decodercseZdZdZ        d2ded ed ed ed ed ededededdffdd Z d3de j de j de e e j e e j fde j fddZ de j de e j e e j fde e j e e j e e j fffddZde j deed e e j e e j fde e j e e j e e j fffd!d"Zd#eede e j e e j e e j fffd$d%Zd&e jddfd'd(Zd)ede e j e e jffd*d+Zde e j e e j fd,ede e j e e j ffd-d.Zd/ee e j e e j fde e j e e j ffd0d1ZZS)4 RNNTDecoderaRNN decoder module. Args: vocab_size: Vocabulary size. embed_size: Embedding size. hidden_size: Hidden size.. rnn_type: Decoder layers type. num_layers: Number of decoder layers. dropout_rate: Dropout rate for decoder layers. embed_dropout_rate: Dropout rate for embedding layer. embed_pad: Embedding padding symbol ID. lstmrF vocab_size embed_size hidden_sizernn_type num_layers dropout_rateembed_dropout_rate embed_paduse_embed_maskreturnNc st|dvrtd|tjj|||d|_tjj|d|_|dkr*tjj ntjj } tj | ||dddg|_ t d|D]} |j | ||dddg7_ qAtj fd d t |D|_||_||_||_||_t|j|_i|_| |_|jrtd d d d d|_dSdS)zConstruct a RNNDecoder object.)r gruzNot supported: rnn_type=) padding_idxpr r T) batch_firstcsg|] }tjjdqS)r)torchnnDropout).0_rY/home/ubuntu/.local/lib/python3.10/site-packages/funasr/models/transducer/rnnt_decoder.py ;sz(RNNTDecoder.__init__..F)time_mask_width_range num_time_maskapply_freq_maskapply_time_warpN)super__init__ ValueErrorrr Embeddingembedr dropout_embedLSTMGRU ModuleListrnnrange dropout_rnndlayersdtype output_sizernext parametersdevice score_cacherr _embed_mask) selfrrrrrrrrr rnn_classr" __class__r#r%r.s6  zRNNTDecoder.__init__labels label_lensstatescCsX|dur ||d}|||}|jr"|jr"|||d}|||\}}|S)a%Encode source label sequences. Args: labels: Label ID sequences. (B, L) states: Decoder hidden states. ((N, B, D_dec), (N, B, D_dec) or None) or None Returns: dec_out: Decoder output sequences. (B, U, D_dec) Nr) init_statesizer2r1rtrainingr@ rnn_forward)rArErFrG dec_embeddec_outr$r$r%forwardPs zRNNTDecoder.forwardxstatecCs|\}}||d\}}t|jD]M}|jdkrB|j|||||d|||dfd\}\|||d<|||d<n|j|||||dd\}|||d<|j||}q|||ffS)atEncode source label sequences. Args: x: RNN input sequences. (B, D_emb) state: Decoder hidden states. ((N, B, D_dec), (N, B, D_dec) or None) Returns: x: RNN output sequences. (B, D_dec) (h_next, c_next): Decoder hidden states. (N, B, D_dec), (N, B, D_dec) or None) rr r )hx)rHrIr7r9r:r6r8)rArOrPh_prevc_prevh_nextc_nextlayerr$r$r%rKjs  (. zRNNTDecoder.rnn_forwardlabellabel_sequence dec_statecCs^dtt|}||jvr|j|\}}n||}|||\}}||f|j|<|d|fS)aOne-step forward hypothesis. Args: label: Previous label. (1, 1) label_sequence: Current label sequence. dec_state: Previous decoder hidden states. ((N, 1, D_dec), (N, 1, D_dec) or None) Returns: dec_out: Decoder output sequence. (1, D_dec) dec_state: Decoder hidden states. ((N, 1, D_dec), (N, 1, D_dec) or None) r"r)joinmapstrr?r1rK)rArWrXrY str_labelsrMrLr$r$r%scores   zRNNTDecoder.scorehypscCsVtjdd|D|jd}||}|dd|D}|||\}}|d|fS)zOne-step forward hypotheses. Args: hyps: Hypotheses. Returns: dec_out: Decoder output sequences. (B, D_dec) states: Decoder hidden states. ((N, B, D_dec), (N, B, D_dec) or None) cSsg|]}|jdgqS))yseqr!hr$r$r%r&sz+RNNTDecoder.batch_score..r>cSsg|]}|jqSr$)rYrbr$r$r%r&sr )r LongTensorr>r1create_batch_statesrKsqueeze)rAr_rErLrGrMr$r$r% batch_scores  zRNNTDecoder.batch_scorer>cCs ||_dS)zNSet GPU device to use. Args: device: Device ID. Nrd)rAr>r$r$r% set_devices zRNNTDecoder.set_device batch_sizecCsJtj|j||j|jd}|jdkr!tj|j||j|jd}||fS|dfS)zInitialize decoder states. Args: batch_size: Batch size. Returns: : Initial decoder hidden states. ((N, B, D_dec), (N, B, D_dec) or None) rdr N)rzerosr9r;r>r:)rArjh_nc_nr$r$r%rHs  zRNNTDecoder.init_stateidxcCsT|ddd||dddf|jdkr'|ddd||dddffSdfS)a-Get specified ID state from decoder hidden states. Args: states: Decoder hidden states. ((N, B, D_dec), (N, B, D_dec) or None) idx: State ID to extract. Returns: : Decoder hidden state for given ID. ((N, 1, D_dec), (N, 1, D_dec) or None) rNr r )r:)rArGrnr$r$r% select_states *zRNNTDecoder.select_state new_statescCs@tjdd|Ddd|jdkrtjdd|DddfSdfS)zCreate decoder hidden states. Args: new_states: Decoder hidden states. [N x ((1, D_dec), (1, D_dec) or None)] Returns: states: Decoder hidden states. ((N, B, D_dec), (N, B, D_dec) or None) cSg|]}|dqS)rr$r!sr$r$r%r&z3RNNTDecoder.create_batch_states..r )dimr cSrq)r r$rrr$r$r%r&rtN)rcatr:)rArpr$r$r%rfs  zRNNTDecoder.create_batch_states)r r r r rrrF)N)__name__ __module__ __qualname____doc__intr\floatboolr.rTensorrrrNrKrr^rrhr>ritensorrHrorf __classcell__r$r$rCr%r s   6      "  r )rtypingrrrfunasr.registerrfunasr.models.specaug.specaugr/funasr.models.transducer.beam_search_transducerrregisterrModuler r$r$r$r%s