o i~@sPdZddlmZddlmZddlZddlmZddlmZGdddeZ dS)zESequential implementation of Recurrent Neural Network Language Model.)Tuple)UnionN)AbsLMcseZdZdZ       d$d ed ed ed ed edededeffdd ZddZ de j de j de e j e j ffddZ de j dee j e e j e j ffde j de e j ee j e e j e j ffffddZde j d e j d!e j de e j e j ffd"d#ZZS)%SequentialRNNLMzSequential RNNLM. See also: https://github.com/pytorch/examples/blob/4581968193699de14b56527296262dd76ab43557/word_language_model/model.py NFlstmr vocab_sizeunitnhidnlayers dropout_rate tie_weightsrnn_type ignore_idc st|} |dur |}|}t||_tj|| |d|_|dvr4tt|} | | |||dd|_ n!z ddd|} Wn t yHt dwtj | ||| |dd |_ t |||_|rl|| krft d |jj|j_||_||_||_dS) N) padding_idx)LSTMGRUT)dropout batch_firsttanhrelu)RNN_TANHRNN_RELUzwAn invalid option for `--model` was supplied, options are ['LSTM', 'GRU', 'RNN_TANH' or 'RNN_RELU']) nonlinearityrrz6When using the tied flag, nhid must be equal to emsize)super__init__uppernnDropoutdrop EmbeddingencodergetattrrnnKeyError ValueErrorRNNLineardecoderweightrr r ) selfr r r r rrrrninp rnn_classr __class__[/home/ubuntu/.local/lib/python3.10/site-packages/funasr/models/language_model/seq_rnn_lm.pyrsB       zSequentialRNNLM.__init__cCsht|jtjjr&tj|j|jftjd}tj|j|jftjd}||f}|Stj|j|jftjd}|S)z,Initialize LM state filled with zero values.)dtype) isinstancer%torchrrzerosr r float)r,hcstater1r1r2 zero_stateLszSequentialRNNLM.zero_stateinputhiddenreturncCsz|||}|||\}}||}|||d|d|d}||d|d|d|fS)Nrr)r!r#r%r* contiguousviewsize)r,r<r=emboutputdecodedr1r1r2forwardWs $zSequentialRNNLM.forwardyr:xcCs4||ddd|\}}|jddd}||fS)aHScore new token. Args: y: 1D torch.int64 prefix tokens. state: Scorer state for prefix tokens x: 2D encoder feature that generates ys. Returns: Tuple of torch.float32 scores for next token (n_vocab) and next state for ys r?dim)rA log_softmax)r,rGr:rH new_statelogpr1r1r2scoreeszSequentialRNNLM.scoreysstatesxscsddur dn,t|jtjjr.tjddDddtjddDddfntjdd||ddddf\}|ddksPJ|j|d}|j dd}t|jtjjrz\fd dt dD|fSfd dt dD|fS) aScore new token batch. Args: ys (torch.Tensor): torch.int64 prefix tokens (n_batch, ylen). states (List[Any]): Scorer states for prefix tokens. xs (torch.Tensor): The encoder feature that generates ys (n_batch, xlen, n_feat). Returns: tuple[torch.Tensor, List[Any]]: Tuple of batchfied scores for next token with shape of `(n_batch, n_vocab)` and next state list for ys. rNcSsg|]\}}|qSr1r1.0r8r9r1r1r2 z/SequentialRNNLM.batch_score..r?rJcSsg|]\}}|qSr1r1rSr1r1r2rUrVrIcs,g|]}dd|fdd|ffqSNr1rTi)r9r8r1r2rUs,csg|] }dd|fqSrWr1rX)rQr1r2rUs) r4r%r5rrstackrBshapesqueezerLrange)r,rPrQrRrNr1)r9r8rQr2 batch_score|s"    zSequentialRNNLM.batch_score)rNrrFr r)__name__ __module__ __qualname____doc__intr7boolstrrr;r5TensorrrFrrOr^ __classcell__r1r1r/r2r sh  9  " r) rbtypingrrr5torch.nnrfunasr.train.abs_modelrrr1r1r1r2s