o i>5@sdZddlZddlZddlZddlZddlZddlZddlZddl Z ddlm Z ddl m Z ddl mZddlmZddlmZmZmZddlmZmZmZdd lmZdd lmZdd lmZdd lm Z Gd ddej!Z"Gdddee j#j$Z%dS)z:RNN sequence-to-sequence text translation model (pytorch).N)reporter)label_smoothing_dist) MTInterface)uniform_init_parameters) get_subsamplepad_list to_device)"add_arguments_rnn_attention_common add_arguments_rnn_decoder_common add_arguments_rnn_encoder_common)att_for) decoder_for) encoder_for)fill_missing_argsc@seZdZdZddZdS)ReporterzA chainer reporter wrapper.cCsDtd|i|td|i|td|i|td|i|dS)zReport at every step.lossaccpplbleuN)rreport)selfrrrrrV/home/ubuntu/.local/lib/python3.10/site-packages/espnet/nets/pytorch_backend/e2e_mt.pyr#szReporter.reportN)__name__ __module__ __qualname____doc__rrrrrr s rcseZdZdZeddZeddZeddZedd Zfd d Z d d Z ddZ ddZ dddZ dddZddZZS)E2EzE2E module. :param int idim: dimension of inputs :param int odim: dimension of outputs :param Namespace args: argument Namespace containing options cCs"t|t|t||S)zAdd arguments.)rencoder_add_argumentsattention_add_argumentsdecoder_add_arguments)parserrrr add_arguments4s   zE2E.add_argumentscC|d}t|}|S)zAdd arguments for the encoder.zE2E encoder setting)add_argument_groupr r!grouprrrr< zE2E.encoder_add_argumentscCr#)z Add arguments for the attention.zE2E attention setting)r$r r%rrrrCr'zE2E.attention_add_argumentscCr#)zAdd arguments for the decoder.zE2E decoder setting)r$r r%rrrr Jr'zE2E.decoder_add_argumentsc s(tt|tjj|t||j}|j|_|j |_ t |dd|_ |j |_ |j |_ |j |_|j|_t|_|d|_|d|_d|_t|ddd|_|jrhtj|jrhtd|jt||j|jd }nd}t |d d |_ t |d d |_!tjj"||j#|jd |_$tjj%|j&d|_'t(||j#|j|_)t*||_+t,|||j|j|j+||_-|j.r||krt/d|j#|j0krt/d|j-j$j1|j$_1|j2r|j3rt/d|j-j$j1|j-j4_1|5|j6r|j7|j8d|j9|j:|j;|j<|j=|j |jd d }t>j?di||_@|j6|_6nd |_6d|_When using tie_src_tgt_embedding, idim and odim must be equal.zBWhen using tie_src_tgt_embedding, eunits and dunits must be equal.z?When using tie_classifier, context_residual must be turned off.) beam_sizepenalty ctc_weight maxlenratio minlenratio lm_weightrnnlmnbestspaceblanktgt_langg _r)Dsuperr__init__torchnnModulerr"etypeverbosegetattrr(outdir sym_spacer; sym_blankr<rrsoseospadr subsamplelsm_typeospathisfile train_jsonlogginginforr/r0 EmbeddingeunitsembedDropout dropout_ratedropoutrencr attr dectie_src_tgt_embedding ValueErrordunitsweighttie_classifiercontext_residualoutputinit_like_fairseq report_bleur3r4r6r7r8r9r:argparse Namespace trans_argslogzerorr)ridimodimargs labeldistrh __class__rrr?Qs        z E2E.__init__cCspt|tjj|jjddtjj|jj|jdtjj|j jjddtjj|j jj|jddS)znInitialize weight like Fairseq. Fairseq basically uses W, b, EmbedID.W ~ Uniform(-0.1, 0.1), gg?rN) rr@rAinituniform_rVr` constant_rKr\rrrrrds  zE2E.init_like_fairseqcs|||\}}||\}}}|||\___js+j s/d_ nvd}j |t ||jjj}g} g} dd|D} t| D]I\} } || }fdd| D}fdd|D}d|jjd}|jjd}d|jjd}| |dg7} | |dgg7} qQtj| | d _ tj}t|sj|jjj jSt d |jS) aEE2E forward. :param torch.Tensor xs_pad: batch of padded input sequences (B, Tmax, idim) :param torch.Tensor ilens: batch of lengths of input sequences (B) :param torch.Tensor ys_pad: batch of padded token id sequence tensor (B, Lmax) :return: loss value :rtype: torch.Tensor gNcSs g|] }|ddddqS)ryseqr)r).0 nbest_hyprrr s zE2E.forward..c&g|]}t|dkrjt|qSruintr(rvidxrsrrrxs&cryrzr{r}rsrrrxs  dzloss (=%f) is not correct)!target_language_biasingrZrYrVr\rrrtrainingrerrecognize_beam_batchr@tensorrhr(r9 enumeratejoinreplacer;r<splitnltk bleu_score corpus_bleufloatmathisnanrrrRwarning)rxs_padilensys_padhs_padhlens_lpz nbest_hyps list_of_refshypsy_hatsiy_haty_trueseq_hatseq_true seq_hat_text seq_true_text loss_datarrsrforwardsF       z E2E.forwardcCs^|jr+|dddfd}|ddddf}|ddddf}tj||gdd}||fS)a$Prepend target language IDs to source sentences for multilingual MT. These tags are prepended in source/target sentences as pre-processing. :param torch.Tensor xs_pad: batch of padded input sequences (B, Tmax, idim) :param torch.Tensor ilens: batch of lengths of input sequences (B) :return: source text without language IDs :rtype: torch.Tensor :return: target text without language IDs :rtype: torch.Tensor :return: target language IDs :rtype: torch.Tensor (B, 1) Nrr))dim)r/ unsqueezer@cat)rrrr tgt_lang_idsrrrrs zE2E.target_language_biasingNc Cs|j}||jr-t|dddg}t|ttjt t |dddtj d}nt|dg}t|ttjt t |dtj d}| | ||d|\}} } |j|dd|||} |rk|| S)aOE2E beam search. :param ndarray x: input source text feature (B, T, D) :param Namespace trans_args: argument Namespace containing options :param list char_list: list of characters :param torch.nn.Module rnnlm: language model module :return: N-best decoding results :rtype: list rr)Ndtype)revalr/lenrr@ from_numpynpfromitermapr|int64rZrYrVrr\recognize_beamtrain) rxrhr(r9previlenhhsryrrr translates  ( $z E2E.translatec sj}jr!tjdd|Dtjd}fdd|D}ntjdd|Dtjd}fdd|D}t|j} ||\} } } t t tt| } j| | d|||} |rg| S) a~E2E batch beam search. :param list xs: list of input source text feature arrays [(T_1, D), (T_2, D), ...] :param Namespace trans_args: argument Namespace containing options :param list char_list: list of characters :param torch.nn.Module rnnlm: language model module :return: N-best decoding results :rtype: list css |] }t|ddVqdSr)Nrrvxxrrr Msz&E2E.translate_batch..rc s$g|]}tt|ddqSrrr@rrrsrrrxNs$z'E2E.translate_batch..css|]}t|VqdSNrrrrrrPscsg|] }tt|qSrrrrsrrrxQsN)rrr/rrrrrKrZrYrVr@rlistrr|r\rr) rxsrhr(r9rrrxpadrrrrrrsrtranslate_batch=s    zE2E.translate_batchcCs||t(||||\}}|||||\}}}|j|||}Wdn1s3wY| |S)aE2E attention calculation. :param torch.Tensor xs_pad: batch of padded input sequences (B, Tmax, idim) :param torch.Tensor ilens: batch of lengths of input sequences (B) :param torch.Tensor ys_pad: batch of padded token id sequence tensor (B, Lmax) :return: attention weights with the following shape, 1) multi-head case => attention weights (B, H, Lmax, Tmax), 2) other case => attention weights (B, Lmax, Tmax). :rtype: float ndarray N) rr@no_gradrrZrYrVr\calculate_all_attentionsr)rrrrhpadrratt_wsrrrr_s zE2E.calculate_all_attentionsr)rrrr staticmethodr"rrr r?rdrrrrr __classcell__rrrnrr+s"     k 9  #"r)&rrfrRrrNchainerrnumpyrr@respnet.nets.e2e_asr_commonrespnet.nets.mt_interfacer*espnet.nets.pytorch_backend.initializationr&espnet.nets.pytorch_backend.nets_utilsrrr(espnet.nets.pytorch_backend.rnn.argumentr r r *espnet.nets.pytorch_backend.rnn.attentionsr (espnet.nets.pytorch_backend.rnn.decodersr (espnet.nets.pytorch_backend.rnn.encodersrespnet.utils.fill_missing_argsrChainrrArBrrrrrs*