o ip3@spdZddlZddlmZmZmZmZmZddlZddl m Z ddl m Z m Z GdddeZGdd d e ZdS) zParallel beam search module.N)AnyDictList NamedTupleTuple) pad_sequence) BeamSearch Hypothesisc@seZdZUdZegZejed<egZ ejed<egZ ejed<e Z e eejfed<e Ze ee fed<defdd Zd S) BatchHypothesisz*Batchfied/Vectorized hypothesis data type.yseqscorelengthscoresstatesreturncCs t|jS)zReturn a batch size.)lenr selfrQ/home/ubuntu/.local/lib/python3.10/site-packages/espnet/nets/batch_beam_search.py__len__s zBatchHypothesis.__len__N)__name__ __module__ __qualname____doc__torchtensorr Tensor__annotations__r r dictrrstrrintrrrrrr s r c @speZdZdZdeedefddZdedeedefddZ ded edefd d Z d edeefd dZ de j de j dee j e j e j e j ffddZde j defddZdede j deeee j feeefffddZdede j de j deeee j feeefffddZdedededefddZdede j defd d!Zd ed"ed#eded$eedef d%d&Zd'S)(BatchBeamSearchz!Batch beam search implementation.hypsrcstdkr tSttddDd|jdtjddDtjdtddDfd d |jDfd d |jDd S) zConvert list to batch.rcSg|]}|jqSr)r .0hrrr #z+BatchBeamSearch.batchfy..T) batch_first padding_valuecSsg|]}t|jqSr)rr r%rrrr(%)dtypecSr$r)r r%rrrr(&r)cs&i|]tfddDqS)cg|]}|jqSrrr%krrr('r,6BatchBeamSearch.batchfy...)rrr&r#r0r 's&z+BatchBeamSearch.batchfy..cs i|] fddDqS)cr.r)rr%r0rrr((r,r2rr3r4r0rr5(s )r r r rr)rr reosrrint64scorers)rr#rr4rbatchfys zBatchBeamSearch.batchfyidscsNt|j|j|jfdd|jDfdd|jDdS)Nci|] \}}||qSrrr&r1v)r:rrr50z1BatchBeamSearch._batch_select..cs(i|]\fddDqS)csg|] }j|qSrr8 select_state)r&i)r1rr=rrr(2sz..rr3r:r)r1r=rr51s)r r r rr)r r r r ritemsr)rr#r:rrBr _batch_select+s zBatchBeamSearch._batch_selectrAcsTt|jd|jf|jfdd|jDfdd|jDdS)Ncr;rrr<rArrr5;r>z+BatchBeamSearch._select..c$i|]\}}|j||qSrr?r<rArrrr5<sr r rr)r r r r rrCr)rr#rArrGr_select7s zBatchBeamSearch._select batch_hypscsfddttjDS)zRevert batch to list.c s^g|]+tjdjjfddjDfddjDdqS)Ncsi|] }|j|qSrr/)r&r1rJrArrr5Gsz8BatchBeamSearch.unbatchfy...cs$i|]\}}||j|qSr)r@rr<rKrrr5HrH)r r r r r8rCr3rJrrErr(Cs  z-BatchBeamSearch.unbatchfy..)rangerr )rrJrrMr unbatchfyAs  zBatchBeamSearch.unbatchfyweighted_scorescCs6|d|jd}||j}||j}||||fS)aiBatch-compute topk full token ids and partial token ids. Args: weighted_scores (torch.Tensor): The weighted sum scores for each tokens. Its shape is `(n_beam, self.vocab_size)`. ids (torch.Tensor): The partial token ids to compute topk. Its shape is `(n_beam, self.pre_beam_size)`. Returns: Tuple[torch.Tensor, torch.Tensor, torch.Tensor, torch.Tensor]: The topk full (prev_hyp, new_token) ids and partial (prev_hyp, new_token) ids. Their shapes are all `(self.beam_size,)` )viewtopk beam_sizen_vocab)rrPr:top_ids prev_hyp_ids new_token_idsrrr batch_beamPs   zBatchBeamSearch.batch_beamxc Cs^t}t}|jD]\}}||||<d||<q |td||tj|jg|j ddgS)zGet an initial hypothesis data. Args: x (torch.Tensor): The encoder output feature Returns: Hypothesis: The initial hypothesis. g)device)r rrr ) rr8rCbatch_init_stater9r rrsosr\)rr[ init_states init_scoresr1drrrinit_hypjs  zBatchBeamSearch.init_hyphypcCsJt}t}|jD]\}}||j|j||\||<||<q ||fS)aScore new hypothesis by `self.full_scorers`. Args: hyp (Hypothesis): Hypothesis with prefix tokens to score x (torch.Tensor): Corresponding input feature Returns: Tuple[Dict[str, torch.Tensor], Dict[str, Any]]: Tuple of score dict of `hyp` that has string keys of `self.full_scorers` and tensor score values of shape: `(self.n_vocab,)`, and state dict that has string keys and state values of `self.full_scorers` )r full_scorersrC batch_scorer r)rrcr[rrr1rarrr score_fulls $zBatchBeamSearch.score_fullcCsLt}t}|jD]\}}||j||j||\||<||<q ||fS)aeScore new hypothesis by `self.full_scorers`. Args: hyp (Hypothesis): Hypothesis with prefix tokens to score ids (torch.Tensor): 2D tensor of new partial tokens to score x (torch.Tensor): Corresponding input feature Returns: Tuple[Dict[str, torch.Tensor], Dict[str, Any]]: Tuple of score dict of `hyp` that has string keys of `self.full_scorers` and tensor score values of shape: `(self.n_vocab,)`, and state dict that has string keys and state values of `self.full_scorers` )r part_scorersrCbatch_score_partialr r)rrcr:r[rrr1rarrr score_partialszBatchBeamSearch.score_partialr part_statespart_idxcCs>t}|D]\}}|||<q|D]\}}|||<q|S)aMerge states for new hypothesis. Args: states: states of `self.full_scorers` part_states: states of `self.part_scorers` part_idx (int): The new token id for `part_scores` Returns: Dict[str, torch.Tensor]: The new score dict. Its keys are names of `self.full_scorers` and `self.part_scorers`. Its values are states of the scorers. )rrC)rrrjrk new_statesr1r=rrr merge_statess   zBatchBeamSearch.merge_states running_hypscst|}d}tj|j|j|jd}||j|g|jR\}}j D] }|j |||7}q%j rMj dkr=|n|j } tj | jddd}|||\} } jD] }|j || |7}qY||jj|j|jdd7}g} |} t||D]R\}| }| t||f|j||jfdd|D|fd d| Dfd d|Dfd d| Dd q| S) a Search new tokens for running hypotheses and encoded speech x. Args: running_hyps (BatchHypothesis): Running hypotheses on beam x (torch.Tensor): Encoded speech feature (T, D) Returns: BatchHypothesis: Best sorted hypotheses N)r-r\fullrQ)dimrRcr;rrr<)full_prev_hyp_idrrr5r>z*BatchBeamSearch.search..cr;rrr<)part_prev_hyp_idrrr5r>crFr)rdr@r<)rqrrrr5 rLcs&i|]\}}|j||qSr)rgr@r<)part_new_token_idrrrrrr5 s  )r r rr) rrzerosrVr-r\rfexpandshaperdweights do_pre_beampre_beam_score_keyrT pre_beam_sizerirgr to unsqueezerOziprZappendr append_tokenr merge_scoresrrCrmr9)rrnr[n_batchpart_idsrPrrr1pre_beam_scores part_scoresrj best_hyps prev_hypsfull_new_token_idprev_hypr)rqrsrrrrsearchsl          zBatchBeamSearch.searchmaxlen maxlenratio ended_hypsc s@|jjd}td|jdur.tddfdd|jdd|jdfD||dkrftd t |jtj |dfj |jj tj d fd}|j|||jdd<|jd|jdd<|jt||jdfj k}tj|d d d D] } || } || qtj|dkd d d } || S)aPerform post-processing of beam search iterations. Args: i (int): The length of hypothesis tokens. maxlen (int): The maximum length of tokens in beam search. maxlenratio (int): The maximum length ratio in beam search. running_hyps (BatchHypothesis): The running hypotheses in beam search. ended_hyps (List[Hypothesis]): The ended hypotheses in beam search. Returns: BatchHypothesis: The new running hypotheses. rz the number of running hypothes: Nz best hypo: csg|]}j|qSr) token_list)r&r[rrrr(4sz0BatchBeamSearch.post_process..rRz-adding in the last position in the loop)r\r-F)as_tuplerQ)r rvloggingdebugrjoinr inforcatror6r\r7 resize_as_arangenonzerorSrIr~rD) rrArrrnrryseq_eosis_eosbrc remained_idsrrr post_processsJ         zBatchBeamSearch.post_processN)rrrrrr r r9r!rDrIrOrrrrZrbrr rrfrirmrfloatrrrrrr"sZ     Or")rrtypingrrrrrrtorch.nn.utils.rnnrespnet.nets.beam_searchrr r r"rrrrs