o i'@sldZddlZddlmZddlmZddlZddlZddlm Z ddl m Z ddl m Z Gdd d e ZdS) z2Parallel beam search module for online simulation.N)Path)List)BatchBeamSearch) Hypothesis) end_detectc @seZdZdZdefddZdefddZdefd d Zd efd d Z dde j de de de efddZde j dede efddZdS)BatchBeamSearchOnlineSimaoOnline beam search implementation. This simulates streaming decoding. It requires encoded features of entire utterance and extracts block by block from it as it shoud be done in streaming processing. This is based on Tsunoo et al, "STREAMING TRANSFORMER ASR WITH BLOCKWISE SYNCHRONOUS BEAM SEARCH" (https://arxiv.org/abs/2006.14941). asr_configcCst|}d|_d|_d|_d}|jddd]}t|}d|vrPd|dvr1|dd|_d|dvr@|dd|_d|dvrO|dd|_nd |vrl|d }|durlt d  WddSWdn1svwY|jdus|jdus|jdur|durt|}|jddd }t|}Wdn1swYd|vr|d}|rd|vr|d|_|rd|vr|d|_|rd|vr|d|_dSdSdSdSdS) zSet config file for streaming decoding. Args: asr_config (str): The config file for asr training Nrzutf-8)encoding encoder_conf block_sizehop_size look_aheadconfigzPCannot find config file for streaming decoding: apply batch beam search instead.) rr r ropenyaml safe_loadkeyslogginginfo)selfrtrain_config_filerfargs config_fileenc_argsr\/home/ubuntu/.local/lib/python3.10/site-packages/espnet/nets/batch_beam_search_online_sim.pyset_streaming_configsT           z-BatchBeamSearchOnlineSim.set_streaming_configr cC ||_dS)zvSet block size for streaming decoding. Args: block_size (int): The block size of encoder N)r )rr rrrset_block_sizeG z'BatchBeamSearchOnlineSim.set_block_sizer cCr)zpSet hop size for streaming decoding. Args: hop_size (int): The hop size of encoder N)r )rr rrr set_hop_sizeOr!z%BatchBeamSearchOnlineSim.set_hop_sizercCr)zSet look ahead size for streaming decoding. Args: look_ahead (int): The look ahead size of encoder N)r)rrrrrset_look_aheadWr!z'BatchBeamSearchOnlineSim.set_look_aheadx maxlenratio minlenratioreturnc sd_jrjrjrtjj}n|jd}d}||jdkr+|dd|}n|}|dkr7|jd}n tdt||d}t||d}t dt |jdt dt |t dt | |} g} g} d} d} | rd}||jdkr|dd|}n|} || ||krt dt || |}||dkr||||| } |jjd}g}|jt||jdfjk}t|jdD]2}||r||}||q| s|j|d f|j|d d fvr||jdkrd}d} q|d kr#td d |D|r#t d|d} nt|dkr4||jdkr4d}|ryjrU|tjtj|jdkrU|tj7}n|jd}t d||dkrxt| dkrxjrx| } |d8}g} nAd} | } ||||| } ||jdkr|D]}| |qt| dkrt dd} nt dt| |d7}||ks| szt| dddd}t|dkrt d|dkrgS||td |dS|d}|jD] \}}t |ddj |dd|j |dd|qt d|j!dt d|j!t|jdt dt|j"d urOt d d!#fd"d |jdd Dd#|S)$aPerform beam search. Args: x (torch.Tensor): Encoded speech feature (T, D) maxlenratio (float): Input length ratio to obtain max output length. If maxlenratio=0.0 (default), it uses a end-detect function to automatically find maximum hypothesis lengths minlenratio (float): Input length ratio to obtain min output length. Returns: list[Hypothesis]: N-best decoding results Trzdecoder input length: zmax output length: zmin output length: Fz position Nr$cSsg|]}|qSr)asdict).0lhrrr sz4BatchBeamSearchOnlineSim.forward..zend detected at zGoing to next block: %dzno hypothesis. Finish decoding.zremained hypotheses: cSs|jS)N)score)r%rrrsz2BatchBeamSearchOnlineSim.forward..)keyreversezOthere is no N-best results, perform recognition again with smaller minlenratio.g?z6.2fz * 3z = z for ztotal log probability: z.2fznormalized log probability: z"total number of ended hypotheses: z best hypo: csg|]}j|qSr) token_list)r,r%rrrr.s )$ conservativer r rintshapenarrowmaxsizerrstrinit_hypextenddebugsearch post_processyseqtorcharangelengtheosrange_selectappendrlensortedwarningforwardscoresitemsweightsr/r5join)rr%r&r' cur_end_frame process_idxhmaxlenminlen running_hyps prev_hyps ended_hyps prev_repeatcontinue_decodemove_to_next_blockbestn_batchlocal_ended_hyps is_local_eosihyp nbest_hypskvrr6rrO_s           "      S .  z BatchBeamSearchOnlineSim.forwardhypscCsL|jD]\}}t|dr||t|dr#||j||j|<qdS)aExtend probabilities and states with more encoded chunks. Args: x (torch.Tensor): The extended encoder output feature hyps (Hypothesis): Current list of hypothesis Returns: Hypothesis: The extended hypothesis extend_prob extend_stateN)scorersrQhasattrrirjstates)rr%rhrfdrrrr@s   zBatchBeamSearchOnlineSim.extendN)r$r$)__name__ __module__ __qualname____doc__r>rr9r r"r#rETensorfloatrrrOr@rrrrrs$ ,   r)rrrpathlibrtypingrrErespnet.nets.batch_beam_searchrespnet.nets.beam_searchrespnet.nets.e2e_asr_commonrrrrrrs