o i )@s*dZddlZddlmZmZmZmZmZddlZ ddl Z ddl m Z ddl mZmZde jdeded e jfd d Zd eed ededeed eef ddZdeedeed efddZdeedeed eefddZdeede jde jded eef ddZdeeeeeeffd ed!ed"ed eeeeeefff d#d$Zdeeeeeeffd"ed eeeeeefffd%d&Zd'e jjfd(d)Zdeed eefd*d+Zdeed,ed eefd-d.Z d/eee jd0ed,ed eee jfd1d2Z!d3d4Z"d;d6ed7e jjd8efd9d:Z#dS)z%get_decoder_input..rcsg|] }tj|gddqS)r)dim)torchcatr)blankrrrs)devicenewrto)r r r r labels_unpad decoder_inputr)rr rget_decoder_input s r aux_layer_idenc_num_layersuse_symm_kl_div_loss subsamplec st|tr|rtdd|Dstdt|tdd}ttfdd|}||kr0td|rj|g7}td t|D]+}|||d d ||d }d d |D}d|vritd ||d ||fq>|S) aCheck whether provided auxiliary encoder layer IDs are valid. Return the valid list sorted with duplicates removed. Args: aux_layer_id: Auxiliary encoder layer IDs. enc_num_layers: Number of encoder layers. use_symm_kl_div_loss: Whether symmetric KL divergence loss is used. subsample: Subsampling rate per layer. Returns: valid: Valid list of auxiliary encoder layers. css|]}t|tVqdSN) isinstanceintrlayerrrr <sz2valid_aux_encoder_output_layers..zlaux-transducer-loss-enc-output-layers option takes a list of layer IDs. Correct argument format is: '[0, 1]'Fkeyreversecsd|ko kSS)Nrrxr rrDrz1valid_aux_encoder_output_layers..zgProvided argument for aux-transducer-loss-enc-output-layers is incorrect. IDs should be between [0, %d]cSsg|] }|dkr dndqS)r0FTr)rnrrrrQz3valid_aux_encoder_output_layers..zEncoder layers %d and %d have different shape due to subsampling. Symmetric KL divergence loss doesn't cover such case for now.) r$listall ValueErrorsortedr%filterrangelen) rr r!r" sorted_listvalidr1 sub_range valid_shaperr.rvalid_aux_encoder_output_layers%s<  r>r-prefcCsHt|t|kr dStt|dddD] }||||kr!dSqdS)zCheck if pref is a prefix of x. Args: x: Label ID sequence. pref: Prefix label ID sequence. Returns: : Whether pref is a prefix of x. Fr0T)r9r8)r-r?irrr is_prefix]s rBsubsetcs4g}|D]tfdd|Drq|q|S)zRemove elements of subset if corresponding label ID sequence already exist in x. Args: x: Set of hypotheses. subset: Subset of x. Returns: final: New set of hypotheses. c3s|] }j|jkVqdSr#yseq)rsubx_rrr(szsubtract..)anyappend)r-rCfinalrrGrsubtractrs   rLhyps topk_idxs topk_logpsgammacsvg}t|D]2\}fddt||||D}t|dddd|ttfdd|ddd d q|S) aReturn K hypotheses candidates for expansion from a list of hypothesis. K candidates are selected according to the extended hypotheses probabilities and a prune-by-value method. Where K is equal to beam_size + beta. Args: hyps: Hypotheses. topk_idxs: Indices of candidates hypothesis. topk_logps: Log-probabilities for hypotheses expansions. gamma: Allowed logp difference for prune-by-value method. Return: k_expansions: Best K expansion hypotheses candidates. cs&g|]\}}t|jt|fqSr)r%scorefloat)rkv)hyprrrsz'select_k_expansions..cS|dSNr0rr,rrrr/z%select_k_expansions..)r*r0cs|dkSrWrr,)rP k_best_exprrr/scSrVrWrr,rrrr/rXTr)) enumeratezipmaxrJr6r7)rMrNrOrP k_expansionsrAhyp_ir)rPrUrYrselect_k_expansionss r_ lm_statesidx lm_layers is_wordlmcsP|r}|Si}fddt|D|d<fddt|D|d<|S)aGet ID state from LM hidden states. Args: lm_states: LM hidden states. idx: LM state ID to extract. lm_layers: Number of LM layers. is_wordlm: Whether provided LM is a word-level LM. Returns: idx_state: LM hidden state for given ID. cg|] }d|qScrr&rar`rrrr2z#select_lm_state..rfcrdhrr&rgrrrr2rir8)r`rarbrc idx_staterrgrselect_lm_statesrlcsD|rSi}fddt|D|d<fddt|D|d<|S)zCreate LM hidden states. Args: lm_states: LM hidden states. lm_layers: Number of LM layers. is_wordlm: Whether provided LM is a word-level LM. Returns: new_states: LM hidden states. c$g|]tfddDqS)cg|]}|dqSrerrstater'rrrr5create_lm_batch_states...rstackrr`rqrrz*create_lm_batch_states..rfcrm)crnrhrrorqrrrrrrrsrurvrqrrrwrirj)r`rbrc new_statesrrvrcreate_lm_batch_statess   rylm_modelcsjt|j}|j}|jt|fddt|D}d|i}|dkr3fddt|D|d<|S)zInitialize LM hidden states. Args: lm_model: LM module. Returns: lm_state: Initial LM hidden states. c$g|]}tjjjdqS)rdtyperzerosrrr}r_lm_unitsprrrrwz!init_lm_state..rilstmcr{r|r~rrrrrrwrf)r9rnntypn_unitsnext parametersr8)rzrb lm_units_typrilm_staterrr init_lm_states     rcCs\g}|D]'}dd|D}|j|vr&||j}t||j|j||_q||q|S)zRecombine hypotheses with same label ID sequence. Args: hyps: Hypotheses. Returns: final: Recombined hypotheses. cSsg|]}|jr|jqSrrD)rfrrrrrz"recombine_hyps..)rEindexnp logaddexprQrJ)rMrKrU seq_finalseq_posrrrrecombine_hyps s    rpad_idcs*tdd|Dfdd|D}|S)zLeft pad label ID sequences. Args: labels: Label ID sequence. pad_id: Padding symbol ID. Returns: final: Padded label ID sequences. css|]}t|VqdSr#r9rr-rrrr(1zpad_sequence..cs"g|] }gt||qSrrrmaxlenrrrr3"z pad_sequence..)r\)r rrKrrr pad_sequence&s rrpmax_lenc sdus|dksdd|krSdd}||krCtdd|}tD]\}}|dd|dddf|<q-St}dd}d||ffddt|D} tD]\}}|| |dd||d|ddf<qd| S)a!Check decoder hidden states and left pad or trim if necessary. Args: state: Decoder hidden states. [N x (?, D_dec)] max_len: maximum sequence length. pad_id: Padding symbol ID. Returns: final: Decoder hidden states. [N x (1, max_len, D_dec)] Nr0rcs"g|] }djjqS)r)datarfill_r final_dimsrrprrrUrzcheck_state..)sizer%rZr9r8) rprrcurr_lentrim_valrAslayersddimrKrrr check_state8s"   *rcCst|||ddf}|djj||}t|D]0\}}|d}||kr6||||||ddf<q|||dddf||ddddf<q|S)a&Check decoder hidden states and left pad or trim if necessary. Args: state: Decoder hidden states. [N x (B, ?, D_dec)] max_len: maximum sequence length. pad_id: Padding symbol ID. Returns: final: Decoder hidden states. [N x (B, max_len, dec_dim)] rr0N)r9rrrrrZ)statesrrrrKrArrrrrcheck_batch_states_s  ,rT model_pathmodeltrainingcshdtj|vrtj|dddd}n tj|ddd}|s,dfdd |D}||~d S) zLoad Transducer model with training-only modules and parameters removed. Args: model_path: Model path. model: Transducer model. snapshotcS|Sr#rstoragelocrrrr/z#custom_torch_load..) map_locationrcSrr#rrrrrr/r)mlpctc_linkl_divlm_linerror_calculatorcs,i|]\}tfddDs|qS)c3s|]}|vVqdSr#r)rmodrSrrr(rz/custom_torch_load...)rI)rrT task_keysrr s z%custom_torch_load..N)ospathbasenamerloaditemsload_state_dict)rrrmodel_state_dictrrrcustom_torch_loadys   r)T)$__doc__rtypingrrrrrnumpyrr&espnet.nets.pytorch_backend.nets_utilsr(espnet.nets.transducer_decoder_interfacerr Tensorr%rboolr>rBrLrRr_strrlrynnModulerrrrrrrrrrs   8  (      '