o i@sddlZddlZddlmZddlmmZddlm Z Gdddej Z e ddddZ d d Z e ddd d Zd dZdS)N)jitcs*eZdZdZfddZdddZZS)AlignmentModulezmAlignment Learning Framework proposed for parallel TTS models in: https://arxiv.org/abs/2108.10447 csrttj||ddd|_tj||ddd|_tj||ddd|_tj||ddd|_tj||ddd|_dS)N) kernel_sizepaddingr) super__init__nnConv1dt_conv1t_conv2f_conv1f_conv2f_conv3)selfadimodim __class__S/home/ubuntu/.local/lib/python3.10/site-packages/espnet2/gan_tts/jets/alignments.pyr s zAlignmentModule.__init__NcCs|dd}t||}||}|dd}|dd}t||}t||}||}|dd}|d|d}t j |ddd}| }|dur`|d}| |t j }tj|dd}|S) aWCalculate alignment loss. Args: text (Tensor): Batched text embedding (B, T_text, adim). feats (Tensor): Batched acoustic feature (B, T_feats, odim). x_masks (Tensor): Mask tensor (B, T_text). Returns: Tensor: Log probability of attention matrix (B, T_feats, T_text). rr)pdimN)r) transposeFrelur r rrr unsqueezetorchnorm masked_fillnpinf log_softmax)rtextfeatsx_masksdistscore log_p_attnrrrforwards"       zAlignmentModule.forward)N)__name__ __module__ __qualname____doc__r r- __classcell__rrrrr s  rT)nopythonc CsJ|jd}|jd}tj||ftj d}|dd}t|D]}|dd|df|d|f<qtd|D],}tdt|d|D]}t||d|df|||df|||f|||f<qCq7tj|f|dd}t|dddD],}||dd}||d} | dkrd} n|||f|| |fkr|} n| } | ||<qv|S)Nrr) fill_valuerr) shaper$fullr%rrangesumminmax) r,T_melT_inpQlog_probjiAi_ai_bargmax_irrr_monotonic_alignment_search>s*    "<  rEc Cs|d}|d}|j}d}tj||f|d}t|D]A}||d||d||f} t| } t | } t |  |||dt | f<t||} || | | f}q||}||fS)aExtract duration from an attention probability matrix Args: log_p_attn (Tensor): Batched log probability of attention matrix (B, T_feats, T_text). text_lengths (Tensor): Text length tensor (B,). feats_legnths (Tensor): Feature length tensor (B,). Returns: Tensor: Batched token duration extracted from `log_p_attn` (B, T_text). Tensor: Binarization loss tensor (). rr)deviceN)sizerFr!zerosr7rEdetachcpunumpyr$bincount from_numpytolenarangemean) r, text_lengths feats_lengthsBT_textrFbin_lossdsbcur_log_p_attnviterbi_dst_idxrrrviterbi_decode_s     r]cCs|jd}t|}|tj}t|D]X}||}||}||d|f} | } dgt| } ||d|f} tt | dd| ddD]#\} \} }t | | |dkre| | | ||| f<qHd||| f<qHq|S)Nrrr) r5r$ zeros_likeastypeint32r7cumsumlist enumerateziprOrQ)rWxsrRrSrTxs_avgrXt_textt_featsdd_cumsumxnstartendrrr_average_by_durations    *rocCs<|j}||||g}dd|D}t|}t||}|S)aAverage frame-level features into token-level according to durations Args: ds (Tensor): Batched token duration (B, T_text). xs (Tensor): Batched feature sequences to be averaged (B, T_feats). text_lengths (Tensor): Text length tensor (B,). feats_lengths (Tensor): Feature length tensor (B,). Returns: Tensor: Batched feature averaged according to the token duration (B, T_text). cSsg|] }|qSr)rIrJrK).0argrrr sz'average_by_duration..)rFror!rMrN)rWrerRrSrFargsrfrrraverage_by_durations  rt)rKr$r!torch.nnr torch.nn.functional functionalrnumbarModulerrEr]rortrrrrs  3