o ic@sdZddlmZddlZddlZddlmmZ ddl m Z ddl m Z ddlmZddlmZGdd d ejjZGd d d ejjZdS) z%JETS related loss module for ESPnet2.)TupleN) betabinom)check_argument_types)DurationPredictorLoss)make_non_pad_maskcsteZdZddedeffdd Zdejdejd ejd ejd ejd ejd ejdeejejejejffddZZ S) VarianceLossTF use_maskinguse_weighted_maskingcs^tsJt||ks|rJ||_||_|jrdnd}tjj|d|_t |d|_ dS)aInitialize JETS variance loss module. Args: use_masking (bool): Whether to apply masking for padded part in loss calculation. use_weighted_masking (bool): Whether to weighted masking in loss calculation. nonemean) reductionN) rsuper__init__rr torchnnMSELoss mse_criterionrduration_criterion)selfrr r  __class__M/home/ubuntu/.local/lib/python3.10/site-packages/espnet2/gan_tts/jets/loss.pyrs zVarianceLoss.__init__d_outsdsp_outspse_outsesilensreturncCs$|jr4t||j}||}||}t|d|j} || }|| }|| }|| }|||} |||} |||} |jrt||j}| |j ddd } | | d} | | | } |d} | d}| ||  } | ||  } | | | fS)aCalculate forward propagation. Args: d_outs (LongTensor): Batch of outputs of duration predictor (B, T_text). ds (LongTensor): Batch of durations (B, T_text). p_outs (Tensor): Batch of outputs of pitch predictor (B, T_text, 1). ps (Tensor): Batch of target token-averaged pitch (B, T_text, 1). e_outs (Tensor): Batch of outputs of energy predictor (B, T_text, 1). es (Tensor): Batch of target token-averaged energy (B, T_text, 1). ilens (LongTensor): Batch of the lengths of each input (B,). Returns: Tensor: Duration predictor loss value. Tensor: Pitch predictor loss value. Tensor: Energy predictor loss value. T)dimkeepdimr) rrtodevice masked_select unsqueezerrr floatsumsizemul)rrrrrrrrduration_masks pitch_masks duration_loss pitch_loss energy_lossduration_weights pitch_weightsrrrforward+s2            zVarianceLoss.forward)TF) __name__ __module__ __qualname__boolrrTensorrr4 __classcell__rrrrrs& rc sjeZdZdZddeffdd Zejdfdej dej d ej d e d ej f d d Z dd ej fddZ Z S)ForwardSumLosszHForwardsum loss described at https://openreview.net/forum?id=0NQwnnwAORiT cache_priorcst||_i|_dS)zInitialize forwardsum loss module. Args: cache_prior (bool): Whether to cache beta-binomial prior N)r rr<_cache)rr<rrrros  zForwardSumLoss.__init__r! log_p_attnrolens blank_probr c Cs|d}|||}|j|j|jd}||}tj|dt|d}d}t |D];} t d|| d d} || d|| d|| df d} |tj | | || | d|| | ddd7}q)||}|S) aCalculate forward propagation. Args: log_p_attn (Tensor): Batch of log probability of attention matrix (B, T_feats, T_text). ilens (Tensor): Batch of the lengths of each input (B,). olens (Tensor): Batch of the lengths of each target (B,). blank_prob (float): Blank symbol probability. Returns: Tensor: forwardsum loss value. r)dtyper&)r"rrrrr)valuer"NT) log_probstargets input_lengthstarget_lengths zero_infinity)r+_generate_priorr%rAr&Fpadnplograngeraranger(ctc_loss) rr>rr?r@Bbb_prior log_p_attn_pdlossbidx target_seqcur_log_p_attn_pdrrrr4zs0    zForwardSumLoss.forwardr"cs"t|}|}|}tj|||ftj d}t|D]r}||||} tdt| } |j rB| |j vrB|j | } n+|tj ddt d} |t fdd| D} t | }|d}t|| | | } |j rz| |j vrz| |j | <t| dd} | ||d d | f<q|S) aGenerate alignment prior formulated as beta-binomial distribution Args: text_lengths (Tensor): Batch of the lengths of each input (B,). feats_lengths (Tensor): Batch of the lengths of each target (B,). w (float): Scaling factor; lower -> wider the width. Returns: Tensor: Batched 2d static prior matrix (B, T_feats, T_text). ) fill_value,r")rAcsg|]}|dqSr"r).0tTrr sz2ForwardSumLoss._generate_prior..).NrN)lenmaxrfullrKinfrMitemstrr<r=rNr)arrayrlogpmf from_numpy transpose)r text_lengths feats_lengthswrPT_textT_featsrQrTNkeyprobalphabetak batched_krr\rrHs(       zForwardSumLoss._generate_prior)TrY)r5r6r7__doc__r8rrKerr9r)r4rHr:rrrrr;ls  4r;)rutypingrnumpyrKrtorch.nn.functionalr functionalrI scipy.statsr typeguardr9espnet.nets.pytorch_backend.fastspeech.duration_predictorr&espnet.nets.pytorch_backend.nets_utilsrModulerr;rrrrs     X