o i'@sdZddlmZmZmZmZddlZddlmm Z ddl m Z Gdddejj ZGdddejj ZGd d d ejj ZGd d d ejj ZdS) zkHiFiGAN-related loss modules. This code is modified from https://github.com/kan-bayashi/ParallelWaveGAN. )ListOptionalTupleUnionN) LogMelFbankcspeZdZdZ  ddedeffdd Zdeeee j ee j e j fd e j fd d Z d d Z ddZ ZS)GeneratorAdversarialLossz"Generator adversarial loss module.Tmseaverage_by_discriminators loss_typecsFt||_|dvsJ|d|dkr|j|_dS|j|_dS)zInitialize GeneratorAversarialLoss module. Args: average_by_discriminators (bool): Whether to average the loss by the number of discriminators. loss_type (str): Loss type, "mse" or "hinge". rhinge is not supported.rN)super__init__r _mse_loss criterion _hinge_lossselfr r  __class__P/home/ubuntu/.local/lib/python3.10/site-packages/espnet2/gan_tts/hifigan/loss.pyrs   z!GeneratorAdversarialLoss.__init__outputsreturncCslt|ttfr/d}t|D]\}}t|ttfr|d}|||7}q |jr-||d}|S||}|S)aOCalcualate generator adversarial loss. Args: outputs (Union[List[List[Tensor]], List[Tensor], Tensor]): Discriminator outputs, list of discriminator outputs, or list of list of discriminator outputs.. Returns: Tensor: Generator adversarial loss value. ) isinstancetuplelist enumeraterr )rradv_lossioutputs_rrrforward*s  z GeneratorAdversarialLoss.forwardcCt|||SNFmse_lossnew_onessizerxrrrrGz"GeneratorAdversarialLoss._mse_losscCs | Sr')meanr-rrrrJs z$GeneratorAdversarialLoss._hinge_lossTr)__name__ __module__ __qualname____doc__boolstrrrrtorchTensorr%rr __classcell__rrrrrs  rcseZdZdZ  ddedeffdd Zdeeee j ee j e j fd eeee j ee j e j fd e e j e j ffd d Z d e j d e j fddZ d e j d e j fddZd e j d e j fddZd e j d e j fddZZS)DiscriminatorAdversarialLossz&Discriminator adversarial loss module.Trr r csVt||_|dvsJ|d|dkr!|j|_|j|_dS|j|_|j|_dS)zInitialize DiscriminatorAversarialLoss module. Args: average_by_discriminators (bool): Whether to average the loss by the number of discriminators. loss_type (str): Loss type, "mse" or "hinge". r r rN) rrr _mse_fake_lossfake_criterion_mse_real_lossreal_criterion_hinge_fake_loss_hinge_real_lossrrrrrQs   z%DiscriminatorAdversarialLoss.__init__ outputs_hatrrcCst|ttfrId}d}tt||D]#\}\}}t|ttfr'|d}|d}|||7}|||7}q|jrE||d}||d}||fS||}||}||fS)aCalcualate discriminator adversarial loss. Args: outputs_hat (Union[List[List[Tensor]], List[Tensor], Tensor]): Discriminator outputs, list of discriminator outputs, or list of list of discriminator outputs calculated from generator. outputs (Union[List[List[Tensor]], List[Tensor], Tensor]): Discriminator outputs, list of discriminator outputs, or list of list of discriminator outputs calculated from groundtruth. Returns: Tensor: Discriminator real loss value. Tensor: Discriminator fake loss value. rrr)rrr r!zipr?r=r )rrBr real_loss fake_lossr# outputs_hat_r$rrrr%hs    z$DiscriminatorAdversarialLoss.forwardr.cCr&r'r(r-rrrr>r/z+DiscriminatorAdversarialLoss._mse_real_losscCr&r')r)r* new_zerosr,r-rrrr<r/z+DiscriminatorAdversarialLoss._mse_fake_lossc Cs"tt|d|| SNrr8r0minrGr,r-rrrrAs"z-DiscriminatorAdversarialLoss._hinge_real_lossc Cs$tt| d|| SrHrIr-rrrr@s$z-DiscriminatorAdversarialLoss._hinge_fake_lossr1)r2r3r4r5r6r7rrrr8r9rr%r>r<rAr@r:rrrrr;Ns( 'r;cs~eZdZdZ   ddededeffdd Zd eeeej eej fd eeeej eej fd ej fd d Z Z S)FeatureMatchLosszFeature matching loss module.TFaverage_by_layersr include_final_outputscs t||_||_||_dS)aInitialize FeatureMatchLoss module. Args: average_by_layers (bool): Whether to average the loss by the number of layers. average_by_discriminators (bool): Whether to average the loss by the number of discriminators. include_final_outputs (bool): Whether to include the final output of each discriminator for loss calculation. N)rrrLr rM)rrLr rMrrrrs  zFeatureMatchLoss.__init__ feats_hatfeatsrc Csd}tt||D]<\}\}}d}|js |dd}|dd}tt||D]\}\} } |t| | 7}q'|jrA||d}||7}q |jrO||d}|S)a Calculate feature matching loss. Args: feats_hat (Union[List[List[Tensor]], List[Tensor]]): List of list of discriminator outputs or list of discriminator outputs calcuated from generator's outputs. feats (Union[List[List[Tensor]], List[Tensor]]): List of list of discriminator outputs or list of discriminator outputs calcuated from groundtruth.. Returns: Tensor: Feature matching loss value. rNrr)r!rCrMr)l1_lossdetachrLr ) rrNrOfeat_match_lossr# feats_hat_feats_feat_match_loss_j feat_hat_feat_rrrr%s     zFeatureMatchLoss.forward)TTF) r2r3r4r5r6rrrr8r9r%r:rrrrrKs&rKcseZdZdZ         d d ed ededeedededeedeededededeeffdd Z d!de j de j dee j de j fddZ Z S)"MelSpectrogramLosszMel-spectrogram loss."VNhannPrTF$@fsn_fft hop_length win_lengthwindown_melsfminfmaxcenter normalizedonesidedlog_basec s0tt||||||||| | | | d |_dS)aInitialize Mel-spectrogram loss. Args: fs (int): Sampling rate. n_fft (int): FFT points. hop_length (int): Hop length. win_length (Optional[int]): Window length. window (str): Window type. n_mels (int): Number of Mel basis. fmin (Optional[int]): Minimum frequency for Mel. fmax (Optional[int]): Maximum frequency for Mel. center (bool): Whether to use center window. normalized (bool): Whether to use normalized one. onesided (bool): Whether to use oneseded one. log_base (Optional[float]): Log base value. ) r`rarbrcrdrerfrgrhrirjrkN)rrr wav_to_mel) rr`rarbrcrdrerfrgrhrirjrkrrrrs  zMelSpectrogramLoss.__init__y_hatyspecrcCsR||d\}}|dur||d\}}n|j|\}}t||}|S)aCalculate Mel-spectrogram loss. Args: y_hat (Tensor): Generated waveform tensor (B, 1, T). y (Tensor): Groundtruth waveform tensor (B, 1, T). spec (Optional[Tensor]): Groundtruth linear amplitude spectrum tensor (B, n_fft, T). if provided, use it instead of groundtruth waveform. Returns: Tensor: Mel-spectrogram loss value. rN)rlsqueezelogmelr)rP)rrmrnromel_hat_melmel_lossrrrr% s  zMelSpectrogramLoss.forward) rZr[r\Nr]r^rNTFTr_r')r2r3r4r5intrr7r6floatrr8r9r%r:rrrrrYsb     4rY)r5typingrrrrr8torch.nn.functionalnn functionalr)'espnet2.tts.feats_extract.log_mel_fbankrModulerr;rKrYrrrrs <N=