o wid@sjddlZddlmZmZmZmZddlZddlZddl m Z ddl m Z ddl mZmZmZddlmZmZmZmZmZmZddlmZdd gZ   d1d ejdeejdeejdeeeefdededejfddZ   d2dejdejdeejdeejdedejf ddZ     d3dejdejdeejdeejdedededejfdd Z!    !  d4dejdejdeejdeejd"ed#ed$eed%ed&eededejfd'd(Z"Gd)ddeeZ#  d5dejdejdeejdeejdejf d*d+Z$Gd,d d eeZ%  d5dejdejdeejdeejdejf d-d.Z&Gd/d0d0eeZ'dS)6N)ListOptionalTupleUnion)make_seq_mask_like)toeplitz)LossTyping typecheck) AudioSignal LengthsTypeLossTypeMaskType NeuralTypeVoidType)loggingSDRLossMSELossF绽|=input input_lengthmaskdimkeepdimepsreturncCs|dur|dur tdt||ddd}||}|dur'tj|||d}|S||}tj|||d}tj|||d}|||}|S)aGCalculate mean along dimension `dim` with optionally averaging only over valid samples (based on the input length). Args: input: signal, for example (B, C, T) or (B, C, D, T) input_length: Optional, length of each example in the batch, shape (B,) mask: Optional, temporal mask for each example in the batch, same shape as the input signal dim: dimension or dimensions to reduce keepdim: Whether to keep the temporal dimension eps: Regularization to avoid division by zero Returns: Mean over dimensions `dim`. N`Argument `input_length` is mutually exclusive with `mask`. Both cannot be used at the same time.rTlengthsliketime_dim valid_ones)rr) RuntimeErrorr expand_astorchmeansum)rrrrrrr& normalizationr)`/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/nemo/collections/audio/losses/audio.pycalculate_means  r+estimatetargetc Cs|dur|dur tdt||ddd}||}t|||dd|d}tt|d|dd|d}|||}||}|durD||}|S)aCCalculate optimal scale-invariant target. Assumes time dimension is the last dimension in the array. Calculate scaled target obtained by solving min_scale || scale * target - estimate ||^2 for each example in batch and each channel (b, c). Args: estimate: tensor, shape (B, C, T) target: tensor, shape (B, C, T) input_length: optional, length of valid samples, shape (B,) mask: optional, mask for input samples, shape (B, T) eps: regularization constant Returns: Scaled target, shape (B, C, T) NrrTrrrrr)r#rr$r+r%abs) r,r-rrrestimate_dot_target target_powscale target_scaledr)r)r*scale_invariant_targetJs  r5ư>:0yE> filter_lengthdiag_regcCs|dur*|dur tdt||krtdt||t||ddd}||}|dur6||}||}|j}|d|d}|d|d}dt t d|dd}tj j ||d } tj j ||d } tj jt| d|d } tj j| | |d } | d d|f} | d d|f} |dur| d || d |7<t| } tj| | }| tj j ||d }tj j||d }|d d|dfj|}|dur||}|S) aCalculate optimal convolution-invariant target for a given estimate. Assumes time dimension is the last dimension in the array. Calculate target filtered with a linear f obtained by solving min_filter || conv(filter, target) - estimate ||^2 for each example in batch and each channel (b, c). Args: estimate: tensor, shape (B, C, T) target: tensor, shape (B, C, T) input_length: optional, length of valid samples, shape (B,) mask: optional, mask for input samples, shape (B, T) filter_length: length of the (convolutional) filter for target diag_reg: relative diagonal regularization for the linear system eps: absolute regularization for the diagonal Returns: Filtered target, shape (B, C, T) Reference: C. Boeddeker et al., Convolutive Transfer Function Invariant SDR training criteria for Multi-Channel Reverberant Speech Separation, 2021 NrzoCurrent min input_length (%d) is smaller than filter_length (%d). This will result in a singular linear system.rTrr/)n.).r)r#r%minrwarningrr$shapeviewmathceillog2fftrfftirfftr0conjrlinalgsolve)r,r-rrr9r:r input_shapen_fftTEtt_corrte_corrTTfiltT_filt target_filtr)r)r*convolution_invariant_targetzsH!  rTTscale_invariantconvolution_invariantconvolution_filter_length remove_meansdr_maxc CsL|r|rtd|j|jksJd|jd|jd|dur4|dur'tdt||ddd }||}|rL|t||dd| d }|t||dd| d }|sT|r]|d kr]t|||| d }n |rht||||| d }||} tt |d|d| d} tt | d|d| d} |dur| d| d| } | | | } dt | | } | S)aECalculate signal-to-distortion ratio per channel. SDR = 10 * log10( ||t||_2^2 / (||e-t||_2^2 + alpha * ||t||^2) where alpha = 10^(-sdr_max/10) Optionally, use scale- or convolution- invariant target signal. Args: estimate: estimated signal, shape (B, C, T) target: target signal, shape (B, C, T) input_length: Optional, length of valid samples, shape (B,) mask: Optional, temporal mask, shape (B, T) scale_invariant: Use scale invariant SDR convolution_invariant: Use convolution invariant SDR convolution_filter_length: Filter length for convolution invariant SDR remove_mean: If True, mean will be removed before calculating SDR eps: Small regularization constant Returns: SDR in dB for each channel, shape (B, C) zRArguments scale_invariant and convolution_invariant cannot be used simultaneously.Estimate shape () not matching target shape ()NrrTrr.r;)r,r-rr)r,r-rr9rr/)rrr ) ValueErrorr?r#rr$r+r5rTr%r0log10)r,r-rrrUrVrWrXrYr distortionr2distortion_powsdrr)r)r*calculate_sdr_batchsB#  rccseZdZdZ        ddeeed ed ed ed ee d edeedeffdd Z e ddZ e ddZ e  ddejdejdeejdeejdejf ddZZS)raA Computes signal-to-distortion ratio (SDR) loss with weighted average across channels. Args: weight: weight for SDR of each output channel, used for averaging the loss across channels. Defaults to `None` (averaging). reduction: batch reduction. Defaults to `mean` over the batch. scale_invariant: If `True`, use scale-invariant SDR. Defaults to `False`. remove_mean: Remove mean before calculating the loss. Defaults to `True`. sdr_max: Soft thresholding of the loss to SDR_max. eps: Small value for regularization. Nr&Fr6Tr8weight reductionrUrVrWrXrYrc st|dur9tdd|Drtd|tjt|ddds*td|t| dd}t d || d ||||_ |d krMtj|_ntd |d |rb|rbt|jjd||_||_||_||_||_||_dS)NcSg|]}|dkqSrr).0wr)r)r* Iz$SDRLoss.__init__..(Weight must be positive! Current value: r;r7atol*Weight should add to one, current weight: rChannel weight set to %srdr&Unexpected reduction mode .zT: arguments scale_invariant and convolution_invariant cannot be used simultaneously.)super__init__anyr^npiscloser'r%tensorreshaperinforegister_bufferrer&reduce __class____name__rUrVrWrXrYr) selfrdrerUrVrWrXrYrr~r)r*ru:s0      zSDRLoss.__init__cCs>d}t|tt|tttdtddtdtdddS)$Input types definitions for SDRLoss.BCrLrToptionalr,r-rr)rr tupler r)r signal_shaper)r)r* input_typeses   zSDRLoss.input_typescCdttdiS)z%Output types definitions for SDRLoss.loss elements_typerr rr)r)r* output_typespzSDRLoss.output_typesr,r-rrrc Csjt|||||j|j|j|j|j|jd }|jdur!tj |dd}n ||j}tj |dd}| |}| S)aFor input batch of multi-channel signals, calculate SDR between estimate and target for each channel, perform averaging across channels (weighting optional), and apply reduction across the batch. Args: estimate: Batch of signals, shape (B, C, T) target: Batch of signals, shape (B, C, T) input_length: Batch of lengths, shape (B,) mask: Batch of temporal masks for each channel, shape (B, C, T) Returns: Scalar loss. ) r,r-rrrUrVrWrXrYrNr;r) rcrUrVrWrXrYrrdr%r&r'r})rr,r-rrrbr)r)r*forwardus$   zSDRLoss.forward)Nr&FFr6TNr8NN)r __module__ __qualname____doc__rrfloatstrboolintrupropertyrrr r%Tensorr __classcell__r)r)rr*r-sZ  +  cCs|j|jksJd|jd|jd|dur,|durtdt||ddd}||}||}|jd kr8d}n|jd kr@d }ntd |jtt|d ||d}|S)aCalculate MSE per channel. MSE = ||estimate - target||_2^2 / input_length Args: estimate: estimated signal, shape (B, C, T) or (B, C, D, T) target: target signal, shape (B, C, T) or (B, C, D, T) input_length: Optional, length of valid samples, shape (B,) mask: Optional, temporal mask, same shape as signals Returns: MSE for each channel, shape (B, C) rZr[r\NrrTrr#Unexpected dimension of the input: r/rrr?r#rr$ndimr+r%r0r,r-rrerrrmser)r)r*calculate_mse_batchs$   rc eZdZdZ   ddeeededeffdd Z e d d Z e d d Z e  ddejdejdeejdeejdejf ddZZS)raQ Computes MSE loss with weighted average across channels. Args: weight: weight for loss of each output channel, used for averaging the loss across channels. Defaults to `None` (averaging). reduction: batch reduction. Defaults to `mean` over the batch. ndim: Number of dimensions for the input signal Nr&rrdrerc0t|dur9tdd|Drtd|tjt|ddds*td|t| dd}t d || d ||||_ |d krMtj|_ntd |d ||_|jdkrad|_n|jdkrjd|_ntd|jt d|jjt d|jt d|j t d|jt d|jdS)NcSrfrgr)rhr)r)r*rkrlz$MSELoss.__init__..rmr;r7rnrprrqrdr&rrrsrrrrrDrLUnexpected input dimension: Initialized %s with weight: %s reduction: %s ndim: %s signal_shape: %srtrurvr^rwrxr'r%ryrzrr{r|rer&r}rrdebugr~rrdrrdrerrr)r*ru2      zMSELoss.__init__cC@t|jtt|jtttdtddt|jtdddS)rrTrrrrrrr rrr)r)r*r  zMSELoss.input_typescCr)z%Output types definitions for MSELoss.rrrrr)r)r*rrzMSELoss.output_typesr,r-rrrcCPt||||d}|jdurtj|dd}n ||j}tj|dd}||}|S)aFor input batch of multi-channel signals, calculate SDR between estimate and target for each channel, perform averaging across channels (weighting optional), and apply reduction across the batch. Args: estimate: Estimate of the target signal target: Target signal input_length: Length of each example in the batch mask: Mask for each signal Returns: Scalar loss. rNr;r)rrdr%r&r'r})rr,r-rrrr)r)r*r   zMSELoss.forwardNr&rrrrrrrrrrrrurrrr r%rrrr)r)rr*r<  ,  cCs|j|jksJd|jd|jd|dur,|durtdt||ddd}||}||}|jd kr8d}n|jd kr@d }ntd |jtt|||d }|S)aCalculate mean absolute error (MAE) per channel. MAE = ||estimate - target||_1 / input_length Args: estimate: estimated signal, shape (B, C, T) or (B, C, D, T) target: target signal, shape (B, C, T) or (B, C, D, T) input_length: Optional, length of valid samples, shape (B,) mask: Optional, temporal mask, same shape as signals Returns: MAE for each channel, shape (B, C) rZr[r\NrrTrrrrrrrrr)r)r*calculate_mae_batchGs$   rc r)MAELossak Computes the mean absolute error (MAE) loss with weighted average across channels. Args: weight: weight for loss of each output channel, used for averaging the loss across channels. Defaults to `None` (averaging). reduction: batch reduction. Defaults to `mean` over the batch. ndim: Number of dimensions for the input signal Nr&rrdrercr)NcSrfrgr)rhr)r)r*rkrlz$MAELoss.__init__..rmr;r7rnrprrqrdr&rrrsrrrrrrrrrrrrrr)r*rurzMAELoss.__init__cCr)z$Input types definitions for MAELoss.rTrrrrr)r)r*rrzMAELoss.input_typescCr)z%Output types definitions for MAELoss.rrrrr)r)r*rrzMAELoss.output_typesr,r-rrrcCr)aFor input batch of multi-channel signals, calculate MAE between estimate and target for each channel, perform averaging across channels (weighting optional), and apply reduction across the batch. Args: estimate: Estimate of the target signal target: Target signal input_length: Length of each example in the batch mask: Mask for each signal Returns: Scalar loss. rNr;r)rrdr%r&r'r})rr,r-rrmaer)r)r*rrzMAELoss.forwardrrrr)r)rr*r{rr)NNrFr)NNr)NNr6r7r8)NNFFr6TNr8r)(rAtypingrrrrnumpyrwr%1nemo.collections.asr.parts.preprocessing.featuresr(nemo.collections.audio.parts.utils.audiornemo.core.classesrr r nemo.core.neural_typesr r r rrr nemo.utilsr__all__rrrrr+r5rTrcrrrrrr)r)r)r*s     / 3 e   Q{ 4q 4