o wi&@sddlZddlmmZddlmZmZddlm Z m Z m Z m Z ddl mZddZGdddeZGd d d eZGd d d eZGd ddeZdS)N)Loss typecheck) AudioSignal LengthsTypeLossTypeSpectrogramType) NeuralTypec CsTttj|||||dd}|d}|d}ttj|d|dddddS) a^Perform STFT and convert to magnitude spectrogram. Args: x (Tensor): Input signal tensor (B, T). fft_size (int): FFT size. hop_size (int): Hop size. win_length (int): Window length. window (str): Window function type. Returns: Tensor: Magnitude spectrogram (B, #frames, fft_size // 2 + 1). T)return_complex).r).gHz>)minr )torch view_as_realstftsqrtclamp transpose)xfft_sizehop_size win_lengthwindowx_stftrealimagrc/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/nemo/collections/tts/losses/stftlosses.pyr5s (rc@s6eZdZdZeddZeddZeddZdS) SpectralConvergenceLossz!Spectral convergence loss module.cCstdttdtdS)NBTDx_magy_mag)rrselfrrr input_typesK  z#SpectralConvergenceLoss.input_typescCdttdiSNloss elements_typerrr%rrr output_typesR z$SpectralConvergenceLoss.output_typescCs2tj||dddtj|ddd}t|}|S)aDCalculate forward propagation. It is assumed that x_mag and y_mag were padded to fit the maximum batch sequence length with silence, hence it is assumed that the norm of these extra padded values are 0. Therefore, input_lengths is not a argument unlike in LogSTFTMagnitudeLoss. Args: x_mag (Tensor): Magnitude spectrogram of predicted signal (B, #frames, #freq_bins). y_mag (Tensor): Magnitude spectrogram of groundtruth signal (B, #frames, #freq_bins). Returns: Tensor: Spectral convergence loss value. fro)r r )pdim)r normmean)r&r#r$r+rrrforwardXs$ zSpectralConvergenceLoss.forwardN __name__ __module__ __qualname____doc__propertyr'r/rr6rrrrrHs  rc@s<eZdZdZeddZeddZedddd ZdS) LogSTFTMagnitudeLosszLog STFT magnitude loss module.cC(tdttdttdtdddS)NrrToptionalr#r$ input_lengths)rrrr%rrrr'm  z LogSTFTMagnitudeLoss.input_typescCr)r*r.r%rrrr/ur0z!LogSTFTMagnitudeLoss.output_typesNrBcCsx|durtt|t|Stjt|t|dd}||jd}tj|ddgd}||}t||jdS)aCalculate forward propagation. Args: x_mag (Tensor): Magnitude spectrogram of predicted signal (B, #frames, #freq_bins). y_mag (Tensor): Magnitude spectrogram of groundtruth signal (B, #frames, #freq_bins). input_lengths (Tensor): Length of groundtruth sample in samples (B). Returns: Tensor: Log STFT magnitude loss value. Nnone) reductionr r )r3r)Fl1_lossr logshapesum)r&r#r$rBr+rrrr6{s zLogSTFTMagnitudeLoss.forwardr7rrrrr=js  r=csNeZdZdZdfdd Zedd Zed d Zed d ddZ Z S)STFTLosszSTFT loss module.xX hann_windowcsDtt|||_||_||_tt|||_t |_ t |_ dS)zInitialize STFT loss module.N) superrL__init__r shift_sizergetattrr rrspectral_convergence_lossr=log_stft_magnitude_loss)r&rrSrr __class__rrrRs zSTFTLoss.__init__cCr>N)rr rTr?ryrBrrrr%rrrr'rCzSTFTLoss.input_typescCsttdttddSNr,)sc_lossmag_lossr.r%rrrr/r(zSTFTLoss.output_typesNrDcCs|jj|jkr|j|j|_t||j|j|j|j}t||j|j|j|j}|j||d}|durUt |t |jd}t ||j dksUJt |d|j d|j |||d}||fS)acCalculate forward propagation. Args: x (Tensor): Predicted signal (B, T). y (Tensor): Groundtruth signal (B, T). input_lengths (Tensor): Length of groundtruth sample in samples (B). Returns: Tensor: Spectral convergence loss value. Tensor: Log STFT magnitude loss value. r"Nr z != rA)rdevicetorrrSrrUr floorfloatmaxrJrV)r&rr[rBr#r$r^r_rrrr6s .zSTFTLoss.forward)rMrNrOrP r8r9r:r;rRr<r'r/rr6 __classcell__rrrWrrLs  rLcsNeZdZdZd fdd ZeddZeddZed d d d Z Z S)MultiResolutionSTFTLossz"Multi resolution STFT loss module.rPcsxtt|t|t|krt|ksJJtj|_t|||D]\}}}|jt ||||g7_q'dS)aInitialize Multi resolution STFT loss module. Args: fft_sizes (list): List of FFT sizes. hop_sizes (list): List of hop sizes. win_lengths (list): List of window lengths. window (str): Window function type. N) rQrgrRlenr nn ModuleList stft_lossesziprL)r& fft_sizes hop_sizes win_lengthsrfssswlrWrrrRs ( z MultiResolutionSTFTLoss.__init__cCr>rYr\r%rrrr'rCz#MultiResolutionSTFTLoss.input_typescCsttdgttdgdSr]r.r%rrrr/s  z$MultiResolutionSTFTLoss.output_typesNrDc Cs^dgt|j}dgt|j}t|jD]\}}||||d\}} |||<| ||<q||fS)aCalculate forward propagation. Args: x (Tensor): Predicted signal (B, T). y (Tensor): Groundtruth signal (B, T). input_lengths (Tensor): Length of groundtruth sample in samples (B). Returns: List[Tensor]: Multi resolution spectral convergence loss value. List[Tensor]: Multi resolution log STFT magnitude loss value. grZ)rhrk enumerate) r&rr[rBr^r_ifsc_lmag_lrrrr6s  zMultiResolutionSTFTLoss.forward)rPrerrrWrrgs  rg)r torch.nn.functionalri functionalrGnemo.core.classesrrnemo.core.neural_types.elementsrrrr"nemo.core.neural_types.neural_typerrrr=rLrgrrrrs, "'4