o i-@sdZddlZddlZddlmZdddZGd d d eZdddZGdddeZdddZ GdddeZ     d ddZ GdddeZ dS)!z=Spec Augment module for preprocessing i.e., data augmentationN) FuncTransPFPILcCs$ddlm}ddlm}|}|dkrq|jd}|||kr|St|||}t||||d} ||d||jd| f|} |||d|jd|| f|} |ri| |d| <| || d<|St | | fdS|dkrddl } ddl m } | | || Std |d ) atime warp for spec augment move random center frame by the random width ~ uniform(-window, window) :param numpy.ndarray x: spectrogram (time, freq) :param int max_time_warp: maximum time frames to warp :param bool inplace: overwrite x with the result :param str mode: "PIL" (default, fast, not differentiable) or "sparse_image_warp" (slow, differentiable) :returns numpy.ndarray: time warped spectrogram (time, freq) r)Image)BICUBICrNsparse_image_warp) spec_augmentzunknown resize mode: z+, choose one from (PIL, sparse_image_warp).)rr PIL.Imagershaperandom randrange fromarrayresizenumpy concatenatetorch espnet.utilsr time_warp from_numpyNotImplementedError)x max_time_warpinplacemoderrwindowtcenterwarpedleftrightrr r!Q/home/ubuntu/.local/lib/python3.10/site-packages/espnet/transform/spec_augment.pyr s6   $(   rc&eZdZeZejZfddZZS)TimeWarpc|s|St|SNsuper__call__selfrtrain __class__r!r"r); zTimeWarp.__call__)__name__ __module__ __qualname__r_func__doc__r) __classcell__r!r!r-r"r$7r$Tc Cs|r|}n|}|jd}tjjd||dfd}|D]0\}} td||} | | 7} | | |kr2q|r?d|dd| | f<q||dd| | f<q|S)zfreq mask for spec agument :param numpy.ndarray x: (time, freq) :param int n_mask: the number of masks :param bool inplace: overwrite :param bool replace_with_zero: pad zero on mask if true else use mean rrr8sizeNcopyr rr randintr mean) rFn_maskreplace_with_zerorclonednum_mel_channelsfsfmask_endf_zeror!r!r" freq_maskAs   rHcr#)FreqMaskcr%r&r'r*r-r!r"r)dr/zFreqMask.__call__)r0r1r2rHr3r4r)r5r!r!r-r"rI`r6rI(c Cs|r|}n|}|jd}tjjd||dfd}|D]/\}} ||dkr&qtd||} | | |kr5q| | 7} |rBd|| | <q||| | <q|S)zfreq mask for spec agument :param numpy.ndarray spec: (time, freq) :param int n_mask: the number of masks :param bool inplace: overwrite :param bool replace_with_zero: pad zero on mask if true else use mean rr8r9r;) specTr@rArrB len_spectrotsrrFt_zeror!r!r" time_maskjs     rPcr#)TimeMaskcr%r&r'r*r-r!r"r)r/zTimeMask.__call__)r0r1r2rPr3r4r)r5r!r!r-r"rQr6rQdc CsVt|tjsJ|jdksJt||||d}t|||||d}t|||||d}|S)aIspec agument apply random time warping and time/freq masking default setting is based on LD (Librispeech double) in Table 2 https://arxiv.org/pdf/1904.08779.pdf :param numpy.ndarray x: (time, freq) :param str resize_mode: "PIL" (fast, nondifferentiable) or "sparse_image_warp" (slow, differentiable) :param int max_time_warp: maximum frames to warp the center frame in spectrogram (W) :param int freq_mask_width: maximum width of the random freq mask (F) :param int n_freq_mask: the number of the random freq mask (m_F) :param int time_mask_width: maximum width of the random time mask (T) :param int n_time_mask: the number of the random time mask (m_T) :param bool inplace: overwrite intermediate array :param bool replace_with_zero: pad zero on mask if true else use mean r8)rr)rrA) isinstancerndarrayndimrrHrP) r resize_modermax_freq_width n_freq_maskmax_time_width n_time_maskrrAr!r!r"r s$r cr#) SpecAugmentcr%r&r'r*r-r!r"r)r/zSpecAugment.__call__)r0r1r2r r3r4r)r5r!r!r-r"r\r6r\)rFr)r7r8TF)rJr8TF)rrrRr8rSr8TT) r4r respnet.transform.functionalrrr$rHrIrPrQr r\r!r!r!r"s(  -     0