o iJ#@stdZddlZddlmZddlZddlmZGdddejjZ GdddejjZ Gd d d Z Gd d d Z dS) zSinc convolutions.N)Union)check_argument_typescs6eZdZdZfddZdejdejfddZZS)LogCompressionzLLog Compression Activation. Activation function `log(abs(x) + 1)`. cstdS)z Initialize.N)super__init__)self __class__L/home/ubuntu/.local/lib/python3.10/site-packages/espnet2/layers/sinc_conv.pyrszLogCompression.__init__xreturncCstt|dS)zXForward. Applies the Log Compression function elementwise on tensor x. )torchlogabs)rr r r r forwardszLogCompression.forward) __name__ __module__ __qualname____doc__rrTensorr __classcell__r r rr r s rcseZdZdZ      d%deded ed ed ed ed ededeeefffdd Ze de j de j fddZ e de j de j fddZ e de j de j fddZddZdefddZde j de j fd d!Zd"edefd#d$ZZS)&SincConva*Sinc Convolution. This module performs a convolution using Sinc filters in time domain as kernel. Sinc filters function as band passes in spectral domain. The filtering is done as a convolution in time domain, and no transformation to spectral domain is necessary. This implementation of the Sinc convolution is heavily inspired by Ravanelli et al. https://github.com/mravanelli/SincNet, and adapted for the ESpnet toolkit. Combine Sinc convolutions with a log compression activation function, as in: https://arxiv.org/abs/2010.07597 Notes: Currently, the same filters are applied to all input channels. The windowing function is applied on the kernel to obtained a smoother filter, and not on the input values, which is different to traditional ASR. rrhammingmel> in_channels out_channels kernel_sizestridepaddingdilation window_func scale_typefsc s tsJt|j|jd} || vr tdt| | ||_t t d} || vr9tdt| | ||_ ||_ ||_ ||_||_||_||_t| |_|jddkr`tdd|_|jd} dtjtd | | |_|td | | |_|dS) aInitialize Sinc convolutions. Args: in_channels: Number of input channels. out_channels: Number of output channels. kernel_size: Sinc filter kernel size (needs to be an odd number). stride: See torch.nn.functional.conv1d. padding: See torch.nn.functional.conv1d. dilation: See torch.nn.functional.conv1d. window_func: Window function on the filter, one of ["hamming", "none"]. fs (str, int, float): Sample rate of the input data )nonerz!Window function has to be one of )rbarkzScale has to be one of rz"SincConv: Kernel size must be odd.Nr)rrr none_windowhamming_windowNotImplementedErrorlistkeysr#MelScale BarkScalescalerrrr!r"r floatr% ValueErrorfmathpirlinspace_x_window init_filters) rrrrr r!r"r#r$r% window_funcs scale_choicesNrr r r3s@       zSincConv.__init__r r cCs|d}t||S)zSinc function.gư>)rsin)r x2r r r sincpsz SincConv.sinccCs t|S)z!Identity-like windowing function.)r ones_liker r r r r)vs zSincConv.none_windowcCs<d|dd}|d}ddtdtj||S)zHamming Windowing function.r(rrgHzG?gq= ףp?@)sizefliprcosr4r5)r Lr r r r*{s  zSincConv.hamming_windowcCs6|j|j|j}t||j}tjj|dd|_dS)z*Initialize filters with filterbank values.T) requires_gradN) r0bankrr%rdivnn Parameterr3)rr3r r r r9szSincConv.init_filtersdevicec Cs2t|jdddf}t|jdddft|jdddf|jdddf}|j||_|j||_t|dd|jdd}t|dd|jdd}t|t|d|j}||j}| d}d|d| d}tj |||gdd} | | dd| d} | |_ dS)zmCalculate coefficients. This function (re-)calculates the filter convolutions coefficients. Nrr?r(dim)rrr3r7tor8matmulviewr=rD unsqueezecatrC sinc_filters) rrLf_minsf_maxsf_mins_xf_maxs_xkernel kernel_left kernel_centerfiltersr r r _create_filterssB   zSincConv._create_filtersxscCs4||jtjjj||j|j|j|j |j d}|S)zSinc convolution forward function. Args: xs: Batch in form of torch.Tensor (B, C_in, D_in). Returns: xs: Batch in form of torch.Tensor (B, C_out, D_out). )r!r r"groups) r_rLrrJ functionalconv1drVr!r r"r)rr`r r r rs zSincConv.forwardidimcCs4|d|j|j|jdd}||jd}|S)z*Obtain the output dimension of the filter.r(r)r!r"rr )rrdD_outr r r get_odims"zSincConv.get_odim)rrrrrr)rrrrintstrrr1r staticmethodrrr?r)r*r9r_rrfrr r rr rsJ  =rc@DeZdZdZeddZeddZedede de j fd d Z d S) r.zMel frequency scale.cCsdtt|ddS)zConvert Hz to mel.@@?)rrrI)r3r r r convertzMelScale.convertcCsdtt|ddS)zConvert mel to Hz.rlrkrm)rexprIrAr r r invertrozMelScale.invertchannelsr%r cCsvtsJtd}t|d}t|||||d}||}|dd|dd}}tj||gddS)uObtain initialization values for the mel scale. Args: channels: Number of channels. fs: Sample rate. Returns: torch.Tensor: Filter start frequencíes. torch.Tensor: Filter stop frequencies. g>@rNr(NrrO)rrtensorr6rnrqstack)clsrrr% min_frequency max_frequency frequenciesf1f2r r r rHs  z MelScale.bankN rrrrrirnrq classmethodrgr1rrrHr r r r r.s  r.c@rj) r/zBark frequency scale. Has wider bandwidths at lower frequencies, see: Critical bandwidth: BARK Zwicker and Terhardt, 1980 cCs8t|d}t|dd}t|dd}|ddS)zConvert Hz to Bark.@@rBffffff?rmgGz?R@9@rrIpow)r3br r r rns  zBarkScale.convertcCs@t|dd}t|d}t|dd}t|d}|dS)zConvert Bark to Hz.rrgt\;0?rmrrNr~r)r r3r r r rqs   zBarkScale.invertrrr%r cCstsJtd}t|d}t|||||}||}|t||d}|t||d}tj||gddS)u Obtain initialization values for the Bark scale. Args: channels: Number of channels. fs: Sample rate. Returns: torch.Tensor: Filter start frequencíes. torch.Tensor: Filter stop frequencíes. gQ@g?r(rrO)rrrtr6rnrqrIru)rvrrr%min_center_frequencymax_center_frequencycenter_frequenciesrzr{r r r rHs  zBarkScale.bankNr|r r r r r/s  r/) rr4typingrr typeguardrrJModulerrr.r/r r r r s  %