o ߥi*%@sddlZddlmZddlmmZddlmZdZGdddejZGdddej Z Gdd d ejZ Gd d d ejZ Gd d d ejZ GdddejZGdddejZdS)N)Tensor:0yE>cs*eZdZdZdfdd ZddZZS) GlobalLayerNorma?Calculate Global Layer Normalization. Args: dim : (int or list or torch.Size) Input shape from an expected input of size. eps : float A value added to the denominator for numerical stability. elementwise_affine : bool A boolean value that when set to True, this module has learnable per-element affine parameters initialized to ones (for weights) and zeros (for biases). Example: ------- >>> x = torch.randn(5, 10, 20) >>> GLN = GlobalLayerNorm(10, 3) >>> x_norm = GLN(x) rTcstt|||_||_||_|jrM|dkr-tt |jd|_ tt |jd|_ |dkrKtt |jdd|_ tt |jdd|_ dSdS| dd| dddS)Nweightbias)superr__init__dimepselementwise_affinenn Parametertorchonesrzerosr register_parameter)selfr shaper r __class__e/home/ubuntu/.local/lib/python3.10/site-packages/modelscope/models/audio/separation/m2/conv_module.pyr s zGlobalLayerNorm.__init__cCs|dkr>> x = torch.randn(5, 10, 20) >>> CLN = CumulativeLayerNorm(10) >>> x_norm = CLN(x) Tcstt|j||dddS)Nr)rr )r r)r )rr rrrrr cs  zCumulativeLayerNorm.__init__csx|dkr |dddd}t|}|dddd}|dkr:t|dd}t|}t|dd}|S)zReturns the normalized tensor. Args: x : torch.Tensor Tensor size [N, C, K, S] or [N, C, L] rrrrr)r permute contiguousr r"r transposerr rrrr"gs    zCumulativeLayerNorm.forward)Tr#rrrrr)Ssr)cs8eZdZdZdeffdd ZdedefddZZS) Transposez; Wrapper class of torch.transpose() for Sequential module. rcstt|||_dSN)r r.r r)rrrrrr s zTranspose.__init__r returncCs |j|jSr/)r,rr-rrrr"s zTranspose.forward) r$r%r&r'tupler rr"r(rrrrr.sr.csXeZdZdZ   ddedededed ed ed d ffd d Zded efddZZ S)DepthwiseConv1daYWhen groups == in_channels and out_channels == K * in_channels, where K is a positive integer, this operation is termed in literature as depthwise convolution. Args: in_channels (int): Number of channels in the input out_channels (int): Number of channels produced by the convolution kernel_size (int or tuple): Size of the convolving kernel stride (int, optional): Stride of the convolution. Default: 1 padding (int or tuple, optional): Zero-padding added to both sides of the input. Default: 0 bias (bool, optional): If True, adds a learnable bias to the output. Default: True Inputs: inputs - **inputs** (batch, in_channels, time): Tensor containing input vector Returns: outputs - **outputs** (batch, out_channels, time): Tensor produces by depthwise 1-D convolution. rrF in_channels out_channels kernel_sizestridepaddingr r0Nc s@tt|||dksJdtj|||||||d|_dS)Nrz7out_channels should be constant multiple of in_channels)r3r4r5groupsr6r7r )r r2r rConv1dconv)rr3r4r5r6r7r rrrr s  zDepthwiseConv1d.__init__inputscCs ||Sr/)r:rr;rrrr"s zDepthwiseConv1d.forward)rrF) r$r%r&r'intboolr rr"r(rrrrr2s*r2c sPeZdZdZ   ddedededed d f fd d Zd ed efddZZ S) ConvModulea Conformer convolution module starts with a pointwise convolution and a gated linear unit (GLU). This is followed by a single 1-D depthwise convolution layer. Batchnorm is deployed just after the convolution to aid training deep models. Args: in_channels (int): Number of channels in the input kernel_size (int or tuple, optional): Size of the convolving kernel Default: 17 dropout_p (float, optional): probability of dropout Inputs: inputs inputs (batch, time, dim): Tensor contains input sequences Outputs: outputs outputs (batch, time, dim): Tensor produces by conformer convolution module. r皙?r3r5expansion_factor dropout_pr0Nc sdtt||dddksJd|dksJdttddt|||d|ddd|_dS) Nrrrz5kernel_size should be a odd number for 'SAME' paddingz+Currently, Only Supports expansion_factor 2r)r)r6r7)r r?r r Sequentialr.r2 sequential)rr3r5rBrCrrrr s&  zConvModule.__init__r;cCs|||ddS)Nrr)rEr,r<rrrr"szConvModule.forward)r@rrA) r$r%r&r'r=floatr rr"r(rrrrr?s"r?cs&eZdZdfdd ZddZZS) DilatedDenseNetr@c stt|||_||_tjddd|_|dd|_|jdf|_ t |jD]c}d|}||d|dd}t |d |dtjdd||fddt |d |dtj |j|d|j|j |df|jd d t |d |dtj|d d t |d |dt|jq'dS)N)rrrrg)valuerrpad{}rconv{}F)r5dilationr8r norm{}T)affineprelu{})r rGr depthr3r ConstantPad2dpadtwidthr5rangesetattrformatConv2dInstanceNorm2dPReLU)rrQlorderr3idil pad_lengthrrrr s8    zDilatedDenseNet.__init__cCst|d}|dddd}|}t|jD];}t|d|d|}t|d|d|}t|d|d|}t|d|d|}tj||gdd }q|dddd}|dS) NrrrrrKrLrNrP)r ) r unsqueezer*rUrQgetattrrWcatsqueeze)rr x_perskipr\outout1rrrr"s  zDilatedDenseNet.forward)rrHrI)r$r%r&r r"r(rrrrrGsrGcs,eZdZejdffdd ZddZZS)FFConvMDilatedrAc sBtt||t||ttdd|dt||_dS)Nrr@)rQr[r3) r r rrDLinearSiLUrGDropoutmdl)rdim_indim_out norm_klassdropoutrrrr s   zFFConvMDilated.__init__cCs||}|Sr/)rk)rr outputrrrr"s zFFConvMDilated.forward)r$r%r&r LayerNormr r"r(rrrrrgsrg)rtorch.nnr torch.nn.initinitrEPSModulerrqr)r.r2r?rGrgrrrrs  H, *+)