o iL@s ddlZddlZddlmZddlmmZGdddejZGdddejZ de de de d e fd d Z Gd d d e Z GdddeZ dddZGdddejZGdddejjZGdddejZGdddejZGdddejZGdddejjZdS)Ncs6eZdZ        d fdd Zdd ZZS) _BatchNorm1dNh㈵>皙?TFc sVt||_||_|dur|r|d}n|dur|d}tj|||||d|_dS)N)epsmomentumaffinetrack_running_stats)super__init__combine_batch_timeskip_transposenn BatchNorm1dnorm) self input_shape input_sizerrr r r r __class__a/home/ubuntu/.local/lib/python3.10/site-packages/funasr/models/sond/encoder/ecapa_tdnn_encoder.pyr s    z_BatchNorm1d.__init__cCs|j}|jr,|jdkr||d|d|d}n||d|d|d|d}n |js5|dd}||}|jrD||}|S|jsM|dd}|S)Nrrr)shaper ndimreshaper transposer)rxshape_orx_nrrrforward$s $    z_BatchNorm1d.forward)NNrrTTFF)__name__ __module__ __qualname__r r" __classcell__rrrrrsrcsVeZdZ         dfdd Zd d Zd ed ed efddZddZZS)_Conv1dNrsameTreflectFc st||_||_||_||_| |_d|_| |_|dur&|dur&t d|dur/| |}t j |||j|j|jd|| d|_ dS)NFz.Must provide one of input_shape or in_channelsr)stridedilationpaddinggroupsbias)r r kernel_sizer*r+r, padding_mode unsqueezer ValueError_check_input_shaperConv1dconv) r out_channelsr/r in_channelsr*r+r,r-r.r0rrrrr :s,   z_Conv1d.__init__cCs|js |dd}|jr|d}|jdkr"|||j|j|j}n#|jdkr8|jd|j}t ||df}n |jdkr>nt d|j| |}|jrR| d}|js[|dd}|S)Nrrr(causalrvalidz1Padding must be 'same', 'valid' or 'causal'. Got ) rrr1r,_manage_paddingr/r+r*Fpadr2r5squeeze)rrnum_padwxrrrr"bs$        z_Conv1d.forwardr/r+r*cCs.|jd}t||||}tj|||jd}|S)Nr)mode)rget_padding_elemr;r<r0)rrr/r+r*L_inr,rrrr:s z_Conv1d._manage_paddingcCsrt|dkr d|_d}n|jr|d}nt|dkr|d}n tdtt||jddkr7td|j|S)z@Checks the input shape and returns the number of input channels.rTrrz"conv1d expects 2d, 3d inputs. Got rz4The field kernel size must be an odd number. Got %s.)lenr1rr2strr/)rrr7rrrr3s    z_Conv1d._check_input_shape) NNrrr(rTr)F) r#r$r%r r"intr:r3r&rrrrr'9s(( r'rBr*r/r+cCs|dkr%t||||d}||d||}|d|dg}|S|||dd|d}||d||dg}|S)Nrr)mathceil)rBr*r/r+n_stepsL_outr,rrrrAsrAceZdZfddZZS)r4ctj|ddi|dSNrTr r rargskwargsrrrr zConv1d.__init__r#r$r%r r&rrrrr4r4crJ)rcrKrLrMrNrrrr rQzBatchNorm1d.__init__rRrrrrrrSrcCst|jdks J|dur|}tj||j|jd t||| dk}|dur1|j}|dur8|j}tj |||d}|S)Nr)devicedtype)rUrT) rCrmaxlongitemtorcharangerTrUexpandr1 as_tensor)lengthmax_lenrUrTmaskrrrlength_to_masksr`cs,eZdZejdffdd ZddZZS) TDNNBlockrcs:tt|t|||||d|_||_t|d|_dS)N)r7r6r/r+r-r)r rar r4r5 activationrr)rr7r6r/r+rcr-rrrr s zTDNNBlock.__init__cCs||||SN)rrcr5)rrrrrr"szTDNNBlock.forward)r#r$r%rReLUr r"r&rrrrras racs*eZdZdZd fdd ZddZZS) Res2NetBlockaAn implementation of Res2NetBlock w/ dilation. Arguments --------- in_channels : int The number of channels expected in the input. out_channels : int The number of output channels. scale : int The scale of the Res2Net block. kernel_size: int The kernel size of the Res2Net block. dilation : int The dilation of the Res2Net block. Example ------- >>> inp_tensor = torch.rand([8, 120, 64]).transpose(1, 2) >>> layer = Res2NetBlock(64, 64, scale=4, dilation=3) >>> out_tensor = layer(inp_tensor).transpose(1, 2) >>> out_tensor.shape torch.Size([8, 120, 64]) rrcsptt|||dksJ||dksJ||||tfddt|dD|_||_dS)Nrcsg|] }tdqS))r/r+)ra).0ir+hidden_channel in_channelr/rr sz)Res2NetBlock.__init__..r)r rfr r ModuleListrangeblocksscale)rr7r6rqr/r+rrjrr s  zRes2NetBlock.__init__cCsg}ttj||jddD])\}}|dkr|}n|dkr&|j|d|}n |j|d||}||q tj|dd}|S)Nrdimr) enumeraterYchunkrqrpappendcat)rryrix_iy_irrrr"s zRes2NetBlock.forward)rgrrr#r$r%__doc__r r"r&rrrrrfsrfcs*eZdZdZfddZdddZZS)SEBlocka3An implementation of squeeze-and-excitation block. Arguments --------- in_channels : int The number of input channels. se_channels : int The number of output channels after squeeze. out_channels : int The number of output channels. Example ------- >>> inp_tensor = torch.rand([8, 120, 64]).transpose(1, 2) >>> se_layer = SEBlock(64, 16, 64) >>> lengths = torch.rand((8,)) >>> out_tensor = se_layer(inp_tensor, lengths).transpose(1, 2) >>> out_tensor.shape torch.Size([8, 120, 64]) csNtt|t||dd|_tjjdd|_t||dd|_ tj |_ dS)Nrr7r6r/T)inplace) r r}r r4conv1rYrrereluconv2Sigmoidsigmoid)rr7 se_channelsr6rrrr =s zSEBlock.__init__NcCs|jd}|dur+t||||jd}|d}|jddd}||jddd|}n|jddd}|||}|| |}||S)Nrr^rTrrTrskeepdim) rr`rTr1summeanrrrr)rrlengthsLr_totalsrrrr"Es  zSEBlock.forwardrdr{rrrrr}'s r}cs,eZdZdZd fdd Zd ddZZS) AttentiveStatisticsPoolingaTThis class implements an attentive statistic pooling layer for each channel. It returns the concatenated mean and std of the input tensor. Arguments --------- channels: int The number of input channels. attention_channels: int The number of attention channels. Example ------- >>> inp_tensor = torch.rand([8, 120, 64]).transpose(1, 2) >>> asp_layer = AttentiveStatisticsPooling(64) >>> lengths = torch.rand((8,)) >>> out_tensor = asp_layer(inp_tensor, lengths).transpose(1, 2) >>> out_tensor.shape torch.Size([8, 1, 128]) Tcs^td|_||_|rt|d|dd|_nt||dd|_t|_t ||dd|_ dS)Ng-q=rrr~) r r rglobal_contextratdnnrTanhtanhr4r5)rchannelsattention_channelsrrrrr js  z#AttentiveStatisticsPooling.__init__Nc Cs(|jd}d|jfdd}|durtj|jd|jd}t||||jd}|d }|jr_|jdd d  }||||\}}|d d d |}|d d d |}tj |||gd d } n|} | | || } | |dkt d } tj| dd } ||| \}}tj ||fd d } | d} | S)zCalculates mean and std for a batch (input tensor). Arguments --------- x : torch.Tensor Tensor of shape [N, C, L]. rrcSs@|||}t||||d||}||fS)Nr)rrYsqrtr1powclamp)rmrsrrstdrrr_compute_statisticss*z?AttentiveStatisticsPooling.forward.._compute_statisticsNr)rTrrTrrrz-inf)rrrYonesrTr`r1rrfloatrepeatrwr5rr masked_fillr;softmax) rrrrrr_rrrattn pooled_statsrrrr"vs(   z"AttentiveStatisticsPooling.forward)rTrdr{rrrrrUs rcs<eZdZdZddddejjdffdd Zd dd ZZ S) SERes2NetBlockaAn implementation of building block in ECAPA-TDNN, i.e., TDNN-Res2Net-TDNN-SEBlock. Arguments ---------- out_channels: int The number of output channels. res2net_scale: int The scale of the Res2Net block. kernel_size: int The kernel size of the TDNN blocks. dilation: int The dilation of the Res2Net block. activation : torch class A class for constructing the activation layers. groups: int Number of blocked connections from input channels to output channels. Example ------- >>> x = torch.rand(8, 120, 64).transpose(1, 2) >>> conv = SERes2NetBlock(64, 64, res2net_scale=4) >>> out = conv(x).transpose(1, 2) >>> out.shape torch.Size([8, 120, 64]) rgrrc st||_t||dd||d|_t||||||_t||dd||d|_t||||_ d|_ ||kr?t ||dd|_ dSdS)Nr)r/r+rcr-r~) r r r6ratdnn1rf res2net_blocktdnn2r}se_blockshortcutr4) rr7r6 res2net_scalerr/r+rcr-rrrr s<   zSERes2NetBlock.__init__NcCsF|}|jr ||}||}||}||}|||}||Srd)rrrrr)rrrresidualrrrr"s     zSERes2NetBlock.forwardrd) r#r$r%r|rYrrer r"r&rrrrrs*rc sbeZdZdZdejjgdgdgdddddgd d d f fd d ZdddZdddZ Z S) ECAPA_TDNNaAn implementation of the speaker embedding model in a paper. "ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification" (https://arxiv.org/abs/2005.07143). Arguments --------- activation : torch class A class for constructing the activation layers. channels : list of ints Output channels for TDNN/SERes2Net layer. kernel_sizes : list of ints List of kernel sizes for each layer. dilations : list of ints List of dilations for kernels in each layer. lin_neurons : int Number of neurons in linear layers. groups : list of ints List of groups for kernels in each layer. Example ------- >>> input_feats = torch.rand([5, 120, 80]) >>> compute_embedding = ECAPA_TDNN(80, lin_neurons=192) >>> outputs = compute_embedding(input_feats) >>> outputs.shape torch.Size([5, 1, 192]) )rrri)rrrr)rrrrrrgT)rrrrrrc s6tt|t|ksJt|t|ksJ||_t|_| |_| |_|j t ||d|d|d|| dt dt|dD]}|j t ||d|||| |||||| |dqEt |d|d|d|d|| dd|_ t|d|| d|_t|ddd|_t|dd|dd |_dS) Nrr)rrr/r+rcr-r)r-)rrrrbr~)r r rCrrrnrp window_size window_shiftrvrarormfarasprasp_bnr4fc)rr lin_neuronsrcr kernel_sizes dilationsrrrrr-rrrirrrr sb       zECAPA_TDNN.__init__Nc Cs|jd}tt||j}|jd}t|||ddfd}g}t|D]?}||j||j|j}} |j |dddd|| f|durQt ||d|jndd}| |}| |}||q%t j|ddS)Nrrr)rrr)rrErFrGrrr;r<rorrYrrrrvrw) rrrtt num_chunkr< stat_listristedrrrwindowed_poolinges      zECAPA_TDNN.windowed_poolingc Cs|dd}g}|jD]}z|||d}Wn ty"||}Ynw||q tj|dddd}||}|jdurV|j||d}| |}| |}| d}|S| ||}|dd}|S)zReturns the embedding vector. Arguments --------- x : torch.Tensor Tensor of shape (batch, time, channel). lengths: torch.Tensor Tensor of shape (batch, ) rrrNrr) rrp TypeErrorrvrYrwrrrrrr=r)rrrxllayerrrrr"|s(           zECAPA_TDNN.forwardrd) r#r$r%r|rYrrer rr"r&rrrrrs" Mr)NNN)rFrYtorch.nnrtorch.nn.functional functionalr;Modulerr'rErAr4rr`rarfr}rrrrrrrs  2n <.SS