o ipH@sddlmZddlZddlmZddlmZmZGdddejZ GdddejZ Gdd d ejZ Gd d d ejZ Gd d d ejZ dS))ListN)conv2d_output_shapeconvtransp2d_output_shapecs(eZdZ dfdd ZddZZS) GLSTMFcst|dksJ|f|r!ddks!J||_||_||_t|_tfddt|D|_ t|D]}|j tfddt|DqBdS)atGrouped LSTM. Reference: Efficient Sequence Learning with Group Recurrent Networks; Gao et al., 2018 Args: hidden_size (int): total hidden size of all LSTMs in each grouped LSTM layer i.e., hidden size of each LSTM is `hidden_size // groups` groups (int): number of LSTMs in each grouped LSTM layer layers (int): number of grouped LSTM layers bidirectional (bool): whether to use BLSTM or unidirectional LSTM rearrange (bool): whether to apply the rearrange operation after each grouped LSTM layer rrcsg|]}tqS)nn LayerNorm.0_) hidden_sizerM/home/ubuntu/.local/lib/python3.10/site-packages/espnet2/enh/layers/dc_crn.py -sz"GLSTM.__init__..c s,g|]}tjr dndddqS)rT) batch_first bidirectional)r LSTMr )r hidden_size_trrr1sN) super__init__groupslayers rearranger ModuleList lstm_listrangelnappend)selfrrrrrlayer __class__)rrrrrs&    zGLSTM.__init__csP|dddd}}||dtjjddtjfddtjDddtj ddd j dtdj D]<j rg ||jddd||dtjjddtjfd dtjDddj qNdd|ddddS) zGrouped LSTM forward. Args: x (torch.Tensor): (B, C, T, D) Returns: out (torch.Tensor): (B, C, T, D) rrrdimcs&g|]}jd||dqSrrr i)outr rrrM&z!GLSTM.forward..) start_dimend_dimcs&g|]}j||dqSr'r(r)r!r+r rrr\r,) transpose contiguoussizeviewtorchchunkrstackrflattenrrrreshapecat)r xBTrr0rforward>s4 &z GLSTM.forward)rrrFF__name__ __module__ __qualname__rr> __classcell__rrr"rrs.rcs&eZdZdfdd ZddZZS) GluConv2drcsDttj|||||d|_tj|||||d|_t|_dS)aConv2d with Gated Linear Units (GLU). Input and output shapes are the same as regular Conv2d layers. Reference: Section III-B in [1] Args: in_channels (int): number of input channels out_channels (int): number of output channels kernel_size (int/tuple): kernel size in Conv2d stride (int/tuple): stride size in Conv2d padding (int/tuple): padding size in Conv2d ) in_channels out_channels kernel_sizestridepaddingN)rrr Conv2dconv1conv2Sigmoidsigmoid)r rErFrGrHrIr"rrrhs zGluConv2d.__init__cC"||}|||}||S)zConvGLU forward. Args: x (torch.Tensor): (B, C_in, H_in, W_in) Returns: out (torch.Tensor): (B, C_out, H_out, W_out) )rKrNrLr r;r+gaterrrr> zGluConv2d.forwardr'r?rrr"rrDgs rDcs*eZdZ  dfdd ZddZZS)GluConvTranspose2drrrcsHttj||||||d|_tj||||||d|_t|_dS)avConvTranspose2d with Gated Linear Units (GLU). Input and output shapes are the same as regular ConvTranspose2d layers. Reference: Section III-B in [1] Args: in_channels (int): number of input channels out_channels (int): number of output channels kernel_size (int/tuple): kernel size in ConvTranspose2d stride (int/tuple): stride size in ConvTranspose2d padding (int/tuple): padding size in ConvTranspose2d output_padding (int/tuple): Additional size added to one side of each dimension in the output shape )rErFrGrHrIoutput_paddingN)rrr ConvTranspose2ddeconv1deconv2rMrN)r rErFrGrHrIrUr"rrrs$  zGluConvTranspose2d.__init__cCrO)zDeconvGLU forward. Args: x (torch.Tensor): (B, C_in, H_in, W_in) Returns: out (torch.Tensor): (B, C_out, H_out, W_out) )rWrNrXrPrrrr>rRzGluConvTranspose2d.forward)rrTr?rrr"rrSs ,rScs8eZdZ         d fd d Zd d ZZS)DenselyConnectedBlockrrrrrrrTFc  st| dks J| t|_|} d\} }dg}t| dD]A}|jttj| ||d|dt |tj dd| |} t | |f|d|d\}}|||| krZ||ksbJ||| |fq!| ru|jt | ||||| d d S|jt | ||||dd S) aDensely-Connected Convolutional Block. Args: in_channels (int): number of input channels out_channels (int): number of output channels hid_channels (int): number of output channels in intermediate Conv layers kernel_size (tuple): kernel size for all but the last Conv layers padding (tuple): padding for all but the last Conv layers last_kernel_size (tuple): kernel size for the last GluConv layer last_stride (tuple): stride for the last GluConv layer last_padding (tuple): padding for the last GluConv layer last_output_padding (tuple): output padding for the last GluConvTranspose2d (only used when `transposed=True`) layers (int): total number of Conv layers transposed (bool): True to use GluConvTranspose2d in the last layer False to use GluConv2d in the last layer r)*rcrr)rGrHrIT)inplacerGrHpad)rGrHrIrUN)rrr rconvrr SequentialrJ BatchNorm2dELUrrSrD)r rErF hid_channelsrGrIlast_kernel_size last_stride last_paddinglast_output_paddingr transposed in_channelr=D hidden_sizesr tdimhdimr"rrrsb     " zDenselyConnectedBlock.__init__cCsh|jd|}||g}t|j}t|jddD]\}}|tj|dd}||dkr1||q|S)zDenselyConnectedBlock forward. Args: input (torch.Tensor): (B, C, T_in, F_in) Returns: out (torch.Tensor): (B, C, T_out, F_out) rrNr%)rhlen enumerater5r:r)r inputr+outputs num_layersidxr!rrrr>&s   zDenselyConnectedBlock.forward) rZr[r]r^r`r]rTraFr?rrr"rrYsVrYcsPeZdZgdddddddddddd d d d d fd effd d ZddZZS)DC_CRN)r @rZr[r]r^r`rardrFinput_channelscsFt|ddksJ|t|_d}|g}|}tdt|D]5}|jt||d||||||||| dd t ||f|||d\}}||||ksWJ||fq"||d}|| kshJ|| ft || |||d |_ t|_ t|_ tt|dddD]}|j t|||||||| | | | dd ||}t ||f| | | d\}}||kr||ksJ||||f|dkr||d}n|}t||f|||d\}}||ksJ||f||d|}|dksJ||d|f|j t||d|||||||d|f| d d qtj||d |_tj||d |_d S)ab Densely-Connected Convolutional Recurrent Network (DC-CRN). Reference: Fig. 3 and Section III-B in [1] Args: input_dim (int): input feature dimension input_channels (list): number of input channels for the stacked DenselyConnectedBlock layers Its length should be (`number of DenselyConnectedBlock layers`). It is recommended to use even number of channels to avoid AssertError when `glstm_bidirectional=True`. enc_hid_channels (int): common number of intermediate channels for all DenselyConnectedBlock of the encoder enc_kernel_size (tuple): common kernel size for all DenselyConnectedBlock of the encoder enc_padding (tuple): common padding for all DenselyConnectedBlock of the encoder enc_last_kernel_size (tuple): common kernel size for the last Conv layer in all DenselyConnectedBlock of the encoder enc_last_stride (tuple): common stride for the last Conv layer in all DenselyConnectedBlock of the encoder enc_last_padding (tuple): common padding for the last Conv layer in all DenselyConnectedBlock of the encoder enc_layers (int): common total number of Conv layers for all DenselyConnectedBlock layers of the encoder skip_last_kernel_size (tuple): common kernel size for the last Conv layer in all DenselyConnectedBlock of the skip pathways skip_last_stride (tuple): common stride for the last Conv layer in all DenselyConnectedBlock of the skip pathways skip_last_padding (tuple): common padding for the last Conv layer in all DenselyConnectedBlock of the skip pathways glstm_groups (int): number of groups in each Grouped LSTM layer glstm_layers (int): number of Grouped LSTM layers glstm_bidirectional (bool): whether to use BLSTM or unidirectional LSTM in Grouped LSTM layers glstm_rearrange (bool): whether to apply the rearrange operation after each grouped LSTM layer output_channels (int): number of output channels (must be an even number to recover both real and imaginary parts) rrrbrF) rErFrlrGrIrmrnrorrqrfr$)rrrrrT) rErFrlrGrIrmrnrorprrq) in_features out_featuresN)rrr rconv_encrrwrrYrrglstm skip_pathway deconv_decrLinearfc_realfc_imag)r input_dimrenc_hid_channelsenc_kernel_size enc_paddingenc_last_kernel_sizeenc_last_strideenc_last_padding enc_layersskip_last_kernel_sizeskip_last_strideskip_last_padding glstm_groups glstm_layersglstm_bidirectionalglstm_rearrangeoutput_channelsr=rtrvr*ruhsenc_hdimout_chhpaddingr"rrr9s <         zDC_CRN.__init__c Cs |}g}t|jD] \}}||}||q t|}||d}|jd|d}tj||fdd}tt|j dD]!}|j ||}|j|d|||d}tj||fdd}q=|j d|}tj |ddd\} } | | } | | } tj | | gdd}|S)zDC-CRN forward. Args: x (torch.Tensor): Concatenated real and imaginary spectrum features (B, input_channels[0], T, F) Returns: out (torch.Tensor): (B, 2, output_channels, T, F) r$rrr%r)rxrrrwrrr5r:rrr6rrr7) r r;r+conv_outr|r!num_outres deconv_out dout_real dout_imagout_realout_imagrrrr>s&    zDC_CRN.forward)r@rArBrrr>rCrrr"rr}8s,#r})typingrr5torch.nnr espnet2.enh.layers.conv_utilsrrModulerrDrSrYr}rrrrs  X.:i