o i8@sddlZddlmZddlmZddlmZeejedkZ Gdddej j Z Gdd d ej j Z Gd d d ej j ZGd d d ej j ZGdddej j ZdS)N)parse) ComplexTensor) get_layerz1.9.0cs8eZdZdZddddejjffdd Zdd ZZ S) Conv2DActNormz9Basic Conv2D + activation + instance norm building block.rr rFc srtt||rtj|||||}n tjj|||||dd}t|} tjj||dd} tj || | |_ dS)Nreflect padding_mode:0yE>eps) superr__init__torchnnConvTranspose2dConv2dr GroupNorm Sequentiallayer) self in_channels out_channelskszstridepaddingupsample activationconvactnorm __class__S/home/ubuntu/.local/lib/python3.10/site-packages/espnet2/enh/layers/tcndenseunet.pyr s    zConv2DActNorm.__init__cCs ||SNrrinpr(r(r)forward%s zConv2DActNorm.forward __name__ __module__ __qualname____doc__rrELUrr. __classcell__r(r(r&r)r srcs0eZdZdZejjffdd ZddZZ S) FreqWiseBlockzFreqWiseBlock, see iNeuBe paper. Block that applies pointwise 2D convolution over STFT-like image tensor on frequency axis. The input is assumed to be [batch, image_channels, frames, freq]. cs>tt|t||ddd|d|_t||ddd|d|_dS)Nr r )rrr")rr6rr bottleneck freq_proc)rr num_freqsrr"r&r(r)r2s   zFreqWiseBlock.__init__cCs,|||dddddddd}|S)Nrrr r )r:r9permute)rr-outr(r(r)r.<szFreqWiseBlock.forwardr/r(r(r&r)r6)s r6cs:eZdZdZddddejjdffdd Zdd ZZ S) DenseBlockansingle DenseNet block as used in iNeuBe model. Args: in_channels: number of input channels (image axis). out_channels: number of output channels (image axis). num_freqs: number of complex frequencies in the input STFT complex image-like tensor. The input is batch, image_channels, frames, freqs. pre_blocks: dense block before point-wise convolution block over frequency axis. freq_proc_blocks: number of frequency axis processing blocks. post_blocks: dense block after point-wise convolution block over frequency axis. ksz: kernel size used in densenet Conv2D layers. activation: activation function to use in the whole iNeuBe model, you can use any torch supported activation e.g. 'relu' or 'elu'. hid_chans: number of hidden channels in densenet Conv2D. r r r c  s:tt||dks J|dksJtjg|_d} t|D]} t|| | | |dd|d} |j | | d7} q tjg|_ t|D]} t || | || |d} |j | | d7} qFtjg|_ t|dD]} t|| | | |dd|d} |j | | d7} qlt|| | ||dd|d} |j | dS)Nr rr7r8) rr>rrr ModuleList pre_blocksrangerappendfreq_proc_blocksr6 post_blocks)rrrr;rArDrErr" hid_chans tot_layersindxc_layerlastr&r(r)rWs^              zDenseBlock.__init__cCs||g}|jD]}|t|d}||q|jD]}|t|d}||q|jD]}|t|d}||q,|S)Nr )rArcatrCrDrE)rinputr= pre_blockc_out freq_block post_blockr(r(r)r.s      zDenseBlock.forwardr/r(r(r&r)r>EsBr>cs6eZdZdZdddejjffdd ZddZZ S) TCNResBlockasingle depth-wise separable TCN block as used in iNeuBe TCN. Args: in_chan: number of input feature channels. out_chan: number of output feature channels. ksz: kernel size. stride: stride in depth-wise convolution. dilation: dilation in depth-wise convolution. activation: activation function to use in the whole iNeuBe model, you can use any torch supported activation e.g. 'relu' or 'elu'. rr c sjtt||}tjj||||||d|d}tj||d} tjtjj||ddt||| |_ dS)Nr )r dilationrgroupsr rr) rrQrrrConv1drrrr) rin_chanout_chanrrrRr"r dconv point_convr&r(r)rs&  zTCNResBlock.__init__cCs|||Sr*r+r,r(r(r)r.szTCNResBlock.forwardr/r(r(r&r)rQs  rQc sLeZdZdZdddddddddd ejjf fd d Zd d ZddZ Z S) TCNDenseUNetaTCNDenseNet block from iNeuBe Reference: Lu, Y. J., Cornell, S., Chang, X., Zhang, W., Li, C., Ni, Z., ... & Watanabe, S. Towards Low-Distortion Multi-Channel Speech Enhancement: The ESPNET-Se Submission to the L3DAS22 Challenge. ICASSP 2022 p. 9201-9205. Args: n_spk: number of output sources/speakers. in_freqs: number of complex STFT frequencies. mic_channels: number of microphones channels (only fixed-array geometry supported). hid_chans: number of channels in the subsampling/upsampling conv layers. hid_chans_dense: number of channels in the densenet layers (reduce this to reduce VRAM requirements). ksz_dense: kernel size in the densenet layers thorough iNeuBe. ksz_tcn: kernel size in the TCN submodule. tcn_repeats: number of repetitions of blocks in the TCN submodule. tcn_blocks: number of blocks in the TCN submodule. tcn_channels: number of channels in the TCN submodule. activation: activation function to use in the whole iNeuBe model, you can use any torch supported activation e.g. 'relu' or 'elu'. r ir?rric  stt|||_||_||_|d} tjtjj |jd|dddddt ||| || |d} | | }tj g|_ |j | tt|D]%}t||ddd| d }t |||||| |d}tj||}|j |qH|j t||dddd| d |j t|d|d ddd| d |j t|d | ddd| d g|_t|D]}t| D]}|jt| | |d|| d qqtjj|j|_tj g|_|jt| d|d ddd| d d |jt|d|dddd| d d |jt|d |ddd| d d tt|D]5}|t||d}t |d|d||| |d}t|d|ddd| d d }tj||}|j|q tjt |d|d|jd|| |dtj|dd|jddd}|j|dS)Nr rr7r r r )rr"rFrr8rZ)rRr"T)r"r!r )rrYrn_spkr mic_channelsrrrrr> _get_depthr@encoderrCrBlenrtcnrQdecoderr)rr]in_freqsr^rFhid_chans_dense ksz_denseksz_tcn tcn_repeats tcn_blocks tcn_channelsr"r;firstfreq_axis_dims layer_indx downsample denseblocksrIrxdec_indx c_num_freqsr!rJr&r(r)rs(            zTCNDenseUNet.__init__cCs:d}g}|dkrt|d}|||d7}|dks|S)Nrr r )intrC)rnum_freqn_layersfreqsr(r(r)r_s  zTCNDenseUNet._get_depthcCsT|dddd}|j\}}}}||jksJt|j|jfd}|dd}|||jd||j }g}|}|j D] } | |}| |q9|jddksNJ| | dd} | }t|jD]\} } t||| d fd} | | }q`||d|jd|j }trt|dddf|dddf}|St|dddf|dddf}|S)amforward. Args: tf_rep (torch.Tensor): 4D tensor (multi-channel complex STFT of mixture) of shape [B, T, C, F] batch, frames, microphones, frequencies. Returns: out (torch.Tensor): complex 3D tensor monaural STFT of the targets shape is [B, T, F] batch, frames, frequencies. rr rr N)r<shaper^rrKrealimag transposereshaperr`rCrbsqueeze unsqueeze enumeratercr]is_torch_1_9_pluscomplexr)rtf_repbszmics_frames inp_featsenc_outbuffer enc_layertcn_outrH dec_layerc_inputr=r(r(r)r.s2     $"zTCNDenseUNet.forward) r0r1r2r3rrr4rr_r.r5r(r(r&r)rYs"+ rY)rpackaging.versionrVtorch_complex.tensorr)espnet2.torch_utils.get_layer_from_stringr __version__rrModulerr6r>rQrYr(r(r(r)s   g*