o siy@sddlmZmZmZddlZddlZddlmZddlZddl m Z ddl m Z m Z ddl mZdd l mZdd lmZdd lmZdd lmZmZmZmZdd lmZmZGdddejZGdddejZGdddejZGdddejZ GdddejZ!GdddejZ"GdddeZ#edee$fddZ%edee$fdd Z&Gd!d"d"ejZ'Gd#d$d$ejZ(Gd%d&d&ejZ)Gd'd(d(e)Z*Gd)d*d*e)Z+dS)+)ListTupleOptionalN)nn) complex_nn)norms activations)BaseDCUMaskNet)GlobLN)has_arg)DCUNET_ARCHITECTURES)_DilatedConvNorm_NormAct _ConvNormAct _ConvNorm)script_if_tracing pad_x_to_ycs(eZdZdZfddZddZZS)_Chop1dz5To ensure the output length is the same as the input.cst||_dSN)super__init__ chop_size)selfr __class__Q/home/ubuntu/.local/lib/python3.10/site-packages/asteroid/masknn/convolutional.pyrs  z_Chop1d.__init__cCs|dd|j fS)N.)r contiguousrxrrrforwardsz_Chop1d.forward__name__ __module__ __qualname____doc__rr" __classcell__rrrrrs rcs.eZdZdZ  dfdd ZddZZS) Conv1DBlockaOne dimensional convolutional block, as proposed in [1]. Args: in_chan (int): Number of input channels. hid_chan (int): Number of hidden channels in the depth-wise convolution. skip_out_chan (int): Number of channels in the skip convolution. If 0 or None, `Conv1DBlock` won't have any skip connections. Corresponds to the the block in v1 or the paper. The `forward` return res instead of [res, skip] in this case. kernel_size (int): Size of the depth-wise convolutional kernel. padding (int): Padding of the depth-wise convolution. dilation (int): Dilation of the depth-wise convolution. norm_type (str, optional): Type of normalization to use. To choose from - ``'gLN'``: global Layernorm. - ``'cLN'``: channelwise Layernorm. - ``'cgLN'``: cumulative global Layernorm. - Any norm supported by :func:`~.norms.get` causal (bool, optional) : Whether or not the convolutions are causal References [1] : "Conv-TasNet: Surpassing ideal time-frequency magnitude masking for speech separation" TASLP 2019 Yi Luo, Nima Mesgarani https://arxiv.org/abs/1809.07454 gLNFc stt|||_t|} t||d} tj||||||d} |r+t| t |} t| t | || t | ||_ t||d|_ |rRt||d|_ dSdS)Nr)paddingdilationgroups)rr)r skip_out_chanr getrConv1d SequentialrPReLU shared_blockres_conv skip_conv) rin_chanhid_chanr. kernel_sizer+r, norm_typecausal conv_norm in_conv1d depth_conv1drrrr:s*   zConv1DBlock.__init__cCs0||}||}|js|S||}||fS)z"Input shape $(batch, feats, seq)$.)r3r4r.r5)rr! shared_outres_outskip_outrrrr"Zs   zConv1DBlock.forward)r*Fr#rrrrr)s $ r)csFeZdZdZ          dfd d Zd d ZddZZS) TDConvNetaTemporal Convolutional network used in ConvTasnet. Args: in_chan (int): Number of input filters. n_src (int): Number of masks to estimate. out_chan (int, optional): Number of bins in the estimated masks. If ``None``, `out_chan = in_chan`. n_blocks (int, optional): Number of convolutional blocks in each repeat. Defaults to 8. n_repeats (int, optional): Number of repeats. Defaults to 3. bn_chan (int, optional): Number of channels after the bottleneck. hid_chan (int, optional): Number of channels in the convolutional blocks. skip_chan (int, optional): Number of channels in the skip connections. If 0 or None, TDConvNet won't have any skip connections and the masks will be computed from the residual output. Corresponds to the ConvTasnet architecture in v1 or the paper. conv_kernel_size (int, optional): Kernel size in convolutional blocks. norm_type (str, optional): To choose from ``'BN'``, ``'gLN'``, ``'cLN'``. mask_act (str, optional): Which non-linear function to generate mask. causal (bool, optional) : Whether or not the convolutions are causal. References [1] : "Conv-TasNet: Surpassing ideal time-frequency magnitude masking for speech separation" TASLP 2019 Yi Luo, Nima Mesgarani https://arxiv.org/abs/1809.07454 Nr*reluFc s`tt|||_||_|r|n|}||_||_||_||_||_ ||_ | |_ | |_ | |_ | |_t| |} t||d}t| ||_t|_t|D]0}t|D])}| sd| dd|d}n| dd|}|jt|||| |d|| | dqUqO|r|n|}t|||d}tt||_t| }t|dr|dd|_dS||_dS)Nrr)r+r,r9r:dimrG)rrArr6n_srcout_chann_blocks n_repeatsbn_chanr7 skip_chanconv_kernel_sizer9mask_actr:r r/rr0r1 bottleneck ModuleListTCNrangeappendr)r2mask_netr r output_act)rr6rIrJrKrLrMr7rNrOr9rPr: layer_normbottleneck_convrr!r+ mask_conv_inp mask_conv mask_nl_classrrrrsV        zTDConvNet.__init__cCs|\}}}||}tjdg|jd}|jD]}||}|jr*|\} } || }n|} || }q|jr6|n|} || } | ||j |j |} | | } | S)Forward. Args: mixture_w (:class:`torch.Tensor`): Tensor of shape $(batch, nfilters, nframes)$ Returns: :class:`torch.Tensor`: estimated mask of shape $(batch, nsrc, nfilters, nframes)$ )device) sizerQtorchtensorr`rSrNrVviewrIrJrW)r mixture_wbatch_n_framesoutputskip_connectionlayertcn_outresidualskipmask_inpscoreest_maskrrrr"s      zTDConvNet.forwardc Cs:|j|j|j|j|j|j|j|j|j|j |j |j d }|S)N r6rJrMr7rNrOrKrLrIr9rPr:rrrconfigrrr get_configszTDConvNet.get_config) NrBrCrDrErDrCr*rFFr$r%r&r'rr"rur(rrrrrAds!@rAcsDeZdZdZ         dfd d Zd d Zd dZZS) TDConvNetppuoImproved Temporal Convolutional network used in [1] (TDCN++) Args: in_chan (int): Number of input filters. n_src (int): Number of masks to estimate. out_chan (int, optional): Number of bins in the estimated masks. If ``None``, `out_chan = in_chan`. n_blocks (int, optional): Number of convolutional blocks in each repeat. Defaults to 8. n_repeats (int, optional): Number of repeats. Defaults to 3. bn_chan (int, optional): Number of channels after the bottleneck. hid_chan (int, optional): Number of channels in the convolutional blocks. skip_chan (int, optional): Number of channels in the skip connections. If 0 or None, TDConvNet won't have any skip connections and the masks will be computed from the residual output. Corresponds to the ConvTasnet architecture in v1 or the paper. kernel_size (int, optional): Kernel size in convolutional blocks. norm_type (str, optional): To choose from ``'BN'``, ``'gLN'``, ``'cLN'``. mask_act (str, optional): Which non-linear function to generate mask. References [1] : Kavalerov, Ilya et al. “Universal Sound Separation.” in WASPAA 2019 .. note:: The differences wrt to ConvTasnet's TCN are: 1. Channel wise layer norm instead of global 2. Longer-range skip-residual connections from earlier repeat inputs to later repeat inputs after passing them through dense layer. 3. Learnable scaling parameter after each dense layer. The scaling parameter for the second dense layer in each convolutional block (which is applied rightbefore the residual connection) is initialized to an exponentially decaying scalar equal to 0.9**L, where L is the layer or block index. NrBrCrDrEfgLNrFc  st||_||_|r|n|}||_||_||_||_||_||_ | |_ | |_ | |_ t | |} t||d} t| | |_t|_t|D]$}t|D]}| dd|d}|jt|||| |d|| dqPqJt|_t|dD] }|jt||dqztddtd|D}|d||d}tj|dd|_|r|n|}t|||d}tt ||_!t"| }t#|d r|dd |_$n||_$|r|n|}t%|||_&dS) Nrr)r+r,r9cSsg|]}d|qS)g?r).0lrrr Lz(TDConvNetpp.__init__..rT) requires_gradrGrH)'rrr6rIrJrKrLrMr7rNrOr9rPr r/rr0r1rQrRrSrTrUr) dense_skiprbTensor unsqueezeexpandclone Parameter scaling_paramr2rVr r rWLinear consistency)rr6rIrJrKrLrMr7rNrOr9rPrXrYrZr!r+rr[r\r]out_sizerrrrs^          zTDConvNetpp.__init__cCs*|\}}}||}|}d}t|jD]P}|dkr(|j|d||}|}t|jD]7} ||j| } |j| |} |jrI| \} } || }n| \} }| dkrZ|j|| dfnd}| |} || }q-q|jrk|n|}| |}| ||j |j |}| |}||d}tjj|d}||fS)r^r_rrg?)rarQrTrLr~rKrSrNrrVrdrIrJrWrmeanrbr functionalsoftmax)rrerf n_filtersrhri output_copyrjrZr!irlrmrnrgscalerorprqweightsrrrr"^s4     zTDConvNetpp.forwardc Cs6|j|j|j|j|j|j|j|j|j|j |j d }|S)N r6rJrMr7rNrOrKrLrIr9rPrrsrrrrus zTDConvNetpp.get_config) NrBrCrDrErDrCrxrFrvrrrrrws+F-rwcs6eZdZdZ  d fdd ZdejfddZZS) DCUNetComplexEncoderBlockaEncoder block as proposed in [1]. Args: in_chan (int): Number of input channels. out_chan (int): Number of output channels. kernel_size (Tuple[int, int]): Convolution kernel size. stride (Tuple[int, int]): Convolution stride. padding (Tuple[int, int]): Convolution padding. norm_type (str, optional): Type of normalization to use. See :mod:`~asteroid.masknn.norms` for valid values. activation (str, optional): Type of activation to use. See :mod:`~asteroid.masknn.activations` for valid values. References [1] : "Phase-aware Speech Enhancement with Deep Complex U-Net", Hyeong-Seok Choi et al. https://arxiv.org/abs/1903.03107 bN leaky_reluc sLttj||||||dud|_t|||_t|}||_ dSN)bias) rrr ComplexConv2dconvr get_complexnormr activation) rr6rJr8strider+r9ractivation_classrrrrs   z"DCUNetComplexEncoderBlock.__init__r!cC||||Sr)rrrr rrrr"z!DCUNetComplexEncoderBlock.forward)rr r$r%r&r'rr ComplexTensorr"r(rrrrrs rcs8eZdZdZ   d fdd Zdejfdd ZZS) DCUNetComplexDecoderBlockaDecoder block as proposed in [1]. Args: in_chan (int): Number of input channels. out_chan (int): Number of output channels. kernel_size (Tuple[int, int]): Convolution kernel size. stride (Tuple[int, int]): Convolution stride. padding (Tuple[int, int]): Convolution padding. norm_type (str, optional): Type of normalization to use. See :mod:`~asteroid.masknn.norms` for valid values. activation (str, optional): Type of activation to use. See :mod:`~asteroid.masknn.activations` for valid values. References [1] : "Phase-aware Speech Enhancement with Deep Complex U-Net", Hyeong-Seok Choi et al. https://arxiv.org/abs/1903.03107 rrrrc srt||_||_||_||_||_||_tj |||||||dud|_ t |||_ t |} | |_dSr)rrr6rJr8rr+output_paddingrComplexConvTranspose2ddeconvr rrr r) rr6rJr8rr+rr9rrrrrrs   z"DCUNetComplexDecoderBlock.__init__r!cCrr)rrrr rrrr"rz!DCUNetComplexDecoderBlock.forward)rrrrrrrrrsrcs6eZdZdZeZd fdd ZddZddZZ S) DCUMaskNetaMasking part of DCUNet, as proposed in [1]. Valid `architecture` values for the ``default_architecture`` classmethod are: "Large-DCUNet-20", "DCUNet-20", "DCUNet-16", "DCUNet-10" and "mini". Valid `fix_length_mode` values are [None, "pad", "trim"]. Input shape is expected to be $(batch, nfreqs, time)$, with $nfreqs - 1$ divisible by $f_0 * f_1 * ... * f_N$ where $f_k$ are the frequency strides of the encoders, and $time - 1$ is divisible by $t_0 * t_1 * ... * t_N$ where $t_N$ are the time strides of the encoders. References [1] : "Phase-aware Speech Enhancement with Deep Complex U-Net", Hyeong-Seok Choi et al. https://arxiv.org/abs/1903.03107 Nc s|||_tjdd|Ddd|_ddlmmtjd fdd|Dfdd|dd Dt j |d d |dS) NcSsg|] \}}}}}|qSrr)ryrg enc_striderrrr{sz'DCUMaskNet.__init__..r)axisrrrcg|]}|qSrrryargs)rrrr{r|crrrr)rrrr{r|r)encodersdecoders output_layerr) fix_length_modenpprodencoders_stride_product convolutionalrrrrrr)rrrrkwargsrrrrs  zDCUMaskNet.__init__cCst|j|t|jSr)_fix_dcu_input_dimsrrb from_numpyrr rrrfix_input_dims szDCUMaskNet.fix_input_dimscCst|j||Sr)_fix_dcu_output_dimsr)routr!rrrfix_output_dims%szDCUMaskNet.fix_output_dimsr) r$r%r&r'r_architecturesrrrr(rrrrrs rrcCst|d}t|d}|jdd|r!td|d|jd|jdd|}|rm|dur)epsr8cg|] }tdqS)rJr6upsampling_depth)UBlockryrgrMr6rrrr{sz%SuDORMRF.__init__..rr)r8r+rGrH)rrr6rIrM num_blocksrrPr GroupNormlnr0l1r1rTsmreshape_before_masksConv2dmr r/r rW)rr6rIrMrrrPr]rrrr\s4      zSuDORMRF.__init__cCsR||}||}||}|j|jkr||}||d}||}|S)Nr) rrrrMr6rrrrWr rrrr"s      zSuDORMRF.forwardcC"|j|j|j|j|j|jd}|SNr6rIrMrrrPrrsrrrruzSuDORMRF.get_config)rDrrrrvrrrrrLs4 rcr) SuDORMRFImprovedaOImproved SuDORMRF mask network, as described in [1]. Args: in_chan (int): Number of input channels. Also number of output channels. n_src (int): Number of sources in the input mixtures. bn_chan (int, optional): Number of bins in the bottleneck layer and the UNet blocks. num_blocks (int): Number of of UBlocks upsampling_depth (int): Depth of upsampling mask_act (str): Name of output activation. References [1] : "Sudo rm -rf: Efficient Networks for Universal Audio Source Separation", Tzinis et al. MLSP 2020. rDrrrFc st|_||_|_||_|_||_t|_ t j dd|_ t j fddt|D|_t |d}t t ||_t|}t|drZ|dd|_dS||_dS)Nrrcrr) UConvBlockrrrrr{rz-SuDORMRFImproved.__init__..rGrH)rrr6rIrMrrrPr rrr0rQr1rTrr2rVr r/r rW) rr6rIrMrrrPr\r]rrrrs(      zSuDORMRFImproved.__init__cCsP||}||}||}||}||jd|j|jd}||}|S)Nrr) rrQrrVrdrrIr6rWr rrrr"s     zSuDORMRFImproved.forwardcCrrrrsrrrrurzSuDORMRFImproved.get_config)rDrrrFrvrrrrrs, rcseZdZdfdd ZZS) _BaseUBlockrDrErFc stt||ddd|d|_||_t|_|jt ||dd|d|dt d|D]}|dkr5d}nd}|jt ||d|d||d|dq,|dkrYt jj dd|_ dSdS)Nr)rr- use_globln)kSizerr-drrr) scale_factor)rrrproj_1x1depthrrRspp_dwrUrrTrbUpsample upsampler)rrJr6rrrrrrrrsJ       z_BaseUBlock.__init__)rDrErF)r$r%r&rr(rrrrrsrc*eZdZdZd fdd ZddZZS) rzlUpsampling block. Based on the following principle: ``REDUCE ---> SPLIT ---> TRANSFORM --> MERGE`` rDrErcs@tj|||ddt||dddd|_t||_t||_dS)NFrr)r-)rrr conv_1x1_expr final_norm module_actrrJr6rrrrr.s zUBlock.__init__c Cs||}|jd|g}td|jD]}|j||d}||qt|jdD]}||d}|d|dd|djdf|d<q+|| |d}| ||S)zn Args: x: input feature map Returns: transformed feature map rrr.N) rrrTrrUrpoprrrr) rr!output1rikout_krgresampled_out_kexpandedrrrr"4s  (zUBlock.forwardrDrErr#rrrrr(srcr) rzBlock which performs successive downsampling and upsampling in order to be able to analyze the input features in multiple resolutions. rDrErcs6tj|||ddt|dd|_t||d|_dS)NTrr)rrrrrr0r4rrrrrUszUConvBlock.__init__c Cs|}||}|jd|g}td|jD]}|j||d}||qt|jdD]}||d}|d|dd|djdf|d<q/| |d} | | |S)zm Args x: input feature map Returns: transformed feature map rrr.N) rrrrTrrUrrrrr4) rr!rmrrirrrgrrrrrr"Zs  (zUConvBlock.forwardrr#rrrrrPsr),typingrrrnumpyrrbrwarningsrr r baser r utilsr _dcunet_architecturesr_localrrrrutils.torch_utilsrrModulerr)rArwrrrstrrrrrrrrrrrrs>       G -,4-]S,(