o sif@s ddlZddlmZddlmZmZddlZddlmZddl m Z ddlm Z m Z dd l mZdd lmZdd l mZmZGd d d ejZGdddejZGdddejZGdddejZGdddejZGdddejZGdddejZGdddejZGdddeZdS)N)nn)foldunfold) complex_nn)has_arg) activationsnorms)DCCRN_ARCHITECTURES)BaseDCUMaskNet)CumLNGlobLNc8eZdZdZ d fdd ZeddZd d ZZS) SingleRNNaModule for a RNN block. Inspired from https://github.com/yluo42/TAC/blob/master/utility/models.py Licensed under CC BY-NC-SA 3.0 US. Args: rnn_type (str): Select from ``'RNN'``, ``'LSTM'``, ``'GRU'``. Can also be passed in lowercase letters. input_size (int): Dimension of the input feature. The input should have shape [batch, seq_len, input_size]. hidden_size (int): Dimension of the hidden state. n_layers (int, optional): Number of layers used in RNN. Default is 1. dropout (float, optional): Dropout ratio. Default is 0. bidirectional (bool, optional): Whether the RNN layers are bidirectional. Default is ``False``. rrFcsntt||dvsJ|}||_||_||_||_||_||_ t t |||||dt |d|_ dSN)RNNLSTMGRUT) num_layersdropout batch_first bidirectional)superr__init__upperrnn_type input_size hidden_sizen_layersrrgetattrrboolrnnselfrrrrrr __class__M/home/ubuntu/.local/lib/python3.10/site-packages/asteroid/masknn/recurrent.pyr s" zSingleRNN.__init__cCs|j|jrdSdSNrr)rrr$r'r'r( output_size5szSingleRNN.output_sizecCs |j|}||\}}|S)Input shape [batch, seq, feats])r"flatten_parameters)r$inpoutput rnn_output_r'r'r(forward9s zSingleRNN.forwardrrF __name__ __module__ __qualname____doc__rpropertyr+r2 __classcell__r'r'r%r(rs rcr) MulCatRNNa0MulCat RNN block from [1]. Composed of two RNNs, returns ``cat([RNN_1(x) * RNN_2(x), x])``. Args: rnn_type (str): Select from ``'RNN'``, ``'LSTM'``, ``'GRU'``. Can also be passed in lowercase letters. input_size (int): Dimension of the input feature. The input should have shape [batch, seq_len, input_size]. hidden_size (int): Dimension of the hidden state. n_layers (int, optional): Number of layers used in RNN. Default is 1. dropout (float, optional): Dropout ratio. Default is 0. bidirectional (bool, optional): Whether the RNN layers are bidirectional. Default is ``False``. References [1] Eliya Nachmani, Yossi Adi, & Lior Wolf. (2020). Voice Separation with an Unknown Number of Multiple Speakers. rrFcstt||dvsJ|}||_||_||_||_||_||_ t t |||||dt |d|_ t t |||||dt |d|_dSr)rr;rrrrrrrrr rr!rnn1rnn2r#r%r'r(rTs2 zMulCatRNN.__init__cCs|j|jrdnd|jSr))rrrr*r'r'r(r+qszMulCatRNN.output_sizecCsD|j|j||\}}||\}}t|||fdS)r,r)r<r-r=torchcat)r$r. rnn_output1r1 rnn_output2r'r'r(r2us  zMulCatRNN.forwardr3r4r'r'r%r(r;As r;c*eZdZdZd fdd ZddZZS) StackedResidualRNNalStacked RNN with builtin residual connection. Only supports forward RNNs. See StackedResidualBiRNN for bidirectional ones. Args: rnn_type (str): Select from ``'RNN'``, ``'LSTM'``, ``'GRU'``. Can also be passed in lowercase letters. n_units (int): Number of units in recurrent layers. This will also be the expected input size. n_layers (int): Number of recurrent layers. dropout (float): Dropout value, between 0. and 1. (Default: 0.) bidirectional (bool): If True, use bidirectional RNN, else unidirectional. (Default: False) Fc s~tt|||_||_||_||_|dusJd||_t |_ t |D]}|j t ||||dq't|j|_dS)NFzBidirectional not supported yetrrr)rrCrrn_unitsrrrr ModuleListlayersrangeappendrDropout dropout_layer)r$rrGrrrr1r%r'r(rs  zStackedResidualRNN.__init__cCs*|jD]}||}||}||}q|S)|Builtin residual connections + dropout applied before residual. Input shape : [batch, time_axis, feat_axis] )rIrM)r$xr"rnn_out dropped_outr'r'r(r2s   zStackedResidualRNN.forward)rDrEFr5r6r7r8rr2r:r'r'r%r(rC~srCcrB) StackedResidualBiRNNaStacked Bidirectional RNN with builtin residual connection. Residual connections are applied on both RNN directions. Only supports bidiriectional RNNs. See StackedResidualRNN for unidirectional ones. Args: rnn_type (str): Select from ``'RNN'``, ``'LSTM'``, ``'GRU'``. Can also be passed in lowercase letters. n_units (int): Number of units in recurrent layers. This will also be the expected input size. n_layers (int): Number of recurrent layers. dropout (float): Dropout value, between 0. and 1. (Default: 0.) bidirectional (bool): If True, use bidirectional RNN, else unidirectional. (Default: False) rDrETc st||_||_||_||_|dusJd||_t||||d|_t |_ t |dD]}d|}|j t||||dq0t |j|_dS)NTz$Only bidirectional not supported yetrFrr)rrrrGrrrr first_layerrrHrIrJrKrLrM)r$rrGrrrirr%r'r(rs,  zStackedResidualBiRNN.__init__cCsT||}||}tj||gdd|}|jD]}||}||}||}q|S)rNdim)rTrMr>r?rI)r$rOrPrQr"r'r'r(r2s     zStackedResidualBiRNN.forward)rDrETrRr'r'r%r(rSsrScs6eZdZdZ      d fdd Zd d ZZS) DPRNNBlockatDual-Path RNN Block as proposed in [1]. Args: in_chan (int): Number of input channels. hid_size (int): Number of hidden neurons in the RNNs. norm_type (str, optional): Type of normalization to use. To choose from - ``'gLN'``: global Layernorm - ``'cLN'``: channelwise Layernorm bidirectional (bool, optional): True for bidirectional Inter-Chunk RNN. rnn_type (str, optional): Type of RNN used. Choose from ``'RNN'``, ``'LSTM'`` and ``'GRU'``. num_layers (int, optional): Number of layers used in each RNN. dropout (float, optional): Dropout ratio. Must be in [0, 1]. References [1] "Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separation", Yi Luo, Zhuo Chen and Takuya Yoshioka. https://arxiv.org/abs/1910.06379 gLNTrFrrc stt||r t|||||dd|_t||||||d|_nt|||||dd|_t||||||d|_t|jj ||_ t |||_ t|jj ||_t |||_dS)NT)rr)rrYrr; intra_RNN inter_RNNrrLinearr+ intra_linearr get intra_norm inter_linear inter_norm) r$in_chanhid_size norm_typerr use_mulcatrrr%r'r(rsL    zDPRNNBlock.__init__cCs|\}}}}|}|dd||||}||}||}|||||dd}||}||}|dddd||||}||}||}|||||dddd}| |}||S)z4Input shape : [batch, feats, chunk_size, num_chunks]rrVr) size transposereshaper[r^r`r\ra contiguousrb)r$rOBNKLr/r'r'r(r23s   "  $ zDPRNNBlock.forward)rZTrFrrrRr'r'r%r(rYs6rYcsLeZdZdZ          dfd d ZddZddZZS)DPRNNa`Dual-path RNN Network for Single-Channel Source Separation introduced in [1]. Args: in_chan (int): Number of input filters. n_src (int): Number of masks to estimate. out_chan (int or None): Number of bins in the estimated masks. Defaults to `in_chan`. bn_chan (int): Number of channels after the bottleneck. Defaults to 128. hid_size (int): Number of neurons in the RNNs cell state. Defaults to 128. chunk_size (int): window size of overlap and add processing. Defaults to 100. hop_size (int or None): hop size (stride) of overlap and add processing. Default to `chunk_size // 2` (50% overlap). n_repeats (int): Number of repeats. Defaults to 6. norm_type (str, optional): Type of normalization to use. To choose from - ``'gLN'``: global Layernorm - ``'cLN'``: channelwise Layernorm mask_act (str, optional): Which non-linear function to generate mask. bidirectional (bool, optional): True for bidirectional Inter-Chunk RNN (Intra-Chunk is always bidirectional). rnn_type (str, optional): Type of RNN used. Choose between ``'RNN'``, ``'LSTM'`` and ``'GRU'``. num_layers (int, optional): Number of layers in each RNN. dropout (float, optional): Dropout ratio, must be in [0,1]. References [1] "Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separation", Yi Luo, Zhuo Chen and Takuya Yoshioka. https://arxiv.org/abs/1910.06379 NdrZreluTrFrrc stt|||_|dur|n|}||_||_||_||_|dur$|n|d}||_||_ ||_ | |_ | |_ | |_ | |_||_||_| |_t| |}t||d}t|||_g}t|j D]}|t||| | | | ||dg7}qbtj||_t|||d}tt||_tt||dt|_tt||dt |_!tj||ddd|_"t#| }t$|dr|dd|_%dS||_%dS)Nrr)rerrrfrrF)biasrXrW)&rrorrcout_chanbn_chanrd chunk_sizehop_size n_repeatsn_srcremask_actrrrrrfr r_rConv1d Sequential bottleneckrJrYnetConv2dPReLU first_outTanhnet_outSigmoidnet_gatemask_netr r output_act)r$rcrzrurvrdrwrxryrer{rrrfrr layer_normbottleneck_convrrO net_out_conv mask_nl_classr%r'r(rksZ    zDPRNN.__init__c Cs&|\}}}||}t|d|jdf|jdf|jdfd}|jd}|||j|j|}| |}| |}|||j |j|j|}|j|j}t |||j |||df|jdf|jdf|jdfd}|||j |jd}| |||}||}||} | ||j |j|} | S)zForward. Args: mixture_w (:class:`torch.Tensor`): Tensor of shape $(batch, nfilters, nframes)$ Returns: :class:`torch.Tensor`: estimated mask of shape $(batch, nsrc, nfilters, nframes)$ rVrr) kernel_sizepaddingstride)rgr~r unsqueezerwrxshaperirvrrrzrrrrrviewru) r$ mixture_wbatch n_filtersn_framesr/n_chunks to_unfoldscoreest_maskr'r'r(r2s6       z DPRNN.forwardcCsF|j|j|j|j|j|j|j|j|j|j |j |j |j |j |jd}|S)Nrcrurvrdrwrxryrzrer{rrrrrfrr$configr'r'r( get_configs"zDPRNN.get_config) NrprprqNrrrZrsTrFrrr5r6r7r8rr2rr:r'r'r%r(roGs$'I+rocs@eZdZdZ       dfd d Zd d Zd dZZS) LSTMMaskeraLSTM mask network introduced in [1], without skip connections. Args: in_chan (int): Number of input filters. n_src (int): Number of masks to estimate. out_chan (int or None): Number of bins in the estimated masks. Defaults to `in_chan`. rnn_type (str, optional): Type of RNN used. Choose between ``'RNN'``, ``'LSTM'`` and ``'GRU'``. n_layers (int, optional): Number of layers in each RNN. hid_size (int): Number of neurons in the RNNs cell state. mask_act (str, optional): Which non-linear function to generate mask. bidirectional (bool, optional): Whether to use BiLSTM dropout (float, optional): Dropout ratio, must be in [0,1]. References [1]: Yi Luo et al. "Real-time Single-channel Dereverberation and Separation with Time-domain Audio Separation Network", Interspeech 2018 NlstmrD333333?sigmoidTc st||_||_|dur|n|}||_||_||_||_||_||_ | |_ t |} t | dr9| dd|_n| |_|t| d} | rMt||_nt||_ttd|||| |dt| |j||j|_dS)NrXrrWr)rrrr)rrrcrzrurrrdrr{rr r_rrintrbn_layerr rr}rr]masker) r$rcrzrurrrdrr{rrout_sizer%r'r(r s>      zLSTMMasker.__init__cCsF|jd}||}||dddd}|||j|jd}|S)NrrV)rrrrhrrzru)r$rO batch_sizeto_sep est_masksr'r'r(r2<s  zLSTMMasker.forwardc Cs.|j|j|j|j|j|j|j|j|jd }|S)N rcrzrurrrdrr{rrrr'r'r(rCs zLSTMMasker.get_config)NrrDrrrTrr'r'r%r(rs3rcs4eZdZdZ d fdd Zdejfd d ZZS) DCCRMaskNetRNNaRNN (LSTM) layer between encoders and decoders introduced in [1]. Args: in_size (int): Number of inputs to the RNN. Must be the product of non-batch, non-time dimensions of output shape of last encoder, i.e. if the last encoder output shape is $(batch, nchans, nfreqs, time)$, `in_size` must be $nchans * nfreqs$. hid_size (int, optional): Number of units in RNN. rnn_type (str, optional): Type of RNN to use. See ``SingleRNN`` for valid values. n_layers (int, optional): Number of layers used in RNN. norm_type (Optional[str], optional): Norm to use after linear. See ``asteroid.masknn.norms`` for valid values. (Not used in [1]). rnn_kwargs (optional): Passed to :func:`~.recurrent.SingleRNN`. References [1] : "DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancement", Yanxin Hu et al. https://arxiv.org/abs/2008.00264 rprrNc sLttj|||fd|i||_ttj|jj||_ t ||_ dS)Nr) rrrComplexSingleRNNr"ComplexMultiplicationWrapperrr]r+linearr get_complexnorm)r$in_sizerdrrre rnn_kwargsr%r'r(rfs  zDCCRMaskNetRNN.__init__rOc Cs|jd|jdgtd|jdR}|||jg|jdddRj|j}|jdgtd|jdR}|jdurE||}|S)zInput shape: [batch, ..., time]rrNrrV)permutendimrJrr"rirr)r$rOr'r'r(r2ss &2  zDCCRMaskNetRNN.forward)rprrN) r5r6r7r8rr ComplexTensorr2r:r'r'r%r(rRs  rcs,eZdZdZeZfddZddZZS) DCCRMaskNetaMasking part of DCCRNet, as proposed in [1]. Valid `architecture` values for the ``default_architecture`` classmethod are: "DCCRN" and "mini". Args: encoders (list of length `N` of tuples of (in_chan, out_chan, kernel_size, stride, padding)): Arguments of encoders of the u-net decoders (list of length `N` of tuples of (in_chan, out_chan, kernel_size, stride, padding)) Arguments of decoders of the u-net n_freqs (int): Number of frequencies (dim 1) of input to ``.forward()``. Must be divisible by $f_0 * f_1 * ... * f_N$ where $f_k$ are the frequency strides of the encoders. Input shape is expected to be $(batch, nfreqs, time)$, with $nfreqs$ divisible by $f_0 * f_1 * ... * f_N$ where $f_k$ are the frequency strides of the encoders. References [1] : "DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancement", Yanxin Hu et al. https://arxiv.org/abs/2008.00264 c stjdd|Ddd|_|j\}}|ddtt||f}ddlmmtj d gfdd |Dt t|t j gfd d |ddDtj|dd |dS) NcSsg|] \}}}}}|qSr'r').0r1 enc_strider'r'r( sz(DCCRMaskNet.__init__..r)axisrVrDCUNetComplexDecoderBlockDCUNetComplexEncoderBlockc3|] }|ddiVqdS activationpreluNr'rargs)rr'r( z'DCCRMaskNet.__init__..c3rrr'r)rr'r(rr)encodersdecoders output_layerr')npprodencoders_stride_productrceil convolutionalrrrrrr>rIdentityrComplexConvTranspose2d)r$rrn_freqskwargs freq_prodr1last_encoder_out_shaper%rr(rs(    zDCCRMaskNet.__init__cCs4|j\}}|jd|rtd|d|jd|S)Nrz?Input shape must be [batch, freq, time] with freq divisible by z, got z instead)rr TypeError)r$rOrr1r'r'r(fix_input_dimss zDCCRMaskNet.fix_input_dims) r5r6r7r8r _architecturesrrr:r'r'r%r(rs  r)r>rtorch.nn.functionalrrnumpyrrutilsrr r _dccrn_architecturesr baser r rModulerr;rCrSrYrorrrr'r'r'r(s(     3=-=_.^.