o i@sddlZddlmZddlmZmZmZmZddlZddl m Z ddl m m Z ddlmZddlmZmZddlmZddZGd d d e jZGd d d e jZGd ddeZdS)N) OrderedDict)DictListOptionalTuple)DPMulCat) merge_feature split_feature) AbsSeparatorc Cs|dd}|dd\}}t||}||}||}||d|}||} |jg|d|R} td| d||} |  |j } | d} |j g|| |R} | d| | | jg|dR} | S)aReconstructs a signal from a framed representation. Adds potentially overlapping frames of a signal with shape `[..., frames, frame_length]`, offsetting subsequent frames by `frame_step`. The resulting tensor has shape `[..., output_size]` where output_size = (frames - 1) * frame_step + frame_length Args: signal: A [..., frames, frame_length] Tensor. All dimensions may be unknown, and rank must be at least 2. frame_step: An integer denoting overlap offsets. Must be less than or equal to frame_length. Returns: A Tensor with shape [..., output_size] containing the overlap-added frames of signal's inner-most two dimensions. output_size = (frames - 1) * frame_step + frame_length Based on https://github.com/tensorflow/tensorflow/blob/r1.12/tensorflow/contrib/signal/python/ops/reconstruction_ops.py Nr)sizemathgcdviewtorcharangeunfoldclonedetachlongtodevice contiguous new_zeros index_add_) signal frame_stepouter_dimensionsframes frame_lengthsubframe_length subframe_stepsubframes_per_frame output_sizeoutput_subframessubframe_signalframeresultr*Z/home/ubuntu/.local/lib/python3.10/site-packages/espnet2/enh/separator/svoice_separator.pyoverlap_and_adds"  r,cs.eZdZdedeffdd ZddZZS)Encoderenc_kernel_size enc_feat_dimcs2ttjd|||ddd|_t|_dS)Nr F) kernel_sizestridebias)super__init__nnConv1dconvReLU nonlinear)selfr.r/ __class__r*r+r5?s zEncoder.__init__cCs t|d}|||}|S)Nr )r unsqueezer:r8)r;mixture mixture_wr*r*r+forwardKs zEncoder.forward)__name__ __module__ __qualname__intr5rA __classcell__r*r*r<r+r->s r-cs$eZdZfddZddZZS)Decodercst||_dSN)r4r5r1)r;r1r<r*r+r5Rs  zDecoder.__init__cCs6t|dd}td|jf|}t||jd}|S)Nr0r )r transposer6 AvgPool2dr1r,)r; est_sourcer*r*r+rAVszDecoder.forward)rBrCrDr5rArFr*r*r<r+rGQs rGcseZdZdZ     ddeded ed ed ed ed ededeffdd Z ddejdejde e de e ejeje ffddZeddZZS)SVoiceSeparatoraSVoice model for speech separation. Reference: Voice Separation with an Unknown Number of Multiple Speakers; E. Nachmani et al., 2020; https://arxiv.org/abs/2003.01531 Args: enc_dim: int, dimension of the encoder module's output. (Default: 128) kernel_size: int, the kernel size of Conv1D layer in both encoder and decoder modules. (Default: 8) hidden_size: int, dimension of the hidden state in RNN layers. (Default: 128) num_spk: int, the number of speakers in the output. (Default: 2) num_layers: int, number of stacked MulCat blocks. (Default: 4) segment_size: dual-path segment size. (Default: 20) bidirectional: bool, whether the RNN layers are bidirectional. (Default: True) input_normalize: bool, whether to apply GroupNorm on the input Tensor. (Default: False) r0TF input_dimenc_dimr1 hidden_sizenum_spk num_layers segment_size bidirectionalinput_normalizec sNt||_||_||_t|||_t||_t ||||||| d|_ dS)N) input_sizerRr%rSrTrVrW) r4r5_num_spkrQrUr-encoderrGdecoderr rnn_model) r;rPrQr1rRrSrTrUrVrWr<r*r+r5ss    zSVoiceSeparator.__init__Ninputilens additionalreturncCs|d}||}t||j\}}||}g} tt|D]@} t|| |} | |j d|j |j |j d} | | } | d} t | d|| f} t| jdd} |jr]| | q| } qi} | || fS)aForward. Args: input (torch.Tensor or ComplexTensor): Encoded feature [B, T, N] ilens (torch.Tensor): input lengths [Batch] additional (Dict or None): other data included in model NOTE: not used in this model Returns: masked (List[Union(torch.Tensor, ComplexTensor)]): [(B, T, N), ...] ilens (torch.Tensor): (B,) others predicted data, e.g. masks: OrderedDict[ 'mask_spk1': torch.Tensor(Batch, Frames, Freq), 'mask_spk2': torch.Tensor(Batch, Frames, Freq), ... 'mask_spkn': torch.Tensor(Batch, Frames, Freq), ] r rr0r )dim)rrZr rUr\rangelenrrshaperYrQr[Fpadlistunbindtrainingappend)r;r]r^r_T_mixr@ enc_segmentsenc_rest output_alloutputsii output_iiT_estothersr*r*r+rAs&       zSVoiceSeparator.forwardcCs|jSrH)rY)r;r*r*r+rSszSVoiceSeparator.num_spk)r0rNrOTFrH)rBrCrD__doc__rEboolr5rTensorrrrrrrApropertyrSrFr*r*r<r+rM^sL  " 4rM)r collectionsrtypingrrrrrtorch.nnr6torch.nn.functional functionalreespnet2.enh.layers.dpmulcatrespnet2.enh.layers.dprnnrr #espnet2.enh.separator.abs_separatorr r,Moduler-rGrMr*r*r*r+s    0