o %ݫi8@sdZddlZddlZddlZddlmZddlmmZddl m m Z ddl mZddlmZmZddlmZddlmZdZGdd d ejZGd d d ejZd*d d ZGdddejZGdddejZGdddZGdddejZ GdddejZ!GdddejZ"GdddejZ#GdddejZ$GdddejZ%d d!Z&Gd"d#d#ejZ'Gd$d%d%ejZ(Gd&d'd'ejZ)Gd(d)d)ejZ*dS)+zLibrary to support dual-path speech separation. Authors * Cem Subakan 2020 * Mirco Ravanelli 2020 * Samuele Cornell 2020 * Mirko Bronzi 2020 * Jianyuan Zhong 2020 N)ConformerEncoder)PositionalEncodingTransformerEncoder)SwishLinear:0yE>c*eZdZdZdfdd ZddZZS) GlobalLayerNormarCalculate Global Layer Normalization. Arguments --------- dim : (int or list or torch.Size) Input shape from an expected input of size. shape : tuple Expected shape of the input. eps : float A value added to the denominator for numerical stability. elementwise_affine : bool A boolean value that when set to True, this module has learnable per-element affine parameters initialized to ones (for weights) and zeros (for biases). Example ------- >>> x = torch.randn(5, 10, 20) >>> GLN = GlobalLayerNorm(10, 3) >>> x_norm = GLN(x) rTcst||_||_||_|jrK|dkr+tt|jd|_ tt |jd|_ |dkrItt|jdd|_ tt |jdd|_ dSdS| dd| dddS)Nweightbias) super__init__dimepselementwise_affinenn Parametertorchonesrzerosrregister_parameter)selfrshaperr __class__V/home/ubuntu/.local/lib/python3.10/site-packages/speechbrain/lobes/models/dual_path.pyr5s  zGlobalLayerNorm.__init__cCs|dkr>> x = torch.randn(5, 10, 20) >>> CLN = CumulativeLayerNorm(10) >>> x_norm = CLN(x) Trcstj|||ddS)Nrrrr)rrrrrrr rzCumulativeLayerNorm.__init__csx|dkr |dddd}t|}|dddd}|dkr:t|dd}t|}t|dd}|S)zReturns the normalized tensor. Arguments --------- x : torch.Tensor torch.Tensor size [N, C, K, S] or [N, C, L] Returns ------- out : torch.Tensor The normalized outputs. r rr!r r )rpermute contiguousrr'r transposerr%rrr r's    zCumulativeLayerNorm.forward)Trr(rrrr r.osr.cCsP|dkr t||d|dS|dkrt|d|dS|dkr#tjd||dSt|S)z0Just a wrapper to select the normalization type.glnTr/clnlnr )r)r r.r GroupNorm BatchNorm1d)normrrrrrr select_norms r<cs*eZdZdZd fdd ZddZZS) EncoderaConvolutional Encoder Layer. Arguments --------- kernel_size : int Length of filters. out_channels : int Number of output channels. in_channels : int Number of input channels. Example ------- >>> x = torch.randn(2, 1000) >>> encoder = Encoder(kernel_size=4, out_channels=64) >>> h = encoder(x) >>> h.shape torch.Size([2, 64, 499]) r!@r cs0ttj||||dddd|_||_dS)Nr!r F) in_channels out_channels kernel_sizestridegroupsr)rrrConv1dconv1dr?)rrAr@r?rrr rs  zEncoder.__init__cCs0|jdkr tj|dd}||}t|}|S)aReturn the encoded output. Arguments --------- x : torch.Tensor Input tensor with dimensionality [B, L]. Returns ------- x : torch.Tensor Encoded tensor with dimensionality [B, N, T_out]. where B = Batchsize L = Number of timepoints N = Number of filters T_out = Number of timepoints at the output of the encoder r r)r?r unsqueezerEFrelur5rrr r's   zEncoder.forward)r!r>r r(rrrr r=s r=cs,eZdZdZfddZfddZZS)DecoderazA decoder layer that consists of ConvTranspose1d. Arguments --------- *args : tuple **kwargs : dict Arguments passed through to nn.ConvTranspose1d Example ------- >>> x = torch.randn(2, 100, 1000) >>> decoder = Decoder(kernel_size=4, in_channels=100, out_channels=1) >>> h = decoder(x) >>> h.shape torch.Size([2, 1003]) cstj|i|dSNr0)rargskwargsrrr rr1zDecoder.__init__csr|dvrtd|jt|dkr|nt|d}t|dkr2tj|dd}|St|}|S)apReturn the decoded output. Arguments --------- x : torch.Tensor Input tensor with dimensionality [B, N, L]. where, B = Batchsize, N = number of filters L = time points Returns ------- out : torch.Tensor The decoded outputs. )r!r z{} accept 3/4D tensor as inputr r rF) r RuntimeErrorformatr)rr'rrGsqueezer5rrr r's  $ zDecoder.forwardr(rrrr rJs rJc@s eZdZdZddZddZdS) IdentityBlockaThis block is used when we want to have identity transformation within the Dual_path block. Arguments --------- **kwargs : dict Arguments are ignored. Example ------- >>> x = torch.randn(10, 100) >>> IB = IdentityBlock() >>> xhat = IB(x) cKsdSrKr)rrMrrr _init__.zIdentityBlock._init__cCs|SrKrr5rrr __call__1rSzIdentityBlock.__call__N)r)r*r+r,rRrTrrrr rQs rQc6eZdZdZ      d fdd Zd d ZZS) FastTransformerBlockaThis block is used to implement fast transformer models with efficient attention. The implementations are taken from https://fast-transformers.github.io/ Arguments --------- attention_type : str Specifies the type of attention. Check https://fast-transformers.github.io/ for details. out_channels : int Dimensionality of the representation. num_layers : int Number of layers. nhead : int Number of attention heads. d_ffn : int Dimensionality of positional feed-forward. dropout : float Dropout drop rate. activation : str Activation function. reformer_bucket_size : int bucket size for reformer. Example ------- # >>> x = torch.randn(10, 100, 64) # >>> block = FastTransformerBlock('linear', 64) # >>> x = block(x) # >>> x.shape # torch.Size([10, 100, 64]) rrI c sTtddlm} | j|||||||||||d } | |_||_||_dS)Nr)TransformerEncoderBuilder) attention_typen_layersn_headsfeed_forward_dimensionsquery_dimensionsvalue_dimensionsdropoutattention_dropout chunk_size) rrfast_transformers.buildersr[ from_kwargsgetmdlr\reformer_bucket_size) rr\r@ num_layersnheadd_ffnrb activationrir[builderrrr rWs   zFastTransformerBlock.__init__c Cs|jdkrA|jd|jd|jd}|j}tj|t|d||d|gdd}| |}|ddd| ddfS| |S)aYReturns the transformed input. Arguments --------- x : torch.Tensor Tensor shaper [B, L, N]. where, B = Batchsize, N = number of filters L = time points Returns ------- out : torch.Tensor The transformed outputs. reformerr!r rrFN) r\rirdevicercatrsizetorh)rr%pad_sizerqx_paddedrrr r'us "  zFastTransformerBlock.forward)rWrXrYrrIrZr(rrrr rV5s%rVcr ) PyTorchPositionalEncodingaePositional encoder for the pytorch transformer. Arguments --------- d_model : int Representation dimensionality. dropout : float Dropout drop prob. max_len : int Max sequence length. Example ------- >>> x = torch.randn(10, 100, 64) >>> enc = PyTorchPositionalEncoding(64) >>> x = enc(x) 皙?csttj|d|_t||}tjd|tjd d}t td|dt d |}t |||dddddf<t|||dddddf<| ddd}|d|dS)N)pr)dtyper r!g@pe)rrrDropoutrbrrarangefloatrGexpmathlogsincosr4register_buffer)rd_modelrbmax_lenr|positiondiv_termrrr rs     z"PyTorchPositionalEncoding.__init__cCs*||jd|dddf}||S)aPReturns the encoded output. Arguments --------- x : torch.Tensor Tensor shape [B, L, N], where, B = Batchsize, N = number of filters L = time points Returns ------- out : torch.Tensor The encoded output. Nr)r|rsrbr5rrr r's  z!PyTorchPositionalEncoding.forward)rxryr(rrrr rwsrwcrU) PytorchTransformerBlockaA wrapper that uses the pytorch transformer block. Arguments --------- out_channels : int Dimensionality of the representation. num_layers : int Number of layers. nhead : int Number of attention heads. d_ffn : int Dimensionality of positional feed forward. dropout : float Dropout drop rate. activation : str Activation function. use_positional_encoding : bool If true we use a positional encoding. Example ------- >>> x = torch.randn(10, 100, 64) >>> block = PytorchTransformerBlock(64) >>> x = block(x) >>> x.shape torch.Size([10, 100, 64]) rWrXrxrITc sJttj|||||d}tj||d|_|r t||_dSd|_dS)N)rrkdim_feedforwardrbrm)rj)rrrTransformerEncoderLayerrrhrw pos_encoder) rr@rjrkrlrbrmuse_positional_encoding encoder_layerrrr rs  z PytorchTransformerBlock.__init__cCs|jdur ||}||S)aWReturns the transformed output. Arguments --------- x : torch.Tensor Tensor shape [B, L, N] where, B = Batchsize, N = number of filters L = time points Returns ------- out : torch.Tensor The transformed output. N)rrhr5rrr r's   zPytorchTransformerBlock.forward)rWrXrrxrITr(rrrr rsrcs<eZdZdZ         d fdd Zd d ZZS) SBTransformerBlockaA wrapper for the SpeechBrain implementation of the transformer encoder. Arguments --------- num_layers : int Number of layers. d_model : int Dimensionality of the representation. nhead : int Number of attention heads. d_ffn : int Dimensionality of positional feed forward. input_shape : tuple Shape of input. kdim : int Dimension of the key (Optional). vdim : int Dimension of the value (Optional). dropout : float Dropout rate. activation : str Activation function. use_positional_encoding : bool If true we use a positional encoding. norm_before : bool Use normalization before transformations. attention_type : str Type of attention to use, default "regularMHA" Example ------- >>> x = torch.randn(10, 100, 64) >>> block = SBTransformerBlock(1, 64, 8) >>> x = block(x) >>> x.shape torch.Size([10, 100, 64]) rNrxrIF regularMHAc spt| |_| dkrtj} n | dkrtj} ntdt||||||||| | | d |_| r6t |d|_ dSdS)NrIgeluunknown activation) rjrkrl input_shaperkdimvdimrbrmnormalize_beforer\ input_size) rrrrReLUGELU ValueErrorrrhrpos_enc) rrjrrkrlrrrrbrmr norm_beforer\rrr rBs. zSBTransformerBlock.__init__cCs0|jr||}|||dS||dS)aXReturns the transformed output. Arguments --------- x : torch.Tensor Tensor shape [B, L, N], where, B = Batchsize, L = time points N = number of filters Returns ------- out : torch.Tensor The transformed output. r)rrrh)rr%rrrr r'ls zSBTransformerBlock.forward) rNNNrxrIFFrr(rrrr rs+*rcs0eZdZdZ   d fdd ZddZZS) SBRNNBlockaaRNNBlock for the dual path pipeline. Arguments --------- input_size : int Dimensionality of the input features. hidden_channels : int Dimensionality of the latent layer of the rnn. num_layers : int Number of the rnn layers. rnn_type : str Type of the the rnn cell. dropout : float Dropout rate bidirectional : bool If True, bidirectional. Example ------- >>> x = torch.randn(10, 100, 64) >>> rnn = SBRNNBlock(64, 100, 1, bidirectional=True) >>> x = rnn(x) >>> x.shape torch.Size([10, 100, 200]) LSTMrTcs(ttt||||||d|_dS)N)rrjrb bidirectional)rrgetattrSBRNNrh)rrhidden_channelsrjrnn_typerbrrrr rs  zSBRNNBlock.__init__cCs||dS)aJReturns the transformed output. Arguments --------- x : torch.Tensor [B, L, N] where, B = Batchsize, N = number of filters L = time points Returns ------- out : torch.Tensor The transformed output. r)rhr5rrr r'szSBRNNBlock.forward)rrTr(rrrr rsrcs8eZdZdZ d fdd ZfddZd d ZZS) DPTNetBlockaThe DPT Net block. Arguments --------- d_model : int Number of expected features in the input (required). nhead : int Number of heads in the multiheadattention models (required). dim_feedforward : int Dimension of the feedforward network model (default=2048). dropout : float Dropout value (default=0.1). activation : str Activation function of intermediate layer, relu or gelu (default=relu). Examples -------- >>> encoder_layer = DPTNetBlock(d_model=512, nhead=8) >>> src = torch.rand(10, 100, 512) >>> out = encoder_layer(src) >>> out.shape torch.Size([10, 100, 512]) rrIc sddlm}ddlm}ddlm}ddlm} ddlm } t ||||d|_ | ||dd d d |_ |||_||dd||_| ||_| ||_|||_|||_t||_dS) Nr)MultiheadAttention)r}r) LayerNorm)r)rbr!r T)r)torch.nn.modules.activationrtorch.nn.modules.dropoutr}torch.nn.modules.linearrtorch.nn.modules.normalizationrtorch.nn.modules.rnnrrr self_attnrnnrblinear2norm1norm2dropout1dropout2_get_activation_fnrm) rrrkrrbrmrr}rrrrrr rs           zDPTNetBlock.__init__cs"d|vr tj|d<t|dS)Nrm)rHrIr __setstate__)rstaterrr rs zDPTNetBlock.__setstate__cCsx|j|||dddd}|||}||}||d}||}||}||}|||}||}|S)a=Pass the input through the encoder layer. Arguments --------- src : torch.Tensor Tensor shape [B, L, N] where, B = Batchsize, N = number of filters L = time points Returns ------- Encoded outputs. N) attn_maskkey_padding_maskr) rrrrrmrbrrr)rsrcsrc2rrr r's      zDPTNetBlock.forward)rrrI)r)r*r+r,rrr'r-rrrr rs  rcCs |dkrtjS|dkrtjSdS)z/Just a wrapper to get the activation functions.rIrN)rHrIr)rmrrr rs rcs0eZdZdZ   dfdd ZddZZS) Dual_Computation_BlockaEComputation block for dual-path processing. Arguments --------- intra_mdl : torch.nn.module Model to process within the chunks. inter_mdl : torch.nn.module Model to process across the chunks. out_channels : int Dimensionality of inter/intra model. norm : str Normalization type. skip_around_intra : bool Skip connection around the intra layer. linear_layer_after_inter_intra : bool Linear layer or not after inter or intra. Example ------- >>> intra_block = SBTransformerBlock(1, 64, 8) >>> inter_block = SBTransformerBlock(1, 64, 8) >>> dual_comp_block = Dual_Computation_Block(intra_block, inter_block, 64) >>> x = torch.randn(10, 64, 100, 10) >>> x = dual_comp_block(x) >>> x.shape torch.Size([10, 64, 100, 10]) r8Tcst||_||_||_||_||_|dur&t||d|_t||d|_ |r]t |t r:t |d|j jjd|_nt ||d|_t |t rTt |d|j jjd|_dSt ||d|_dSdS)Nr r!r)rr intra_mdl inter_mdlskip_around_intralinear_layer_after_inter_intrar;r< intra_norm inter_norm isinstancerrrhr hidden_size intra_linear inter_linear)rrrr@r;rrrrr r@s2      zDual_Computation_Block.__init__c Cs|j\}}}}|dddd||||}||}|jr%||}|||||}|dddd}|jdurA||}|j rH||}|dddd||||}| |}|jrf| |}|||||}|dddd}|jdur| |}||}|S)aReturns the output tensor. Arguments --------- x : torch.Tensor Input tensor of dimension [B, N, K, S]. Returns ------- out: torch.Tensor Output tensor of dimension [B, N, K, S]. where, B = Batchsize, N = number of filters K = time points in each chunk S = the number of chunks rr r!r N) rr2r3viewrrrr;rrrrr) rr%BNKSintrainteroutrrr r'js*"    "    zDual_Computation_Block.forward)r8TTr(rrrr r#s!*rcsReZdZdZ        dfd d Zd d Zd dZddZddZZ S)Dual_Path_ModelaThe dual path model which is the basis for dualpathrnn, sepformer, dptnet. Arguments --------- in_channels : int Number of channels at the output of the encoder. out_channels : int Number of channels that would be inputted to the intra and inter blocks. intra_model : torch.nn.module Model to process within the chunks. inter_model : torch.nn.module model to process across the chunks, num_layers : int Number of layers of Dual Computation Block. norm : str Normalization type. K : int Chunk length. num_spks : int Number of sources (speakers). skip_around_intra : bool Skip connection around intra. linear_layer_after_inter_intra : bool Linear layer after inter and intra. use_global_pos_enc : bool Global positional encodings. max_length : int Maximum sequence length. Example ------- >>> intra_block = SBTransformerBlock(1, 64, 8) >>> inter_block = SBTransformerBlock(1, 64, 8) >>> dual_path_model = Dual_Path_Model(64, 64, intra_block, inter_block, num_spks=2) >>> x = torch.randn(10, 64, 2000) >>> x = dual_path_model(x) >>> x.shape torch.Size([2, 10, 64, 2000]) r r8r!TF Nc  s t||_||_||_t||d|_tj||ddd|_ | |_ |j r*t | |_ t g|_t|D]} |jtt||||| | dq4tj|||dd|_tj||ddd|_t|_t|_tt||dt|_tt||dt|_dS)Nr r F)r)rr)rA) rrrnum_spksrjr<r;rrDrEuse_global_pos_encrr ModuleListdual_mdlrangeappendcopydeepcopyrConv2dconv2d end_conv1x1PReLUprelurrm SequentialTanhoutputSigmoid output_gate)rr?r@ intra_model inter_modelrjr;rrrrr max_lengthirrr rsF        zDual_Path_Model.__init__c Cs||}||}|jr#||dddd||dd}|||j\}}t|j D] }|j ||}q1| |}| |}|j \}}}}|||jd||}|||}||||}||}|j \}}} |||j|| }||}|dd}|S)aReturns the output tensor. Arguments --------- x : torch.Tensor Input tensor of dimension [B, N, L]. Returns ------- out : torch.Tensor Output tensor of dimension [spks, B, N, L] where, spks = Number of speakers B = Batchsize, N = number of filters L = the number of time points r rpg?r)r;rErrr4rs _Segmentationrrrjrrrrrr _over_addrrrrm) rr%gaprr_rrrLrrr r's*          zDual_Path_Model.forwardc Cs|j\}}}|d}|||||}|dkr3tt||||j|j}tj||gdd}tt||||j|j} tj| || gdd}||fS)aPadding the audio times. Arguments --------- input : torch.Tensor Tensor of size [B, N, L]. where, B = Batchsize, N = number of filters L = time points K : int Chunks of length. Returns ------- output : torch.Tensor Padded inputs gap : int Size of padding r!rrF) rrTensorrtyper{rtrqrr) rinputrrrrPrpad_padrrr _paddingFs zDual_Path_Model._paddingc Cs|j\}}}|d}|||\}}|ddddd| f||d|}|dddd|df||d|} tj|| gdd||d|dd}||fS)aThe segmentation stage splits Arguments --------- input : torch.Tensor Tensor with dim [B, N, L]. K : int Length of the chunks. Return ------ output : torch.Tensor Tensor with dim [B, N, K, S]. where, B = Batchsize, N = number of filters K = time points in each chunk S = the number of chunks L = the number of time points gap : int Size of padding r!Nrpr rF)rrr3rrrrr4) rrrrrrrrinput1input2rrr rns ,*$ zDual_Path_Model._Segmentationc Cs|j\}}}}|d}|dd||d|d}|ddddddd|f||ddddd|df}|dddddd|df||dddddd| f} || }|dkrv|ddddd| f}|S)aMerge the sequence with the overlap-and-add method. Arguments --------- input : torch.Tensor Tensor with dim [B, N, K, S]. gap : int Padding length. Return ------ output : torch.Tensor Tensor with dim [B, N, L]. where, B = Batchsize, N = number of filters K = time points in each chunk S = the number of chunks L = the number of time points r!r rpNr)rr4r3r) rrrrrrrrrrrrr rs DFzDual_Path_Model._over_add)r r8rr!TTFr) r)r*r+r,rr'rrrr-rrrr rs.7>("rcsXeZdZdZ         dfd d ZddZddZZS)SepformerWrappera The wrapper for the sepformer model which combines the Encoder, Masknet and the decoder https://arxiv.org/abs/2010.13154 Arguments --------- encoder_kernel_size: int The kernel size used in the encoder encoder_in_nchannels: int The number of channels of the input audio encoder_out_nchannels: int The number of filters used in the encoder. Also, number of channels that would be inputted to the intra and inter blocks. masknet_chunksize: int The chunk length that is to be processed by the intra blocks masknet_numlayers: int The number of layers of combination of inter and intra blocks masknet_norm: str, The normalization type to be used in the masknet Should be one of 'ln' -- layernorm, 'gln' -- globallayernorm 'cln' -- cumulative layernorm, 'bn' -- batchnorm -- see the select_norm function above for more details masknet_useextralinearlayer: bool Whether or not to use a linear layer at the output of intra and inter blocks masknet_extraskipconnection: bool This introduces extra skip connections around the intra block masknet_numspks: int This determines the number of speakers to estimate intra_numlayers: int This determines the number of layers in the intra block inter_numlayers: int This determines the number of layers in the inter block intra_nhead: int This determines the number of parallel attention heads in the intra block inter_nhead: int This determines the number of parallel attention heads in the inter block intra_dffn: int The number of dimensions in the positional feedforward model in the inter block inter_dffn: int The number of dimensions in the positional feedforward model in the intra block intra_use_positional: bool Whether or not to use positional encodings in the intra block inter_use_positional: bool Whether or not to use positional encodings in the inter block intra_norm_before: bool Whether or not we use normalization before the transformations in the intra block inter_norm_before: bool Whether or not we use normalization before the transformations in the inter block Example ------- >>> model = SepformerWrapper() >>> inp = torch.rand(1, 160) >>> result = model.forward(inp) >>> result.shape torch.Size([1, 160, 2]) r rr!r8FTrXrYc stt|||d|_t| || |||d}t| || |||d}t|||||||| ||d |_t||||ddd|_| |_ |j|j|jfD]}| |qGdS)N)rAr@r?)rjrrkrlrr) r?r@rrrjr;rrrrr!F)r?r@rArBr) rrr=encoderrrmasknetrJdecoderrreset_layer_recursively)rencoder_kernel_sizeencoder_in_nchannelsencoder_out_nchannelsmasknet_chunksizemasknet_numlayers masknet_normmasknet_useextralinearlayermasknet_extraskipconnectionmasknet_numspksintra_numlayersinter_numlayers intra_nhead inter_nhead intra_dffn inter_dffnintra_use_positionalinter_use_positionalintra_norm_beforeinter_norm_beforerrmodulerrr rsZ     zSepformerWrapper.__init__cCs6t|dr ||D] }||kr||q dS)z+Reinitializes the parameters of the networkreset_parametersN)hasattrrmodulesr)rlayer child_layerrrr r4s   z(SepformerWrapper.reset_layer_recursivelycs|}|}t|gj}||tjfddtjDdd}|d}|d}||krEt |ddd||f}|S|ddd|ddf}|S)z:Processes the input tensor x and returns an output tensor.cs g|] }|dqS)rp)rrG).0rrsep_hrr Esz,SepformerWrapper.forward..rprFr rN) rrrstackrrrrrsrHr)rmixmix_west_mask est_sourceT_originT_estrrr r'<s"    zSepformerWrapper.forward)rr rrr!r8FTr!rXrXrXrXrYrYTTTT)r)r*r+r,rrr'r-rrrr rs0;Grcs>eZdZdZ          d fd d Zd d ZZS)SBConformerEncoderBlockaA wrapper for the SpeechBrain implementation of the ConformerEncoder. Arguments --------- num_layers : int Number of layers. d_model : int Dimensionality of the representation. nhead : int Number of attention heads. d_ffn : int Dimensionality of positional feed forward. input_shape : tuple Shape of input. kdim : int Dimension of the key (Optional). vdim : int Dimension of the value (Optional). dropout : float Dropout rate. activation : str Activation function. kernel_size: int Kernel size in the conformer encoder bias: bool Use bias or not in the convolution part of conformer encoder use_positional_encoding : bool If true we use a positional encoding. attention_type : str The type of attention to use, default "RelPosMHAXL" Example ------- >>> x = torch.randn(10, 100, 64) >>> block = SBConformerEncoderBlock(1, 64, 8) >>> from speechbrain.lobes.models.transformer.Transformer import PositionalEncoding >>> pos_enc = PositionalEncoding(64) >>> pos_embs = pos_enc(torch.ones(1, 199, 64)) >>> x = block(x) >>> x.shape torch.Size([10, 100, 64]) rNrxswishT RelPosMHAXLc st| |_| |_| dkrtj} n| dkrtj} n | dkr"t} ntdt |||||||| | | | d |_ |jdkrCt |d|_ dS|jdkrU|jrSt |d|_ dSdStd ) NrIrr$r) rjrkrlrrrrbrmrArr\r&rrUnsupported attention type) rrrr\rrrrrrrhrr)rrjrrkrlrrrrbrmrArrr\rrr rs<   z SBConformerEncoderBlock.__init__cCs|jdkr(|tj|jd|jddd|jd|jd}|j||ddS|jdkrE|jr>||}|||dS||dStd) a4Returns the transformed output. Arguments --------- x : torch.Tensor Tensor shape [B, L, N], where, B = Batchsize, L = time points N = number of filters Returns ------- Transformed output r&rr r!)rq)pos_embsrr') r\rrrrrqrhrr)rr%rr(rrr r's $  zSBConformerEncoderBlock.forward) rNNNrxr$r%TTr&r(rrrr r#Ws04r#)r)+r,rrrtorch.nnrtorch.nn.functional functionalrHspeechbrain.nnet.RNNnnetRNNr.speechbrain.lobes.models.transformer.Conformerr0speechbrain.lobes.models.transformer.Transformerrrspeechbrain.nnet.activationsrspeechbrain.nnet.linearrEPSModuler rr.r<r=ConvTranspose1drJrQrVrwrrrrrrrrr#rrrr sB     Q 4 <3d5MhBU  %