o ´©iB(ã@sXdZddlZddlmZddlmZddlmZGdd„dejƒZGdd „d ejƒZ dS) z(Encoder self-attention layer definition.éN)Únn)Ú LayerNorm)ÚVariablecs2eZdZdZ   d ‡fdd„ Zd dd „Z‡ZS) ÚEncoder_Conformer_LayeráÃEncoder layer module. Args: size (int): Input dimension. self_attn (torch.nn.Module): Self-attention module instance. `MultiHeadedAttention` or `RelPositionMultiHeadedAttention` instance can be used as the argument. feed_forward (torch.nn.Module): Feed-forward module instance. `PositionwiseFeedForward`, `MultiLayeredConv1d`, or `Conv1dLinear` instance can be used as the argument. feed_forward_macaron (torch.nn.Module): Additional feed-forward module instance. `PositionwiseFeedForward`, `MultiLayeredConv1d`, or `Conv1dLinear` instance can be used as the argument. conv_module (torch.nn.Module): Convolution module instance. `ConvlutionModule` instance can be used as the argument. dropout_rate (float): Dropout rate. normalize_before (bool): Whether to use layer_norm before the first block. concat_after (bool): Whether to concat attention layer's input and output. if True, additional linear will be applied. i.e. x -> x + linear(concat(x, att(x))) if False, no additional linear will be applied. i.e. x -> x + att(x) TFrc s¼tt|ƒ ¡||_||_||_||_t|ƒ|_t|ƒ|_ |dur*t|ƒ|_ d|_ nd|_ |jdur)rr+r,Ú channel_sizer-r.r/r0Út_lengÚd_dimÚx_newÚx_k_vÚ pad_beforeÚ pad_afterÚx_padr2rrr r4ÝsD     *&ÿ&ÿ>>>>> ý   zEncoderLayer.forward)TFr5r6rrrr r<¦s !ör<) r:r)rÚ$funasr.models.transformer.layer_normrÚtorch.autogradrÚModulerr<rrrr Ús