o ix@s8dZddlmZmZmZddlZGdddejjZdS)z'Conformer block for Transducer encoder.)DictOptionalTupleNcseZdZdZejjidfdedejjdejjdejjdejjdejjd e d e d d ffd d Z dedej d d fddZ ddejdejdejdeejd eejejejff ddZ  ddejdejdejdeded eejejff ddZZS) ConformeraConformer module definition. Args: block_size: Input/output size. self_att: Self-attention module instance. feed_forward: Feed-forward module instance. feed_forward_macaron: Feed-forward module instance for macaron network. conv_mod: Convolution module instance. norm_class: Normalization module class. norm_args: Normalization module arguments. dropout_rate: Dropout rate. g block_sizeself_att feed_forwardfeed_forward_macaronconv_mod norm_class norm_args dropout_ratereturnNc st||_||_||_d|_||_||fi||_||fi||_||fi||_ ||fi||_ ||fi||_ t j ||_||_d|_dS)zConstruct a Conformer object.g?N)super__init__rrr feed_forward_scaler norm_feed_forward norm_self_att norm_macaron norm_conv norm_finaltorchnnDropoutdropoutrcache) selfrrrr r r r r  __class__c/home/ubuntu/.local/lib/python3.10/site-packages/espnet2/asr_transducer/encoder/blocks/conformer.pyrs  zConformer.__init__ left_contextdevicecCs:tjd||jf|dtjd|j|jjdf|dg|_dS)zInitialize/Reset self-attention and convolution modules cache for streaming. Args: left_context: Number of left frames during chunk-by-chunk inference. device: Device to use for cache tensor. )r"N)rzerosrr kernel_sizer)rr!r"rrr reset_streaming_cache9s   zConformer.reset_streaming_cachexpos_encmask chunk_maskc Cs|}||}||j|||}|}||}|}|||j||||||d}|}||}||\}}|||}|}||}||j|| |}| |}|||fS)aEncode input sequences. Args: x: Conformer input sequences. (B, T, D_block) pos_enc: Positional embedding sequences. (B, 2 * (T - 1), D_block) mask: Source mask. (B, T) chunk_mask: Chunk mask. (T_2, T_2) Returns: x: Conformer output sequences. (B, T, D_block) mask: Source mask. (B, T) pos_enc: Positional embedding sequences. (B, 2 * (T - 1), D_block) )r*) rrrr rrrr rrr)rr'r(r)r*residualx_q_rrr forwardPs6       zConformer.forwardr right_contextc Cs*|}||}||j||}|}||}|dkr)tj|jd|gdd}n|}|}|dkrC|dd|| | ddf} n|dd| dddf} ||j||||||d}|}||}|j ||jd|d\}} ||}|}| |}||j| |}| |}| | g|_||fS)aEncode chunk of input sequence. Args: x: Conformer input sequences. (B, T, D_block) pos_enc: Positional embedding sequences. (B, 2 * (T - 1), D_block) mask: Source mask. (B, T_2) left_context: Number of frames in left context. right_context: Number of frames in right context. Returns: x: Conformer output sequences. (B, T, D_block) pos_enc: Positional embedding sequences. (B, 2 * (T - 1), D_block) rr#)dimN)r!)rr/) rrr rrcatrrrr rrr) rr'r(r)r!r/r+keyval att_cache conv_cacherrr chunk_forwards@  $      zConformer.chunk_forward)N)rr)__name__ __module__ __qualname____doc__rr LayerNormintModulerfloatrr"r&Tensorrrr.r6 __classcell__rrrr rsh  " ?r) r:typingrrrrrr=rrrrr s