o i)@sdZddlmZmZmZmZddlZddlmZddl m Z m Z m Z m Z ddlmZddlmZmZmZmZGdd d ejjZdS) zEncoder for Transducer model.)AnyDictListTupleN)check_argument_types)build_body_blocksbuild_input_blockbuild_main_parametersbuild_positional_encoding)validate_architecture)TooShortUttErrorcheck_short_uttmake_chunk_maskmake_source_maskc seZdZdZiifdedeeeefdeeefdeeefddf fdd Z d ed edefd d Z dede j ddfddZ de jde jdee je jffddZ  dde jde jde jdedede jf ddZZS)EncoderzEncoder module definition. Args: input_size: Input size. body_conf: Encoder body configuration. input_conf: Encoder input configuration. main_conf: Encoder main configuration. input_size body_conf input_conf main_confreturnNcstts Jt|||\}}tdi|}t|||_t|||_t ||||_ ||_ |d|_ |d|_ |d|_|d|_dS)zConstruct an Encoder object.dynamic_chunk_trainingshort_chunk_thresholdshort_chunk_sizeleft_chunk_sizeN)super__init__rr r rembedr pos_encrencoders output_sizerrrr)selfrrrr embed_sizer main_params __class__rZ/home/ubuntu/.local/lib/python3.10/site-packages/espnet2/asr_transducer/encoder/encoder.pyr"s       zEncoder.__init__size hop_lengthcCs|j||S)aPReturn the corresponding number of sample for a given chunk size, in frames. Where size is the number of features frames after applying subsampling. Args: size: Number of frames after subsampling. hop_length: Frontend's hop length Returns: : Number of raw samples )rget_size_before_subsampling)r!r'r(rrr&get_encoder_input_raw_size>s z"Encoder.get_encoder_input_raw_size left_contextdevicecCs|j||S)zInitialize/Reset encoder streaming cache. Args: left_context: Number of frames in left context. device: Device ID. )rreset_streaming_cache)r!r+r,rrr&r-MszEncoder.reset_streaming_cachexx_lenc Cst|jj|d\}}|r$td|ddd|d|d|t|}|||\}}||}|jre|d}t d|d }|||j krP|}n||j d}t |d||j|jd} nd} |j|||| d }||d dfS) aEncode input sequences. Args: x: Encoder input features. (B, T_in, F) x_len: Encoder input features lengths. (B,) Returns: x: Encoder outputs. (B, T_out, D_enc) x_len: Encoder outputs lenghts. (B,) zhas z) frames and is too short for subsampling z(it needs more than z frames), return empty results)r0)rr,N) chunk_maskr)r rsubsampling_factorr'r rrrtorchrandintitemrrrrr,reqsum) r!r.r/ short_status limit_sizemaskrmax_len chunk_sizer1rrr&forwardWsD   zEncoder.forward rprocessed_frames right_contextc Cst|}|||\}}|dkr,tj||jdd|d}||k}tj||gdd}|j||d}|j j |||||d}|dkrP|ddd| ddf}|S)aEncode input sequences as chunks. Args: x: Encoder input features. (1, T_in, F) x_len: Encoder input features lengths. (1,) processed_frames: Number of frames already seen. left_context: Number of frames in left context. right_context: Number of frames in right context. Returns: x: Encoder outputs. (B, T_out, D_enc) r)r,r0)dim)r+)r+r@N) rrr3aranger,viewflipcatrr chunk_forward) r!r.r/r?r+r@r:processed_maskrrrr&rFs(zEncoder.chunk_forward)r>r)__name__ __module__ __qualname____doc__intrrstrrrr*r3r,r-Tensorrr=tensorrF __classcell__rrr$r&rsN    Ar)rKtypingrrrrr3 typeguardr'espnet2.asr_transducer.encoder.buildingrrr r )espnet2.asr_transducer.encoder.validationr espnet2.asr_transducer.utilsr r rrnnModulerrrrr&s