o i @s8dZddlZddlZddlmZGdddejjZdS)zPositional encoding modules.N) _pre_hookc sleZdZdZ ddedededdffd d Zdd ejd eddfddZ dd ejd edejfddZ Z S)RelPositionalEncodingzRelative positional encoding. Args: size: Module size. max_len: Maximum input length. dropout_rate: Dropout rate. size dropout_ratemax_lenreturnNcsLt||_d|_tjj|d|_|t d d|| t dS)z.Construct a RelativePositionalEncoding object.N)pr) super__init__rpetorchnnDropoutdropout extend_petensorexpand"_register_load_state_dict_pre_hookr)selfrrr __class__n/home/ubuntu/.local/lib/python3.10/site-packages/espnet2/asr_transducer/encoder/modules/positional_encoding.pyr s zRelPositionalEncoding.__init__rx left_contextcCs|d|}|jdur3|jd|ddkr3|jj|jks&|jj|jkr1|jj|j|jd|_dSt||j}t||j}tjd|tjd d}t tjd|jdtjdt d|j }t |||dddddf<t|||dddddf<t|dg d}t d|||dddddf<td|||dddddf<|dd d}tj||gdd j|j|jd |_dS) zReset positional encoding. Args: x: Input sequences. (B, T, ?) left_context: Number of frames in left context. r N)devicedtyper)r g@)dim)r r)rrr rtorzerosarangefloat32 unsqueezeexpmathlogsincosflipcat)rrrtime1 pe_positive pe_negativepositiondiv_termrrrr"s.   $$ zRelPositionalEncoding.extend_pecCsj|j||d|d|}|jdd|jdd|d|jdd|df}||}|S)zCompute positional encoding. Args: x: Input sequences. (B, T, ?) left_context: Number of frames in left context. Returns: pos_enc: Positional embedding sequences. (B, 2 * (T - 1), ?) )rr Nr)rrrr)rrrr/pos_encrrrforwardGs 8 zRelPositionalEncoding.forward)rr)r) __name__ __module__ __qualname____doc__intfloatr rTensorrr5 __classcell__rrrrr s $%r)r9r)r1espnet.nets.pytorch_backend.transformer.embeddingrrModulerrrrrs