o ϯi@sPddlZddlmZddlZddlmZGdddejZGdddejZdS)N)Tuple)nncsFeZdZdZd dededdffdd Zd ejdejfd d ZZ S) PositionalEncodinga? Applies a positional encoding to a tensor with shape (batch_size x seq_len x embed_dim). The positional encoding is computed as follows: PE(pos,2i) = sin(pos/10000^(2i/dmodel)) PE(pos,2i+1) = cos(pos/10000^(2i/dmodel)) where pos = position, pos in [0, seq_len) dmodel = data embedding dimension = embed_dim i = dimension index, i in [0, embed_dim) Reference: "Attention Is All You Need" https://arxiv.org/abs/1706.03762 Implementation Reference: https://pytorch.org/tutorials/beginner/transformer_tutorial.html  embed_dimseq_lenreturnNcsttj||tjd}tjd|tjdd}ttd|dt d |}t |||dddddf<t |||dddddf<|d}| d|dS)N)dtyperg@pe) super__init__torchzerosfloatarange unsqueezeexpmathlogsincosregister_buffer)selfrrr positiondiv_term __class__[/home/ubuntu/.local/lib/python3.10/site-packages/pytorchvideo/layers/positional_encoding.pyrs     zPositionalEncoding.__init__xcCs\|jd|dksJd|jd|||jddd|dddfS)Nr z'Cannot apply position encoding of size z when input has size )r size)rr!rrr forward&s &zPositionalEncoding.forward)r) __name__ __module__ __qualname____doc__intrrTensorr# __classcell__rrrr r s rc sheZdZdZ  ddedeeeefdededd f fd d Zed d Z de j de j fddZ Z S)#SpatioTemporalClsPositionalEncodingzJ Add a cls token and apply a spatiotemporal encoding to a tensor. FTrpatch_embed_shape sep_pos_embedhas_clsrNcstt|dksJd||_||_||_|d|d|_|d|_|jr=t t dd||_ |j|jd}n|j|j}|jrpt t d|j||_ t t d|j||_|jrnt t dd||_dSdSt t d|||_dS)ap Args: embed_dim (int): Embedding dimension for input sequence. patch_embed_shape (Tuple): The number of patches in each dimension (T, H, W) after patch embedding. sep_pos_embed (bool): If set to true, one positional encoding is used for spatial patches and another positional encoding is used for temporal sequence. Otherwise, only one positional encoding is used for all the patches. has_cls (bool): If set to true, a cls token is added in the beginning of each input sequence. z5Patch_embed_shape should be in the form of (T, H, W).r r rN)r rlen cls_embed_onr-_patch_embed_shapenum_spatial_patchnum_temporal_patchr Parameterrr cls_tokenpos_embed_spatialpos_embed_temporalpos_embed_class pos_embed)rrr,r-r. num_patchesrrr r3s0   z,SpatioTemporalClsPositionalEncoding.__init__cCs|jS)N)r2)rrrr r,bsz5SpatioTemporalClsPositionalEncoding.patch_embed_shaper!cCs|j\}}}|jr|j|dd}tj||fdd}|jrB|jd|j dtj |j |j dd}|jrs   $