o ϯiS@sddlmZmZmZmZddlZddlmZddlm Z Gdddej Z ddej dd d d e d e d eee dedeededefddZdS))CallableIterableOptionalTupleN)set_attributescspeZdZdZdddddejdejdejdejd eejd eejd ed dffd dZd e j fddZ Z S)NonLocala Builds Non-local Neural Networks as a generic family of building blocks for capturing long-range dependencies. Non-local Network computes the response at a position as a weighted sum of the features at all positions. This building block can be plugged into many computer vision architectures. More details in the paper: Wang, Xiaolong, Ross Girshick, Abhinav Gupta, and Kaiming He. "Non-local neural networks." In Proceedings of the IEEE conference on CVPR, 2018. N dot_product)poolnorm instantiation conv_thetaconv_phiconv_gconv_outr r r returncsptt|td||||fvsJ|dvs Jd|t|jj|jj|j j|j j hdks6JddS)N)rsoftmaxzUnknown norm type {}z8Nonlocal convolution's input/ output dimension mismatch.) super__init__rlocalsformatlenr out_channelsr rr in_channels)selfr r rrr r r  __class__T/home/ubuntu/.local/lib/python3.10/site-packages/pytorchvideo/layers/nonlocal_net.pyrs"   zNonLocal.__init__cCs|jj}|}|\}}}}}||} |jdur||}||} ||} | ||d} | ||d} | ||d} td| | f} |j dkrY| |d} t j j | dd} n|j dkrg| j d} | | } td| | f}||||||}||}|jdur||}||S) Nz nct,ncp->ntprg)dimrz ntg,ncg->nct)r rsizer r rviewtorcheinsumr nn functionalrshaperr )rx dim_inner x_identityNCTHWthetaphig theta_phispatial_temporal_dim theta_phi_gprrrforward5s0            zNonLocal.forward) __name__ __module__ __qualname____doc__r&Modulerstrrr$Tensorr8 __classcell__rrrrr s.  rrrrrgh㈵>g?) pool_sizer r norm_eps norm_momentumdim_inr*rBr r rCrDc Cs|durd}t|ts J|durd}n||||d}tdd|Dr/tj||gdd}nd}ttj||ddd dtj||ddd dtj||ddd dtj||ddd d|||d S) a Builds Non-local Neural Networks as a generic family of building blocks for capturing long-range dependencies. Non-local Network computes the response at a position as a weighted sum of the features at all positions. This building block can be plugged into many computer vision architectures. More details in the paper: https://arxiv.org/pdf/1711.07971 Args: dim_in (int): number of dimension for the input. dim_inner (int): number of dimension inside of the Non-local block. pool_size (tuple[int]): the kernel size of spatial temporal pooling, temporal pool kernel size, spatial pool kernel size, spatial pool kernel size in order. By default pool_size is None, then there would be no pooling used. instantiation (string): supports two different instantiation method: "dot_product": normalizing correlation matrix with L2. "softmax": normalizing correlation matrix with Softmax. norm (nn.Module): nn.Module for the normalization layer. The default is nn.BatchNorm3d. norm_eps (float): normalization epsilon. norm_momentum (float): normalization momentum. NrA) num_featuresepsmomentumcss|]}|dkVqdS)rNr).0r"rrr sz"create_nonlocal..)rrr) kernel_sizestridepaddingrr)r r rrr r r ) isinstanceranyr& MaxPool3drConv3d) rEr*rBr r rCrD norm_model pool_modelrrrcreate_nonlocal_s(" rT)typingrrrrr$torch.nnr&pytorchvideo.layers.utilsrr=r BatchNorm3dintr>floatrTrrrrs2  Z