o ϯi,!@sddlmZmZddlZddlmZddlmZddlm Z ddddej ej ddd ej d d ej d d ededeedeedeededededeedeedeedededededejf ddZddddej ddd ej d d ej d d ededeedeedeedededeedeedeedededededejfd!d"ZGd#d$d$ejZGd%d&d&ejZd'd(d)d*ej d+d ededeedeedeedededejfd,d-ZdS).)CallableTupleN) ConvReduce3D)set_attributes)r)r )rrrF)rrrgh㈵>g?) conv_kernel_size conv_stride conv_padding conv_biasconvpoolpool_kernel_size pool_stride pool_paddingnormnorm_eps norm_momentum activation in_channels out_channelsr r r r rrrrrrrrrreturncCsj|||||||d}| durdn| || | d}|durdn|}|dur&dn||| | d}t||||dS)u  Creates the basic resnet stem layer. It performs spatiotemporal Convolution, BN, and Relu following by a spatiotemporal pooling. :: Conv3d ↓ Normalization ↓ Activation ↓ Pool3d Normalization options include: BatchNorm3d and None (no normalization). Activation options include: ReLU, Softmax, Sigmoid, and None (no activation). Pool3d options include: AvgPool3d, MaxPool3d, and None (no pooling). Args: in_channels (int): input channel size of the convolution. out_channels (int): output channel size of the convolution. conv_kernel_size (tuple): convolutional kernel size(s). conv_stride (tuple): convolutional stride size(s). conv_padding (tuple): convolutional padding size(s). conv_bias (bool): convolutional bias. If true, adds a learnable bias to the output. conv (callable): Callable used to build the convolution layer. pool (callable): a callable that constructs pooling layer, options include: nn.AvgPool3d, nn.MaxPool3d, and None (not performing pooling). pool_kernel_size (tuple): pooling kernel size(s). pool_stride (tuple): pooling stride size(s). pool_padding (tuple): pooling padding size(s). norm (callable): a callable that constructs normalization layer, options include nn.BatchNorm3d, None (not performing normalization). norm_eps (float): normalization epsilon. norm_momentum (float): normalization momentum. activation (callable): a callable that constructs activation layer, options include: nn.ReLU, nn.Softmax, nn.Sigmoid, and None (not performing activation). Returns: (nn.Module): resnet basic stem layer. rr kernel_sizestridepaddingbiasN num_featuresepsmomentumrrrrrrr)ResNetBasicStem)rrr r r r rrrrrrrrr conv_module norm_moduleactivation_module pool_moduler*L/home/ubuntu/.local/lib/python3.10/site-packages/pytorchvideo/models/stem.pycreate_res_basic_stem s2E  r,)rrr) r r r r rrrrrrrrc Cst|||dddfd|d|dff||f|dddfd|d|dff||fdd}| dur1dn| || | d}| dur>dn| }|durGdn|||| d}t||||d S) u Creates the acoustic resnet stem layer. It performs a spatial and a temporal Convolution in parallel, then performs, BN, and Relu following by a spatiotemporal pooling. :: Conv3d Conv3d ↓ Normalization ↓ Activation ↓ Pool3d Normalization options include: BatchNorm3d and None (no normalization). Activation options include: ReLU, Softmax, Sigmoid, and None (no activation). Pool3d options include: AvgPool3d, MaxPool3d, and None (no pooling). Args: in_channels (int): input channel size of the convolution. out_channels (int): output channel size of the convolution. conv_kernel_size (tuple): convolutional kernel size(s). conv_stride (tuple): convolutional stride size(s), it will be performed as temporal and spatial convolution in parallel. conv_padding (tuple): convolutional padding size(s), it will be performed as temporal and spatial convolution in parallel. conv_bias (bool): convolutional bias. If true, adds a learnable bias to the output. pool (callable): a callable that constructs pooling layer, options include: nn.AvgPool3d, nn.MaxPool3d, and None (not performing pooling). pool_kernel_size (tuple): pooling kernel size(s). pool_stride (tuple): pooling stride size(s). pool_padding (tuple): pooling padding size(s). norm (callable): a callable that constructs normalization layer, options include nn.BatchNorm3d, None (not performing normalization). norm_eps (float): normalization epsilon. norm_momentum (float): normalization momentum. activation (callable): a callable that constructs activation layer, options include: nn.ReLU, nn.Softmax, nn.Sigmoid, and None (not performing activation). Returns: (nn.Module): resnet basic stem layer. rrr sum)rrrrrrreduction_methodNrr#r$)rr%)rrr r r r rrrrrrrrr&r'r(r)r*r*r+create_acoustic_res_basic_stemns8E  r/c s`eZdZdZddddddejdejdejdejddf fd d Zd ejdejfd d Z Z S)r%u ResNet basic 3D stem module. Performs spatiotemporal Convolution, BN, and activation following by a spatiotemporal pooling. :: Conv3d ↓ Normalization ↓ Activation ↓ Pool3d The builder can be found in `create_res_basic_stem`. Nr$rrrrrcs(tt|t|jdusJdS)a Args: conv (torch.nn.modules): convolutional module. norm (torch.nn.modules): normalization module. activation (torch.nn.modules): activation module. pool (torch.nn.modules): pooling module. N)super__init__rlocalsr)selfrrrr __class__r*r+r1s  zResNetBasicStem.__init__xcCsJ||}|jdur||}|jdur||}|jdur#||}|SNr$r3r6r*r*r+forwards       zResNetBasicStem.forward __name__ __module__ __qualname____doc__nnModuler1torchTensorr9 __classcell__r*r*r4r+r%s$r%csBeZdZdZdddejddffddZdejfdd Z Z S) PatchEmbedu Transformer basic patch embedding module. Performs patchifying input, flatten and and transpose. :: PatchModel ↓ flatten ↓ transpose The builder can be found in `create_patch_embed`. N patch_modelrFrcs(tt|t|jdusJdSr7)r0r1rr2rF)r3rFr4r*r+r1s  zPatchEmbed.__init__cCs||}|dddS)Nr r)rFflatten transposer8r*r*r+r9!s zPatchEmbed.forwardr:r*r*r4r+rDs rD)rrI)rrJ)rrrT)r r r r rcCs|||||||d}t|dS)u Creates the transformer basic patch embedding. It performs Convolution, flatten and transpose. :: Conv3d ↓ flatten ↓ transpose Args: in_channels (int): input channel size of the convolution. out_channels (int): output channel size of the convolution. conv_kernel_size (tuple): convolutional kernel size(s). conv_stride (tuple): convolutional stride size(s). conv_padding (tuple): convolutional padding size(s). conv_bias (bool): convolutional bias. If true, adds a learnable bias to the output. conv (callable): Callable used to build the convolution layer. Returns: (nn.Module): transformer patch embedding layer. rrE)rD)rrr r r r rr&r*r*r+create_conv_patch_embed's# rK)typingrrrAtorch.nnr? pytorchvideo.layers.convolutionsrpytorchvideo.layers.utilsrConv3d MaxPool3d BatchNorm3dReLUintboolfloatr@r,r/r%rDrKr*r*r*r+s        h     i0$