o ϯi.1@sddlmZddlmZmZddlZddlmZddlm Z ddl m Z ddl m Z ddlmZmZddlmZd d d ejd d d d d e ejejddejddedededeedeedeededeedeedeededeedededed ed!ed"ed#ejf&d$d%Zd&d'd(d)ejddejd*d+d,d-d.d/d-d0d1eeeefejd2d ejd3d4d5ed6ed7ed8eded ed!ed"ed9ed:eed;eedeed?eeed@eedAeedBeedCedDeedEeedFedGed#ejf0dHdIZdS)J)partial)CallableTupleN)create_conv_2plus1d)create_res_basic_head)Net)create_bottleneck_blockcreate_res_stage)create_res_basic_stemr r )rrrrr)rrr gh㈵>g?)conv_a_kernel_size conv_a_strideconv_a_paddingconv_aconv_b_kernel_size conv_b_strideconv_b_paddingconv_b_num_groupsconv_b_dilationconv_bconv_cnormnorm_eps norm_momentum activationdim_in dim_innerdim_outrrrrrrrrrrrrrrrreturnc Cstdid|d|d|d|d|d|d|d|d |d | d | d | d tt||||dd| d|d|d|d|S)u 2plus1d bottleneck block: a sequence of spatiotemporal Convolution, Normalization, and Activations repeated in the following order: :: Conv3d (conv_a) ↓ Normalization (norm_a) ↓ Activation (act_a) ↓ Conv(2+1)d (conv_b) ↓ Normalization (norm_b) ↓ Activation (act_b) ↓ Conv3d (conv_c) ↓ Normalization (norm_c) Normalization examples include: BatchNorm3d and None (no normalization). Activation examples include: ReLU, Softmax, Sigmoid, and None (no activation). Args: dim_in (int): input channel size to the bottleneck block. dim_inner (int): intermediate channel size of the bottleneck. dim_out (int): output channel size of the bottleneck. conv_a_kernel_size (tuple): convolutional kernel size(s) for conv_a. conv_a_stride (tuple): convolutional stride size(s) for conv_a. conv_a_padding (tuple): convolutional padding(s) for conv_a. conv_a (callable): a callable that constructs the conv_a conv layer, examples include nn.Conv3d, OctaveConv, etc conv_b_kernel_size (tuple): convolutional kernel size(s) for conv_b. conv_b_stride (tuple): convolutional stride size(s) for conv_b. conv_b_padding (tuple): convolutional padding(s) for conv_b. conv_b_num_groups (int): number of groups for groupwise convolution for conv_b. conv_b_dilation (tuple): dilation for 3D convolution for conv_b. conv_b (callable): a callable that constructs the conv_b conv layer, examples include nn.Conv3d, OctaveConv, etc conv_c (callable): a callable that constructs the conv_c conv layer, examples include nn.Conv3d, OctaveConv, etc norm (callable): a callable that constructs normalization layer, examples include nn.BatchNorm3d, None (not performing normalization). norm_eps (float): normalization epsilon. norm_momentum (float): normalization momentum. activation (callable): a callable that constructs activation layer, examples include: nn.ReLU, nn.Softmax, nn.Sigmoid, and None (not performing activation). Returns: (nn.Module): 2plus1d bottleneck block. rr r!rrrrrrrrrr)rrrrrrrrrN)rrr)rr r!rrrrrrrrrrrrrrrr#r#P/home/ubuntu/.local/lib/python3.10/site-packages/pytorchvideo/models/r2plus1d.pycreate_2plus1d_bottleneck_blocksVQ     r%r2ig@)r r()r rr)r r r r )r r r r )r r r r )rrrr)r r rr)r(r(T) input_channel model_depthmodel_num_class dropout_raterrrr stem_dim_outstem_conv_kernel_sizestem_conv_stridestage_conv_a_kernel_sizestage_conv_b_kernel_sizestage_conv_b_num_groupsstage_conv_b_dilationstage_spatial_stridestage_temporal_stridestage_bottleneck head_poolhead_pool_kernel_sizehead_output_sizehead_activationhead_output_with_global_averager*r+r,r-r.r/r0r1r2r3r4r5r6r7r8r9r:r;r<c#CsXtjddddd}||vsJ|d|||}g}t||| | dd| Dd ||d }|||}|d }tt|D]M}|d }||}||||||f} t||||||| |gd d d| |D| || dd| |D| |||||d}!||!|}|d}qEt ||||||||d}"||"t t |dS)u_ Build the R(2+1)D network from:: A closer look at spatiotemporal convolutions for action recognition. Du Tran, Heng Wang, Lorenzo Torresani, Jamie Ray, Yann LeCun, Manohar Paluri. CVPR 2018. R(2+1)D follows the ResNet style architecture including three parts: Stem, Stages and Head. The three parts are assembled in the following order: :: Input ↓ Stem ↓ Stage 1 ↓ . . . ↓ Stage N ↓ Head Args: input_channel (int): number of channels for the input video clip. model_depth (int): the depth of the resnet. model_num_class (int): the number of classes for the video dataset. dropout_rate (float): dropout rate. norm (callable): a callable that constructs normalization layer. norm_eps (float): normalization epsilon. norm_momentum (float): normalization momentum. activation (callable): a callable that constructs activation layer. stem_dim_out (int): output channel size for stem. stem_conv_kernel_size (tuple): convolutional kernel size(s) of stem. stem_conv_stride (tuple): convolutional stride size(s) of stem. stage_conv_a_kernel_size (tuple): convolutional kernel size(s) for conv_a. stage_conv_b_kernel_size (tuple): convolutional kernel size(s) for conv_b. stage_conv_b_num_groups (tuple): number of groups for groupwise convolution for conv_b. 1 for ResNet, and larger than 1 for ResNeXt. stage_conv_b_dilation (tuple): dilation for 3D convolution for conv_b. stage_spatial_stride (tuple): the spatial stride for each stage. stage_temporal_stride (tuple): the temporal stride for each stage. stage_bottleneck (tuple): a callable that constructs bottleneck block layer for each stage. Examples include: create_bottleneck_block, create_2plus1d_bottleneck_block. head_pool (callable): a callable that constructs resnet head pooling layer. head_pool_kernel_size (tuple): the pooling kernel size. head_output_size (tuple): the size of output tensor for head. head_activation (callable): a callable that constructs activation layer. head_output_with_global_average (bool): if True, perform global averaging on the head output. Returns: (nn.Module): basic resnet. z"PYTORCHVIDEO.model.create_r2plus1d)rr)r)rr)r)r$r)r&ez is not in cSg|]}|dqSrr#.0sizer#r#r$ z#create_r2plus1d..N) in_channels out_channelsconv_kernel_size conv_stride conv_paddingpoolrrr)r cSrCrDr#rEr#r#r$rHrIcSrCrDr#rEr#r#r$rH"rI)depthrr r! bottleneckrrrrrrrrrrr) in_features out_featuresrO output_sizepool_kernel_sizer-routput_with_global_average)blocks) torch_C_log_api_usage_oncekeysr appendrangelenr rrnn ModuleList)#r*r+r,r-rrrrr.r/r0r1r2r3r4r5r6r7r8r9r:r;r<_MODEL_STAGE_DEPTH stage_depthsrWstem stage_dim_in stage_dim_outidxstage_dim_innerrPstage_conv_b_stridestageheadr#r#r$create_r2plus1d{sv u      rk) functoolsrtypingrrrXtorch.nnr_ pytorchvideo.layers.convolutionsrpytorchvideo.models.headrpytorchvideo.models.netrpytorchvideo.models.resnetrr pytorchvideo.models.stemr Conv3d BatchNorm3dReLUintfloatModuler% AvgPool3dSoftmaxboolrkr#r#r#r$s             p       &'(/01234