o ϯiXC@sjddlmZmZmZmZddlZddlZddlm Z ddl m Z ddl m Z mZddlmZmZddlmZmZddd e jd d d d de je je jdde jddedededeedeedeededeedeedeededeededed ed!ed"ed#ed$e jf&d%d&Zddd e jddd'd de je je jdde jddedededeedeedeededeedeedeededeededed ed!ed"ed#ed$e jf&d(d)Zd*d+Zd,eddd e jd d d d de je je je jdde je jd-dededed.ed/ed0edeedeedeededeedeedeededeededed1ed ed!ed"ed2ed3ed$e jf0d4d5Zddd e jd d d d de je je jdde jdd6edededed.edeeeeeefdeedeeeeeefdedeedeedeededeededed ed!ed"ed#ed$e jf*d7d8Z d9d:d;d<Z!d=d>d?d@e je jdAdBd e j"d d edddCdDdEdFdGdGdEee e j#dHdddIdJdKedLedMedNed ed#edOedPeedQeedRedSeedTeedUedVedWeedXeeeeeefdYeeeeeefdZeed[eeeeeefd\eed]eed^eed.eeeefd_ed`edaeedbeedcedded$e jfdgd@e je jdAdhd e j"d d edddCdDdEdidjdjdEeee j#dkde j%d,dldmddn dKedLedMedNed ed#edOedPeedQeedRedSeedTeedUedVedWeedXeeeeeefdYeeeeeefdZeed[eeeeeefd\eed]eed^eed.eeeefd_ed`edaeedbeedceddedoeedpedqed$e jfBdrdsZ&d d>d?d@e je jdAdtduddvdweddddvdEddEdGdGeeeefe j#dxdddIdydKedLedMedNed ed#edOedPeedQeedRedSeedTeedUedVedWeedXeeeeeefdYeeeeeefdZeed[eeeeeefd\eed]eed^eed.eeeefd`edaeedbeedcedded$e jf:dzd{Z'Gd|d}d}e jZ(Gd~dde jZ)Gddde jZ*Gddde jZ+dS))CallableListTupleUnionN)set_attributes)create_res_basic_headcreate_res_roi_pooling_head)DetectionBBoxNetworkNet)create_acoustic_res_basic_stemcreate_res_basic_stemr)rr)rrrrrrrrr)rrrrrrrgh㈵>g?)conv_a_kernel_size conv_a_strideconv_a_paddingconv_aconv_b_kernel_size conv_b_strideconv_b_paddingconv_b_num_groupsconv_b_dilationconv_bconv_cnormnorm_eps norm_momentum activationdim_in dim_innerdim_outrrrrrrrrrrrrr r!r"returnc Cs||||||dd}|durdn||||d}|durdn|}| ||||| d| | d} |dur2dn||||d}|dur?dn|}| ||ddd} |durPdn||||d}t|||| ||| |dS) u Bottleneck block: a sequence of spatiotemporal Convolution, Normalization, and Activations repeated in the following order: :: Conv3d (conv_a) ↓ Normalization (norm_a) ↓ Activation (act_a) ↓ Conv3d (conv_b) ↓ Normalization (norm_b) ↓ Activation (act_b) ↓ Conv3d (conv_c) ↓ Normalization (norm_c) Normalization examples include: BatchNorm3d and None (no normalization). Activation examples include: ReLU, Softmax, Sigmoid, and None (no activation). Args: dim_in (int): input channel size to the bottleneck block. dim_inner (int): intermediate channel size of the bottleneck. dim_out (int): output channel size of the bottleneck. conv_a_kernel_size (tuple): convolutional kernel size(s) for conv_a. conv_a_stride (tuple): convolutional stride size(s) for conv_a. conv_a_padding (tuple): convolutional padding(s) for conv_a. conv_a (callable): a callable that constructs the conv_a conv layer, examples include nn.Conv3d, OctaveConv, etc conv_b_kernel_size (tuple): convolutional kernel size(s) for conv_b. conv_b_stride (tuple): convolutional stride size(s) for conv_b. conv_b_padding (tuple): convolutional padding(s) for conv_b. conv_b_num_groups (int): number of groups for groupwise convolution for conv_b. conv_b_dilation (tuple): dilation for 3D convolution for conv_b. conv_b (callable): a callable that constructs the conv_b conv layer, examples include nn.Conv3d, OctaveConv, etc conv_c (callable): a callable that constructs the conv_c conv layer, examples include nn.Conv3d, OctaveConv, etc norm (callable): a callable that constructs normalization layer, examples include nn.BatchNorm3d, None (not performing normalization). norm_eps (float): normalization epsilon. norm_momentum (float): normalization momentum. activation (callable): a callable that constructs activation layer, examples include: nn.ReLU, nn.Softmax, nn.Sigmoid, and None (not performing activation). Returns: (nn.Module): resnet bottleneck block. F in_channels out_channels kernel_sizestridepaddingbiasN num_featuresepsmomentumr(r)r*r+r,r-groupsdilationrr(r)r*r-rnorm_aact_arnorm_bact_brnorm_c)BottleneckBlock)r#r$r%rrrrrrrrrrrrr r!r"r7r8r9r:r;r=N/home/ubuntu/.local/lib/python3.10/site-packages/pytorchvideo/models/resnet.pycreate_bottleneck_blocksZQ     r?rrrc% Cs||||||dd}|durdn||||d}|durdn|}|dddg}|}| dddg}d|d|dg}|}d| d| dg}| fd\}}| dddg}d| d| dg}| |||||d||d}|durmdn||||d}|durzdn|} | |||||d||d}!|durdn||||d}"|durdn|}#| ||d dd } |durdn||||d}$t|||t|!|gt|"|gt|#| g| |$d S) uP Acoustic Bottleneck block: a sequence of spatiotemporal Convolution, Normalization, and Activations repeated in the following order: :: Conv3d (conv_a) ↓ Normalization (norm_a) ↓ Activation (act_a) ↓ --------------------------------- ↓ ↓ Temporal Conv3d (conv_b) Spatial Conv3d (conv_b) ↓ ↓ Normalization (norm_b) Normalization (norm_b) ↓ ↓ Activation (act_b) Activation (act_b) ↓ ↓ --------------------------------- ↓ Conv3d (conv_c) ↓ Normalization (norm_c) Normalization examples include: BatchNorm3d and None (no normalization). Activation examples include: ReLU, Softmax, Sigmoid, and None (no activation). Args: dim_in (int): input channel size to the bottleneck block. dim_inner (int): intermediate channel size of the bottleneck. dim_out (int): output channel size of the bottleneck. conv_a_kernel_size (tuple): convolutional kernel size(s) for conv_a. conv_a_stride (tuple): convolutional stride size(s) for conv_a. conv_a_padding (tuple): convolutional padding(s) for conv_a. conv_a (callable): a callable that constructs the conv_a conv layer, examples include nn.Conv3d, OctaveConv, etc conv_b_kernel_size (tuple): convolutional kernel size(s) for conv_b. conv_b_stride (tuple): convolutional stride size(s) for conv_b. conv_b_padding (tuple): convolutional padding(s) for conv_b. conv_b_num_groups (int): number of groups for groupwise convolution for conv_b. conv_b_dilation (tuple): dilation for 3D convolution for conv_b. conv_b (callable): a callable that constructs the conv_b conv layer, examples include nn.Conv3d, OctaveConv, etc conv_c (callable): a callable that constructs the conv_c conv layer, examples include nn.Conv3d, OctaveConv, etc norm (callable): a callable that constructs normalization layer, examples include nn.BatchNorm3d, None (not performing normalization). norm_eps (float): normalization epsilon. norm_momentum (float): normalization momentum. activation (callable): a callable that constructs activation layer, examples include: nn.ReLU, nn.Softmax, nn.Sigmoid, and None (not performing activation). Returns: (nn.Module): resnet acoustic bottleneck block. Fr'Nr.rrrr2rr5r6)SeparableBottleneckBlocknn ModuleList)%r#r$r%rrrrrrrrrrrrr r!r"r7r8conv_b_1_kernel_sizeconv_b_1_strideconv_b_1_paddingconv_b_2_kernel_sizeconv_b_2_strideconv_b_2_paddingconv_b_1_num_groupsconv_b_2_num_groupsconv_b_1_dilationconv_b_2_dilationconv_b_1norm_b_1act_b_1conv_b_2norm_b_2act_b_2r;r=r=r> create_acoustic_bottleneck_blocksV          rTcCs||S)zH Utility function used in lieu of lamda which are not picklable r=)xyr=r=r> _trivial_sum?srWF) use_shortcut branch_fusionrrrrrrrrrrr conv_skiprr r!activation_bottleneckactivation_block bottleneckrXrYrZr[r\cCstttjt|| }d}|s|dur%||kst|dkr%||||d}t||ks3t|dks3|r<|||d|ddnd||did|d|d |d |d |d |d | d| d| d| d| d|d|d|d|d|d|d||durd|dS||dS)u Residual block. Performs a summation between an identity shortcut in branch1 and a main block in branch2. When the input and output dimensions are different, a convolution followed by a normalization will be performed. :: Input |-------+ ↓ | Block | ↓ | Summation ←-+ ↓ Activation Normalization examples include: BatchNorm3d and None (no normalization). Activation examples include: ReLU, Softmax, Sigmoid, and None (no activation). Transform examples include: BottleneckBlock. Args: dim_in (int): input channel size to the bottleneck block. dim_inner (int): intermediate channel size of the bottleneck. dim_out (int): output channel size of the bottleneck. bottleneck (callable): a callable that constructs bottleneck block layer. Examples include: create_bottleneck_block. use_shortcut (bool): If true, use conv and norm layers in skip connection. branch_fusion (callable): a callable that constructs summation layer. Examples include: lambda x, y: x + y, OctaveSum. conv_a_kernel_size (tuple): convolutional kernel size(s) for conv_a. conv_a_stride (tuple): convolutional stride size(s) for conv_a. conv_a_padding (tuple): convolutional padding(s) for conv_a. conv_a (callable): a callable that constructs the conv_a conv layer, examples include nn.Conv3d, OctaveConv, etc conv_b_kernel_size (tuple): convolutional kernel size(s) for conv_b. conv_b_stride (tuple): convolutional stride size(s) for conv_b. conv_b_padding (tuple): convolutional padding(s) for conv_b. conv_b_num_groups (int): number of groups for groupwise convolution for conv_b. conv_b_dilation (tuple): dilation for 3D convolution for conv_b. conv_b (callable): a callable that constructs the conv_b conv layer, examples include nn.Conv3d, OctaveConv, etc conv_c (callable): a callable that constructs the conv_c conv layer, examples include nn.Conv3d, OctaveConv, etc conv_skip (callable): a callable that constructs the conv_skip conv layer, examples include nn.Conv3d, OctaveConv, etc norm (callable): a callable that constructs normalization layer. Examples include nn.BatchNorm3d, None (not performing normalization). norm_eps (float): normalization epsilon. norm_momentum (float): normalization momentum. activation_bottleneck (callable): a callable that constructs activation layer in bottleneck. Examples include: nn.ReLU, nn.Softmax, nn.Sigmoid, and None (not performing activation). activation_block (callable): a callable that constructs activation layer used at the end of the block. Examples include: nn.ReLU, nn.Softmax, nn.Sigmoid, and None (not performing activation). Returns: (nn.Module): resnet basic block layer. Nrr.rF)r*r+r-r#r$r%rrrrrrrrrrrrr r!r") branch1_conv branch1_normbranch2r"rYr=)tuplemapnpprodzipResBlock)r#r$r%r]rXrYrrrrrrrrrrrrZrr r!r[r\branch1_conv_stride norm_modelr=r=r>create_res_blockFsv^      ridepthcCs"g}t|dtr |g}t|dtr|g}||d|}||d|}t|D]^}tdid|dkr6|n|d|d|d|d||d|dkrM|nd d ||d |d | d |dkra| nd d| d| d| d|d|d|d|d|d|d|}||q*tt|dS)u Create Residual Stage, which composes sequential blocks that make up a ResNet. These blocks could be, for example, Residual blocks, Non-Local layers, or Squeeze-Excitation layers. :: Input ↓ ResBlock ↓ . . . ↓ ResBlock Normalization examples include: BatchNorm3d and None (no normalization). Activation examples include: ReLU, Softmax, Sigmoid, and None (no activation). Bottleneck examples include: create_bottleneck_block. Args: depth (init): number of blocks to create. dim_in (int): input channel size to the bottleneck block. dim_inner (int): intermediate channel size of the bottleneck. dim_out (int): output channel size of the bottleneck. bottleneck (callable): a callable that constructs bottleneck block layer. Examples include: create_bottleneck_block. conv_a_kernel_size (tuple or list of tuple): convolutional kernel size(s) for conv_a. If conv_a_kernel_size is a tuple, use it for all blocks in the stage. If conv_a_kernel_size is a list of tuple, the kernel sizes will be repeated until having same length of depth in the stage. For example, for conv_a_kernel_size = [(3, 1, 1), (1, 1, 1)], the kernel size for the first 6 blocks would be [(3, 1, 1), (1, 1, 1), (3, 1, 1), (1, 1, 1), (3, 1, 1)]. conv_a_stride (tuple): convolutional stride size(s) for conv_a. conv_a_padding (tuple or list of tuple): convolutional padding(s) for conv_a. If conv_a_padding is a tuple, use it for all blocks in the stage. If conv_a_padding is a list of tuple, the padding sizes will be repeated until having same length of depth in the stage. conv_a (callable): a callable that constructs the conv_a conv layer, examples include nn.Conv3d, OctaveConv, etc conv_b_kernel_size (tuple): convolutional kernel size(s) for conv_b. conv_b_stride (tuple): convolutional stride size(s) for conv_b. conv_b_padding (tuple): convolutional padding(s) for conv_b. conv_b_num_groups (int): number of groups for groupwise convolution for conv_b. conv_b_dilation (tuple): dilation for 3D convolution for conv_b. conv_b (callable): a callable that constructs the conv_b conv layer, examples include nn.Conv3d, OctaveConv, etc conv_c (callable): a callable that constructs the conv_c conv layer, examples include nn.Conv3d, OctaveConv, etc norm (callable): a callable that constructs normalization layer. Examples include nn.BatchNorm3d, and None (not performing normalization). norm_eps (float): normalization epsilon. norm_momentum (float): normalization momentum. activation (callable): a callable that constructs activation layer. Examples include: nn.ReLU, nn.Softmax, nn.Sigmoid, and None (not performing activation). Returns: (nn.Module): resnet basic stage layer. rNr#r$r%r]rrrrrrrrrrrrrr r!r[r\) res_blocksr=) isinstanceintrangeriappendResStagerBrC)rjr#r$r%r]rrrrrrrrrrrrr r!r"rkindblockr=r=r>create_res_stagesl`       rs)rr)rrtr)r$r)2erryig?@)rr})rrr r )rrrr)rrrr)rrrr)rrrr)rtr}r}T) input_channel model_depthmodel_num_class dropout_raterr" stem_dim_outstem_conv_kernel_sizestem_conv_stride stem_poolstem_pool_kernel_sizestem_pool_stridestem stage1_poolstage1_pool_kernel_sizestage_conv_a_kernel_sizestage_conv_b_kernel_sizestage_conv_b_num_groupsstage_conv_b_dilationstage_spatial_h_stridestage_spatial_w_stridestage_temporal_strider]head head_poolhead_pool_kernel_sizehead_output_sizehead_activationhead_output_with_global_averager~rrrrrrrrrrrrrrrrrrrrrrrrrc)Csttjd|tvsJ|dtt|}t|dtr)|ft|}t|dtr7|ft|}t|dtrE|ft|}t|trQ|gt|}g}| ||||dd|D| | | dd| D||d } | | |}|d} t t|D]}!| d}"||!}#||!}$||!d d f}%t|$dtrd d|$Dnd d|$D}&d ||!||!f}'t |#||"| ||!|$|%|&||!|'||!dd ||!d d kr||!d n||!d d ||!d d kr||!d n||!d d f||!||!||d }(| |(| }| d } |!dkr| dur| | ||ddq}|dur2|||||||||d}| |t t |dS)uU Build ResNet style models for video recognition. ResNet has three parts: Stem, Stages and Head. Stem is the first Convolution layer (Conv1) with an optional pooling layer. Stages are grouped residual blocks. There are usually multiple stages and each stage may include multiple residual blocks. Head may include pooling, dropout, a fully-connected layer and global spatial temporal averaging. The three parts are assembled in the following order: :: Input ↓ Stem ↓ Stage 1 ↓ . . . ↓ Stage N ↓ Head Args: input_channel (int): number of channels for the input video clip. model_depth (int): the depth of the resnet. Options include: 50, 101, 152. model_num_class (int): the number of classes for the video dataset. dropout_rate (float): dropout rate. norm (callable): a callable that constructs normalization layer. activation (callable): a callable that constructs activation layer. stem_dim_out (int): output channel size to stem. stem_conv_kernel_size (tuple): convolutional kernel size(s) of stem. stem_conv_stride (tuple): convolutional stride size(s) of stem. stem_pool (callable): a callable that constructs resnet head pooling layer. stem_pool_kernel_size (tuple): pooling kernel size(s). stem_pool_stride (tuple): pooling stride size(s). stem (callable): a callable that constructs stem layer. Examples include: create_res_video_stem. stage_conv_a_kernel_size (tuple): convolutional kernel size(s) for conv_a. stage_conv_b_kernel_size (tuple): convolutional kernel size(s) for conv_b. stage_conv_b_num_groups (tuple): number of groups for groupwise convolution for conv_b. 1 for ResNet, and larger than 1 for ResNeXt. stage_conv_b_dilation (tuple): dilation for 3D convolution for conv_b. stage_spatial_h_stride (tuple): the spatial height stride for each stage. stage_spatial_w_stride (tuple): the spatial width stride for each stage. stage_temporal_stride (tuple): the temporal stride for each stage. bottleneck (callable): a callable that constructs bottleneck block layer. Examples include: create_bottleneck_block. head (callable): a callable that constructs the resnet-style head. Ex: create_res_basic_head head_pool (callable): a callable that constructs resnet head pooling layer. head_pool_kernel_size (tuple): the pooling kernel size. head_output_size (tuple): the size of output tensor for head. head_activation (callable): a callable that constructs activation layer. head_output_with_global_average (bool): if True, perform global averaging on the head output. Returns: (nn.Module): basic resnet. z PYTORCHVIDEO.model.create_resnetz is not in rcSg|]}|dqSrr=.0sizer=r=r> z!create_resnet..cSrrr=rr=r=r>rr) r(r)conv_kernel_size conv_stride conv_paddingpoolpool_kernel_size pool_stride pool_paddingrr"rtrcSrrr=rr=r=r>rrcSsg|] }dd|DqS)cSrrr=rr=r=r>rrz,create_resnet...r=)rsizesr=r=r>rsr)rjr#r$r%r]rrrrrrrrrr"Nr@)r*r+r,) in_features out_featuresr output_sizerrr"output_with_global_average)blocks)torch_C_log_api_usage_once_MODEL_STAGE_DEPTHkeysrlrmlenrrornrsr rBrC))r~rrrrr"rrrrrrrrrrrrrrrrr]rrrrrr stage_depthsr stage_dim_in stage_dim_outidxstage_dim_innerrjstage_conv_a_kernelstage_conv_a_stridestage_conv_a_paddingstage_conv_b_stridestager=r=r> create_resnetWs |            rP)rr}r})rrrr)rrrr)rtrr)r}r}g?) r~rrrrr"rrrrrrrrrrrrrrrrr]rrrrrrhead_spatial_resolutionhead_spatial_scalehead_sampling_ratiorrrc ! Cstdid|d|d|d|d|d|d|d|d |d | d | d | d | d|d|d|d|d|d|d|d|dd} ||dtt|d||||||||||d }t| |S)u0 Build ResNet style models for video detection. ResNet has three parts: Stem, Stages and Head. Stem is the first Convolution layer (Conv1) with an optional pooling layer. Stages are grouped residual blocks. There are usually multiple stages and each stage may include multiple residual blocks. Head may include pooling, dropout, a fully-connected layer and global spatial temporal averaging. The three parts are assembled in the following order: :: Input Clip Input Bounding Boxes ↓ ↓ Stem ↓ ↓ ↓ Stage 1 ↓ ↓ ↓ . ↓ . ↓ . ↓ ↓ ↓ Stage N ↓ ↓--------> Head <-------↓ Args: input_channel (int): number of channels for the input video clip. model_depth (int): the depth of the resnet. Options include: 50, 101, 152. model_num_class (int): the number of classes for the video dataset. dropout_rate (float): dropout rate. norm (callable): a callable that constructs normalization layer. activation (callable): a callable that constructs activation layer. stem_dim_out (int): output channel size to stem. stem_conv_kernel_size (tuple): convolutional kernel size(s) of stem. stem_conv_stride (tuple): convolutional stride size(s) of stem. stem_pool (callable): a callable that constructs resnet head pooling layer. stem_pool_kernel_size (tuple): pooling kernel size(s). stem_pool_stride (tuple): pooling stride size(s). stem (callable): a callable that constructs stem layer. Examples include: create_res_video_stem. stage_conv_a_kernel_size (tuple): convolutional kernel size(s) for conv_a. stage_conv_b_kernel_size (tuple): convolutional kernel size(s) for conv_b. stage_conv_b_num_groups (tuple): number of groups for groupwise convolution for conv_b. 1 for ResNet, and larger than 1 for ResNeXt. stage_conv_b_dilation (tuple): dilation for 3D convolution for conv_b. stage_spatial_h_stride (tuple): the spatial height stride for each stage. stage_spatial_w_stride (tuple): the spatial width stride for each stage. stage_temporal_stride (tuple): the temporal stride for each stage. bottleneck (callable): a callable that constructs bottleneck block layer. Examples include: create_bottleneck_block. head (callable): a callable that constructs the detection head which can take in the additional input of bounding boxes. Ex: create_res_roi_pooling_head head_pool (callable): a callable that constructs resnet head pooling layer. head_pool_kernel_size (tuple): the pooling kernel size. head_output_size (tuple): the size of output tensor for head. head_activation (callable): a callable that constructs activation layer. head_output_with_global_average (bool): if True, perform global averaging on the head output. head_spatial_resolution (tuple): h, w sizes of the RoI interpolation. head_spatial_scale (float): scale the input boxes by this number. head_sampling_ratio (int): number of inputs samples to take for each output sample interpolation. 0 to take samples densely. Returns: (nn.Module): basic resnet. r~rrrrr"rrrrrrrrrrrrrrr]rNrr) rrrrrrr"r resolution spatial_scalesampling_ratior=)rrrr )!r~rrrrr"rrrrrrrrrrrrrrrrr]rrrrrrrrrmodelr=r=r>create_resnet_with_roi_headFsx    r) rr)rrr)rrr)rrr)rtrrr~rrrrr"rrrrrrrrrrrrrrrrr]rrrrrcCstditS)u Build ResNet style models for acoustic recognition. ResNet has three parts: Stem, Stages and Head. Stem is the first Convolution layer (Conv1) with an optional pooling layer. Stages are grouped residual blocks. There are usually multiple stages and each stage may include multiple residual blocks. Head may include pooling, dropout, a fully-connected layer and global spatial temporal averaging. The three parts are assembled in the following order: :: Input ↓ Stem ↓ Stage 1 ↓ . . . ↓ Stage N ↓ Head Args: input_channel (int): number of channels for the input video clip. model_depth (int): the depth of the resnet. Options include: 50, 101, 152. model_num_class (int): the number of classes for the video dataset. dropout_rate (float): dropout rate. norm (callable): a callable that constructs normalization layer. activation (callable): a callable that constructs activation layer. stem_dim_out (int): output channel size to stem. stem_conv_kernel_size (tuple): convolutional kernel size(s) of stem. stem_conv_stride (tuple): convolutional stride size(s) of stem. stem_pool (callable): a callable that constructs resnet head pooling layer. stem_pool_kernel_size (tuple): pooling kernel size(s). stem_pool_stride (tuple): pooling stride size(s). stem (callable): a callable that constructs stem layer. Examples include: create_res_video_stem. stage_conv_a_kernel_size (tuple): convolutional kernel size(s) for conv_a. stage_conv_b_kernel_size (tuple): convolutional kernel size(s) for conv_b. stage_conv_b_num_groups (tuple): number of groups for groupwise convolution for conv_b. 1 for ResNet, and larger than 1 for ResNeXt. stage_conv_b_dilation (tuple): dilation for 3D convolution for conv_b. stage_spatial_h_stride (tuple): the spatial height stride for each stage. stage_spatial_w_stride (tuple): the spatial width stride for each stage. stage_temporal_stride (tuple): the temporal stride for each stage. bottleneck (callable): a callable that constructs bottleneck block layer. Examples include: create_bottleneck_block. head_pool (callable): a callable that constructs resnet head pooling layer. head_pool_kernel_size (tuple): the pooling kernel size. head_output_size (tuple): the size of output tensor for head. head_activation (callable): a callable that constructs activation layer. head_output_with_global_average (bool): if True, perform global averaging on the head output. Returns: (nn.Module): audio resnet, that takes spectragram image input with shape: (B, C, T, 1, F), where T is the time dimension and F is the frequency dimension. Nr=)rlocalsrr=r=r>create_acoustic_resnetsprc s`eZdZdZ     d dejdejdejdejdedejf fd d Zdej fd d Z Z S)rfu Residual block. Performs a summation between an identity shortcut in branch1 and a main block in branch2. When the input and output dimensions are different, a convolution followed by a normalization will be performed. :: Input |-------+ ↓ | Block | ↓ | Summation ←-+ ↓ Activation The builder can be found in `create_res_block`. Nr^r_r`r"rYr&cs(tt|t|jdusJdS)a Args: branch1_conv (torch.nn.modules): convolutional module in branch1. branch1_norm (torch.nn.modules): normalization module in branch1. branch2 (torch.nn.modules): bottleneck block module in branch2. activation (torch.nn.modules): activation module. branch_fusion: (Callable): A callable or layer that combines branch1 and branch2. N)super__init__rrr`)selfr^r_r`r"rY __class__r=r>rs  zResBlock.__init__cCsf|jdur||||}n||}|jdur||}||||}|jdur1||}|SN)r^rYr`r_r")rrUshortcutr=r=r>forwards      zResBlock.forward)NNNNN) __name__ __module__ __qualname____doc__rBModulerrrTensorr __classcell__r=r=rr>rfks*rfcsveZdZdZdddejdejdejdejdejd ejd ejd ejd ed dffddZde j d e j fddZ Z S)rAu Separable Bottleneck block: a sequence of spatiotemporal Convolution, Normalization, and Activations repeated in the following order. Requires a tuple of models to be provided to conv_b, norm_b, act_b to perform Convolution, Normalization, and Activations in parallel Separably. :: Conv3d (conv_a) ↓ Normalization (norm_a) ↓ Activation (act_a) ↓ Conv3d(s) (conv_b), ... ↓ (↓) Normalization(s) (norm_b), ... ↓ (↓) Activation(s) (act_b), ... ↓ (↓) Reduce (sum or cat) ↓ Conv3d (conv_c) ↓ Normalization (norm_c) sum) reduce_methodrr7r8rr9r:rr;rr&Nc svtt|ttdd|j|jfDs(J|jd|jd|jd| dvs.J|jdur9d|j_ dSdS)a Args: conv_a (torch.nn.modules): convolutional module. norm_a (torch.nn.modules): normalization module. act_a (torch.nn.modules): activation module. conv_b (torch.nn.modules_list): convolutional module(s). norm_b (torch.nn.modules_list): normalization module(s). act_b (torch.nn.modules_list): activation module(s). conv_c (torch.nn.modules): convolutional module. norm_c (torch.nn.modules): normalization module. reduce_method (str): if multiple conv_b is used, reduce the output with `sum`, or `cat`. cs|]}|duVqdSrr=ropr=r=r> s z4SeparableBottleneckBlock.__init__..z, z has None)rcatNT) rrrrallrrrr;block_final_bn) rrr7r8rr9r:rr;rrr=r>rs       z!SeparableBottleneckBlock.__init__rUcCs|jdur ||}|jdur||}|jdur||}g}tt|jD]*}|j||}|j|dur>|j||}|j|durL|j||}||q'|j dkrdt j |ddj ddd}n |j dkrpt j |dd}||}|jdur||}|S)Nrr)dimF)rkeepdimrr)rr7r8rnrrr9r:rorrstackrrrr;)rrUoutputrqx_r=r=r>rs,            z SeparableBottleneckBlock.forward) rrrrrBrrCstrrrrrrr=r=rr>rAs2'    $rAcseZdZdZddddddddddejdejdejdejdejd ejd ejd ejd dffd dZdejd ejfddZ Z S)r<u Bottleneck block: a sequence of spatiotemporal Convolution, Normalization, and Activations repeated in the following order: :: Conv3d (conv_a) ↓ Normalization (norm_a) ↓ Activation (act_a) ↓ Conv3d (conv_b) ↓ Normalization (norm_b) ↓ Activation (act_b) ↓ Conv3d (conv_c) ↓ Normalization (norm_c) The builder can be found in `create_bottleneck_block`. Nr6rr7r8rr9r:rr;r&c sRtt|ttdd|j|j|jfDsJ|jdur'd|j_ dSdS)a Args: conv_a (torch.nn.modules): convolutional module. norm_a (torch.nn.modules): normalization module. act_a (torch.nn.modules): activation module. conv_b (torch.nn.modules): convolutional module. norm_b (torch.nn.modules): normalization module. act_b (torch.nn.modules): activation module. conv_c (torch.nn.modules): convolutional module. norm_c (torch.nn.modules): normalization module. csrrr=rr=r=r>r6sz+BottleneckBlock.__init__..NT) rrrrrrrrr;r) rrr7r8rr9r:rr;rr=r>rs  "  zBottleneckBlock.__init__rUcCs||}|jdur||}|jdur||}||}|jdur(||}|jdur2||}||}|jdurA||}|Srr6)rrUr=r=r>r;s             zBottleneckBlock.forward) rrrrrBrrrrrrr=r=rr>r<s<   r<csDeZdZdZdejdejffdd Zdej dej fddZ Z S) rpu ResStage composes sequential blocks that make up a ResNet. These blocks could be, for example, Residual blocks, Non-Local layers, or Squeeze-Excitation layers. :: Input ↓ ResBlock ↓ . . . ↓ ResBlock The builder can be found in `create_res_stage`. rkr&cst||_dS)zZ Args: res_blocks (torch.nn.module_list): ResBlock module(s). N)rrrk)rrkrr=r>rgs  zResStage.__init__rUcCs t|jD]\}}||}q|Sr) enumeraterk)rrU_ res_blockr=r=r>ros zResStage.forward) rrrrrBrCrrrrrrr=r=rr>rpRsrp),typingrrrrnumpyrcrtorch.nnrBpytorchvideo.layers.utilsrpytorchvideo.models.headrrpytorchvideo.models.netr r pytorchvideo.models.stemr r Conv3d BatchNorm3dReLUrmfloatrr?rTrWboolrirsr MaxPool3d AvgPool3drSigmoidrrrfrAr<rpr=r=r=r>s,               )              #$*+,-/012345 s   #$*+,-/012345678 6   %&'()* s7`P