o پiB@sdZddlmZddlmZddlmZmZmZm Z m Z m Z m Z ddl Z ddlmZddlmZmZddlmZmZmZmZmZmZmZmZmZmZmZmZm Z dd l!m"Z"dd l#m$Z$dd l%m&Z&m'Z'm(Z(dd l)m*Z*m+Z+m,Z,d gZ-Gdddej.Z/Gdddej.Z0Gdddej.Z1Gdddej.Z2Gdddej.Z3Gdddej.Z4de5de6fddZ7ddd eeed!d"fd#e8d$e8de5d%e6d&ed'edej9fd(d)Z:Gd*d d ej.Z;dd+ej.d,e5d-e6ddfd.d/Zdd7e5d8e6d9ede;fd:d;Z?dd7e5d8e6d9ede;fde5d9edee5effd?d@ZAe*idAeAdBdCd dDdEeAdBdCd dDdFeAdBdGdHdIdCd dJdKeAdBdLdMdId dNdOeAdBdLdMdId dNdPeAdBdLdMdId dNdQeAdBdLdMdId dNdReAdBdLdMdId dNdSeAdBdTdUdId dNdVeAdBdWd dXdYeAdBdWd dXdZeAdBdWd dXd[eAdBdWd dXd\eAdBdWd dXd]eAdBdWd dXd^eAdBdCd_d`dIdadbeAdBdCd_d`dIdcddeAdBdCd_d`dIdaeAdBdCd_d`dIdcddeAdBdIdGdHdLdCdcdeeAdBdCdfd`dIdaeAdCdcdgeAdCdcdgeAdBdCdfd`dIdaeAdCdcdgeAdCdheAdCdcdgeAdBdCdcdfd`dIdieAdBdCdcdfd`dIdieAdCdcdgdj ZBe+dd8e6d9ede;fdkdlZCe+dd8e6d9ede;fdmdnZDe+dd8e6d9ede;fdodpZEe+dd8e6d9ede;fdqdrZFe+dd8e6d9ede;fdsdtZGe+dd8e6d9ede;fdudvZHe+dd8e6d9ede;fdwdxZIe+dd8e6d9ede;fdydzZJe+dd8e6d9ede;fd{d|ZKe+dd8e6d9ede;fd}d~ZLe+dd8e6d9ede;fddZMe+dd8e6d9ede;fddZNe+dd8e6d9ede;fddZOe+dd8e6d9ede;fddZPe+dd8e6d9ede;fddZQe+dd8e6d9ede;fddZRe+dd8e6d9ede;fddZSe+dd8e6d9ede;fddZTe+dd8e6d9ede;fddZUe+dd8e6d9ede;fddZVe,eWdKdOdPdQdRdSdVdYdZd[d\d]dAdEdFddS)a/Pre-Activation ResNet v2 with GroupNorm and Weight Standardization. A PyTorch implementation of ResNetV2 adapted from the Google Big-Transfer (BiT) source code at https://github.com/google-research/big_transfer to match timm interfaces. The BiT weights have been included here as pretrained models from their original .NPZ checkpoints. Additionally, supports non pre-activation bottleneck for use as a backbone for Vision Transformers (ViT) and extra padding support to allow porting of official Hybrid ResNet pretrained weights from https://github.com/google-research/vision_transformer Thanks to the Google team for the above two repositories and associated papers: * Big Transfer (BiT): General Visual Representation Learning - https://arxiv.org/abs/1912.11370 * An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale - https://arxiv.org/abs/2010.11929 * Knowledge distillation: A good teacher is patient and consistent - https://arxiv.org/abs/2106.05237 Original copyright of Google code below, modifications by Ross Wightman, Copyright 2020. ) OrderedDict)partial)AnyCallableDictListOptionalTupleUnionNIMAGENET_INCEPTION_MEANIMAGENET_INCEPTION_STD) GroupNormActBatchNormAct2d EvoNorm2dS0FilterResponseNormTlu2dClassifierHeadDropPath AvgPool2dSame create_pool2d StdConv2d create_conv2d get_act_layerget_norm_act_layermake_divisible)build_model_with_cfg)feature_take_indices)checkpoint_seq named_applyadapt_input_conv)generate_default_cfgsregister_modelregister_model_deprecationsResNetV2ceZdZdZ           ddedeeded ed ed eed ed eedeedeedeedeffdd ZdddZ de j de j fddZ Z S) PreActBasiczAPre-activation basic block (not in typical 'v2' implementations).N?rin_chsout_chs bottle_ratiostridedilationfirst_dilationgroups act_layer conv_layer norm_layer proj_layerdrop_path_ratec  st|p|}| p t} | pttdd} |p|}t||} | dur=|dks/||ks/||kr=| |||||d| | d|_nd|_| ||_| || d|||d|_| | |_ | | |d||d |_ | d krjt | |_dSt |_dS) awInitialize PreActBasic block. Args: in_chs: Input channels. out_chs: Output channels. bottle_ratio: Bottleneck ratio (not used in basic block). stride: Stride for convolution. dilation: Dilation rate. first_dilation: First dilation rate. groups: Group convolution size. act_layer: Activation layer type. conv_layer: Convolution layer type. norm_layer: Normalization layer type. proj_layer: Projection/downsampling layer type. drop_path_rate: Stochastic depth drop rate.  num_groupsNrTr,r-r.preactr1r2r,r-r/)r-r/r)super__init__rrrr downsamplenorm1conv1norm2conv2rnnIdentity drop_pathselfr)r*r+r,r-r.r/r0r1r2r3r4mid_chs __class__H/home/ubuntu/.local/lib/python3.10/site-packages/timm/models/resnetv2.pyr=5s.      $zPreActBasic.__init__returncCtj|jjdS)zLZero-initialize the last convolution weight (not applicable to basic block).N)rCinitzeros_rBweightrGrKrKrLzero_init_lastozPreActBasic.zero_init_lastxcCsN||}|}|jdur||}||}|||}||}||SzoForward pass. Args: x: Input tensor. Returns: Output tensor. N)r?r>r@rBrArErGrUx_preactshortcutrKrKrLforwardss     zPreActBasic.forward) Nr'rrNrNNNNr(rMN__name__ __module__ __qualname____doc__intrfloatrr=rStorchTensorrZ __classcell__rKrKrIrLr&2sP      :r&cr%)PreActBottleneckaPre-activation (v2) bottleneck block. Follows the implementation of "Identity Mappings in Deep Residual Networks": https://github.com/KaimingHe/resnet-1k-layers/blob/master/resnet-pre-act.lua Except it puts the stride on 3x3 conv when available. N?rr(r)r*r+r,r-r.r/r0r1r2r3r4c  st|p|}| p t} | pttdd} |p|}t||} | dur1| |||||d| | d|_nd|_| ||_| || d|_| | |_ | | | d|||d|_ | | |_ | | |d|_ | d krgt | |_dSt|_dS) abInitialize PreActBottleneck block. Args: in_chs: Input channels. out_chs: Output channels. bottle_ratio: Bottleneck ratio. stride: Stride for convolution. dilation: Dilation rate. first_dilation: First dilation rate. groups: Group convolution size. act_layer: Activation layer type. conv_layer: Convolution layer type. norm_layer: Normalization layer type. proj_layer: Projection/downsampling layer type. drop_path_rate: Stochastic depth drop rate. r5r6NTr8rr:r;r)r<r=rrrrr>r?r@rArBnorm3conv3rrCrDrErFrIrKrLr=s2      $zPreActBottleneck.__init__rMcCrN)z,Zero-initialize the last convolution weight.N)rCrOrPrirQrRrKrKrLrSrTzPreActBottleneck.zero_init_lastrUcCs^||}|}|jdur||}||}|||}|||}||}||SrV)r?r>r@rBrArirhrErWrKrKrLrZs     zPreActBottleneck.forward NrgrrNrNNNNr(r[r\rKrKrIrLrfsP       <rfcr%) BottleneckzUNon Pre-activation bottleneck block, equiv to V1.5/V1b Bottleneck. Used for ViT. Nrgrr(r)r*r+r,r-r.r/r0r1r2r3r4c  st|p|}|p tj}| pt} | pttdd} |p|}t||} | dur5| ||||d| | d|_nd|_| || d|_ | | |_ | | | d|||d|_ | | |_ | | |d|_ | |dd|_| d krjt| nt|_|d d |_dS) Nr5r6F)r,r-r9r1r2rr:r; apply_actrT)inplace)r<r=rCReLUrrrrr>r@r?rBrArirhrrDrEact3rFrIrKrLr=s4       zBottleneck.__init__rMcCs*t|jdddurtj|jjdSdS)z+Zero-initialize the last batch norm weight.rQN)getattrrhrCrOrPrQrRrKrKrLrSszBottleneck.zero_init_lastrUcCsp|}|jdur ||}||}||}||}||}||}||}||}|||}|SrV) r>r@r?rBrArirhrErp)rGrUrYrKrKrLrZ s         zBottleneck.forwardrjr[r\rKrKrIrLrksP      -rkcreZdZdZ      ddedededed eed ed eed eeffd d Zde j de j fddZ Z S)DownsampleConvz$1x1 convolution downsampling module.rNTr)r*r,r-r.r9r1r2c sDtt||||d|d|_|rt|_dS||dd|_dS)Nrr,Frl)r<rsr=convrCrDnorm) rGr)r*r,r-r.r9r1r2rIrKrLr==s $zDownsampleConv.__init__rUrMcCs|||SztForward pass. Args: x: Input tensor. Returns: Downsampled tensor. )rvrurGrUrKrKrLrZLs zDownsampleConv.forwardrrNTNN r]r^r_r`rarboolrr=rcrdrZrerKrKrIrLrs:s4 rscrr) DownsampleAvgz/AvgPool downsampling as in 'D' ResNet variants.rNTr)r*r,r-r.r9r1r2c stt||dkr |nd} |dks|dkr.| dkr!|dkr!tntj} | d| ddd|_nt|_|||ddd|_|rEt|_ dS||dd|_ dS)NrTF) ceil_modecount_include_padrtrl) r<r|r=rrC AvgPool2dpoolrDrurv) rGr)r*r,r-r.r9r1r2 avg_stride avg_pool_fnrIrKrLr=[s  $zDownsampleAvg.__init__rUrMcCs||||Srw)rvrurrxrKrKrLrZp zDownsampleAvg.forwardryrzrKrKrIrLr|Xs4 r|cseZdZdZddddedddfdededed ed ed ed ed edee ede dee dee dee de ffdd Z de jde jfddZZS) ResNetStagez ResNet Stage.rgrFNr)r*r,r-depthr+r/avg_down block_dprblock_fnr0r1r2 block_kwargsc stt|d|_|dvrdnd}t| | | d}|rtnt}|}t|_ t |D]2}| r2| |nd}|dkr:|nd}|j t || ||f|||||||d|||}|}d}q*dS) NF)rr}rr})r0r1r2r(r)r,r-r+r/r.r3r4) r<rr=grad_checkpointingdictr|rsrC Sequentialblocksrange add_modulestr)rGr)r*r,r-rr+r/rrrr0r1r2rr. layer_kwargsr3prev_chs block_idxr4rIrKrLr=~s<      zResNetStage.__init__rUrMcCs.|jrtjst|j|}|S||}|S)zForward pass through all blocks in the stage. Args: x: Input tensor. Returns: Output tensor. )rrcjit is_scriptingrrrxrKrKrLrZs   zResNetStage.forward)r]r^r_r`rfrarbr{rrrrr=rcrdrZrerKrKrIrLr|sP     -r stem_typerMcstfdddDS)zCheck if stem type is deep (has multiple convolutions). Args: stem_type: Type of stem to check. Returns: True if stem is deep, False otherwise. csg|]}|vqSrKrK).0srrKrL z is_stem_deep..)deeptiered)anyrrKrrL is_stem_deeprr@Tr5r6r)r*r9r1r2cCsTt}|dvs Jt|rcd|vrd|d|df}n|d|df}|||dddd|d<||d|d <||d|d dd d|d <||d |d <||d |dd d|d <|sb|||d<n|||ddd|d<|su|||d<d|vrtd d|d<tjdddd|d<nd|vrtddddd|d<n tjddd d|d<t|S)N)rfixedsamer deep_fixed deep_samerrr:r}r) kernel_sizer,r@r?rrBrArirhrurvrr(pad)rr,paddingrrmax)rrrC ConstantPad2d MaxPool2drr)r)r*rr9r1r2stemstem_chsrKrKrLcreate_resnetv2_stems0    rc's0eZdZdZdddddddd d d d d ejeedd eddd fde e de e dfde de de de de de de de de de dedededed ed!ed"e f&fd#d$ ZejjdLd"e d%d&fd'd(ZejdMd*e d+e d%d&fd,d-ZejjdNd.e d%ee effd/d0ZejjdLd1e d%d&fd2d3Zejjd%ejfd4d5ZdOde dee d%d&fd6d7Z & 8 dPd9ejd:eee e e fd;e de d%ee eje eje ejfffd?d@Z  dQd:ee e e fdAe dBe fdCdDZ!d9ejd%ejfdEdFZ"dNd9ejdGe d%ejfdHdIZ#d9ejd%ejfdJdKZ$Z%S)Rr$z7Implementation of Pre-activation (v2) ResNet mode. )iir:avgr5rrrFTrgr6r(layerschannels. num_classesin_chans global_pool output_stride width_factorrrrr9basicr+r0r2r1 drop_rater4rSc"st||_||_|}t||d}t|}g|_t||}t||| | ||d|_ | r5t | r3dndnd}|j t |d|d|}d}d }d d t d |t||D}| rb| r_tnt}n| rfJt}t|_tt|||D]N\}\}}}t||}|d krd nd} ||kr|| 9}d } t||| ||| | |||||d }!|}|| 9}|jt ||d|dg7_|jt||!qu||_|_| r||jnt|_t |j|||jdd|_!|j"|ddS)a Args: layers (List[int]) : number of layers in each block channels (List[int]) : number of channels in each block: num_classes (int): number of classification classes (default 1000) in_chans (int): number of input (color) channels. (default 3) global_pool (str): Global pooling type. One of 'avg', 'max', 'avgmax', 'catavgmax' (default 'avg') output_stride (int): output stride of the network, 32, 16, or 8. (default 32) width_factor (int): channel (width) multiplication factor stem_chs (int): stem width (default: 64) stem_type (str): stem type (default: '' == 7x7) avg_down (bool): average pooling in residual downsampling (default: False) preact (bool): pre-activation (default: True) act_layer (Union[str, nn.Module]): activation layer norm_layer (Union[str, nn.Module]): normalization layer conv_layer (nn.Module): convolution module drop_rate: classifier dropout rate (default: 0.) drop_path_rate: stochastic depth rate (default: 0.) zero_init_last: zero-init last weight in residual path (default: False) )r0)r1r2z stem.conv3 stem.convz stem.normr})num_chs reductionmodulercSsg|]}|qSrK)tolist)rrUrKrKrLr9rz%ResNetV2.__init__..r) r,r-rr+rr0r1r2rrzstages.T) pool_typeruse_convrSN)#r<r=rrrr feature_inforrrrappendrrclinspacesumsplitr&rfrkrCrstages enumerateziprrr num_featureshead_hidden_sizerDrvrhead init_weights)"rGrrrrrrrrrrr9rr+r0r2r1rr4rSwf stem_featr curr_strider- block_dprsr stage_idxdcbdprr*r,stagerIrKrLr=sz *  "    zResNetV2.__init__rMNcCsttt|d|dS)zInitialize model weights.rN)rr _init_weights)rGrSrKrKrLreszResNetV2.init_weightsresnet/checkpoint_pathprefixcCst|||dS)zLoad pretrained weights.N) _load_weights)rGrrrKrKrLload_pretrainedjszResNetV2.load_pretrainedcoarsecCstd|rdnddgd}|S)z"Group parameters for optimization.z^stemz^stages\.(\d+))z^stages\.(\d+)\.blocks\.(\d+)N)z^norm)i)rr)r)rGrmatcherrKrKrL group_matcheroszResNetV2.group_matcherenablecCs|jD]}||_qdS)z)Enable or disable gradient checkpointing.N)rr)rGrrrKrKrLset_grad_checkpointing{s zResNetV2.set_grad_checkpointingcCs|jjS)zGet the classifier head.)rfcrRrKrKrLget_classifierszResNetV2.get_classifiercCs||_|j||dS)zReset the classifier head. Args: num_classes: Number of classes for new classifier. global_pool: Global pooling type. N)rrreset)rGrrrKrKrLreset_classifierszResNetV2.reset_classifierNCHWrUindicesrv stop_early output_fmtintermediates_onlycCs&|dvsJdg}td|\}} d} |jdd\} } |jD]} | |}|jdd| d| dfkr6|}q| |vr@||t|j}tjsL|sP|j}n|jd| }t |dd D]$\} }||}| |vr| |kr||rt| |n|}||q]||q]|r|S| |kr| |}||fS) a Forward features that returns intermediates. Args: x: Input image tensor indices: Take last n blocks if int, all if None, select matching indices if sequence norm: Apply norm layer to compatible intermediates stop_early: Stop iterating over blocks when last desired intermediate hit output_fmt: Shape of intermediate feature outputs intermediates_only: Only return intermediate features Returns: )rzOutput shape must be NCHW.rNr}r)start) rshaperrlenrrcrrrrv)rGrUrrvrrr intermediates take_indices max_indexfeat_idxHWrx_downlast_idxrrx_interrKrKrLforward_intermediatess:      zResNetV2.forward_intermediates prune_norm prune_headcCs@td|\}}|jd||_|rt|_|r|dd|S)z@ Prune layers not required for specified intermediates. rNrr)rrrCrDrvr)rGrrrrrrKrKrLprune_intermediate_layerss  z"ResNetV2.prune_intermediate_layerscCs"||}||}||}|S)zForward pass through feature extraction layers. Args: x: Input tensor. Returns: Feature tensor. )rrrvrxrKrKrLforward_featuress  zResNetV2.forward_features pre_logitscCs|r |j||dS||S)zForward pass through classifier head. Args: x: Input features. pre_logits: Return features before final linear layer. Returns: Classification logits or features. )r)r)rGrUrrKrKrL forward_heads zResNetV2.forward_headcCs||}||}|S)zoForward pass. Args: x: Input tensor. Returns: Output logits. )rrrxrKrKrLrZs zResNetV2.forward)TrF)N)NFFrF)rFT)&r]r^r_r`rCrorrrrrar rr{rbrr=rcrignorerrrrrrModulerrrrdr rrrrrZrerKrKrIrLr$s       m    ;  rnamerScCst|tjsd|vr#t|tjr#tjj|jdddtj|jdSt|tjrCtjj |jddd|jdurAtj|jdSdSt|tj tj tj fr^tj |jtj|jdS|rkt|d rm|dSdSdS) zInitialize module weights. Args: module: PyTorch module to initialize. name: Module name. zero_init_last: Zero-initialize last layer weights. head.fcr(g{Gz?)meanstdfan_outrelu)mode nonlinearityNrS) isinstancerCLinearConv2drOnormal_rQrPbiaskaiming_normal_ BatchNorm2d LayerNorm GroupNormones_hasattrrS)rr rSrKrKrLrs    rrmodelrrc Csddl}dd}||}t|jjjjd|||d}|jjj||jj|||d|jj |||dt t |j ddt jr~|j jjjd||d jd kr~|j jj|||d |j jj |||d t|jD]\}\}} t| jD]\} \} } d } |d |dd| ddd}| jj|||d| d| jj|||d| d| jj|||d| d| jj|||d| jj|||d| jj|||d| jj |||d| jj |||d| jj |||d| jdurK||d| d}| jjj||qqdS)NrcSs"|jdkr |gd}t|S)zPossibly convert HWIO to OIHW.r)r:r}rr)ndim transposerc from_numpy) conv_weightsrKrKrLt2ps  z_load_weights..t2prz%root_block/standardized_conv2d/kernelzgroup_norm/gammazgroup_norm/betarzhead/conv2d/kernelzhead/conv2d/biasstandardized_conv2dblockz/unit02d/za/z/kernelzb/zc/za/group_norm/gammazb/group_norm/gammazc/group_norm/gammaza/group_norm/betazb/group_norm/betazc/group_norm/betaza/proj/)numpyloadr rrurQrcopy_rvrrrqrrCrrrrnamed_childrenrr@rBrir?rArhr>)rrrnpr"weights stem_conv_wisnamerjbnamer%cname block_prefixwrKrKrLrs@ $ """ rFvariant pretrainedkwargscKs"tdd}tt||fd|i|S)zCreate a ResNetV2 model. Args: variant: Model variant name. pretrained: Load pretrained weights. **kwargs: Additional model arguments. Returns: ResNetV2 model instance. T)flatten_sequential feature_cfg)rrr$)r6r7r8r:rKrKrL_create_resnetv2=s r;cKs t|f|dttddd|S)zCreate a ResNetV2 model with BiT weights. Args: variant: Model variant name. pretrained: Load pretrained weights. **kwargs: Additional model arguments. Returns: ResNetV2 model instance. rg:0yE>)eps)r7rr1)r;rr)r6r7r8rKrKrL_create_resnetv2_bitPs  r=urlc Ks|dddddttddd |S) Nr)r:r?)rrg?bilinearrr ) r>r input_size pool_sizecrop_pct interpolationr r  first_conv classifierr )r>r8rKrKrL_cfgdsrGz%resnetv2_50x1_bit.goog_distilled_in1kztimm/bicubic) hf_hub_idrD custom_loadz-resnetv2_152x2_bit.goog_teacher_in21k_ft_in1kz1resnetv2_152x2_bit.goog_teacher_in21k_ft_in1k_384)r:rK) rLr')rIrArBrCrDrJz$resnetv2_50x1_bit.goog_in21k_ft_in1k)r:rM)rN)rIrArBrCrJz$resnetv2_50x3_bit.goog_in21k_ft_in1kz%resnetv2_101x1_bit.goog_in21k_ft_in1kz%resnetv2_101x3_bit.goog_in21k_ft_in1kz%resnetv2_152x2_bit.goog_in21k_ft_in1kz%resnetv2_152x4_bit.goog_in21k_ft_in1k)r:rO)rPzresnetv2_50x1_bit.goog_in21kiSU)rIrrJzresnetv2_50x3_bit.goog_in21kzresnetv2_101x1_bit.goog_in21kzresnetv2_101x3_bit.goog_in21kzresnetv2_152x2_bit.goog_in21kzresnetv2_152x4_bit.goog_in21kzresnetv2_18.ra4_e3600_r224_in1kg?)r: rQ)rIrDrCtest_input_size test_crop_pctz resnetv2_18d.ra4_e3600_r224_in1kz stem.conv1)rIrDrCrRrSrE)rIrCrArBrRrDrEgffffff?)rDrE)rD)rIrDrErCrRrS) zresnetv2_34.ra4_e3600_r224_in1kz resnetv2_34d.ra4_e3600_r224_in1kz resnetv2_34d.ra4_e3600_r384_in1kzresnetv2_50.a1h_in1kzresnetv2_50d.untrainedzresnetv2_50t.untrainedzresnetv2_101.a1h_in1kzresnetv2_101d.untrainedzresnetv2_152.untrainedzresnetv2_152d.untrainedzresnetv2_50d_gn.ah_in1kzresnetv2_50d_evos.ah_in1kzresnetv2_50d_frn.untrainedcKt d|gddd|S)zResNetV2-50x1-BiT model.resnetv2_50x1_bitr:rr:rr7rrN)rUr=r7r8rKrKrLrU rUcKrT)zResNetV2-50x3-BiT model.resnetv2_50x3_bitrVr:rXN)r\rYrZrKrKrLr\r[r\cKrT)zResNetV2-101x1-BiT model.resnetv2_101x1_bitr:rr:rrXN)r]rYrZrKrKrLr]r[r]cKrT)zResNetV2-101x3-BiT model.resnetv2_101x3_bitr^r:rXN)r`rYrZrKrKrLr`r[r`cKrT)zResNetV2-152x2-BiT model.resnetv2_152x2_bitr:r$r:r}rXN)rarYrZrKrKrLrar[racKrT)zResNetV2-152x4-BiT model.resnetv2_152x4_bitrbrrXN)rdrYrZrKrKrLrdr[rdcKs6tgddddttd}td d|it|fi|S) zResNetV2-18 model.r}r}r}r}rrrTr'rrrr+r1r2 resnetv2_18r7N)rirrrr;r7r8 model_argsrKrKrLris  ric Ks:tgddddttddd}td d|it|fi|S) z'ResNetV2-18d model (deep stem variant).rerfTr'rrrrr+r1r2rr resnetv2_18dr7N)rnrjrkrKrKrLrns  rncKs2tddddttd}td d|it|fi|S) zResNetV2-34 model.rVrfTr'rh resnetv2_34r7N)rorjrkrKrKrLros roc Ks6tddddttddd}td d|it|fi|S) z'ResNetV2-34d model (deep stem variant).rVrfTr'rrm resnetv2_34dr7N)rprjrkrKrKrLrps rpcK0tgdttd}tdd|it|fi|S)zResNetV2-50 model.rVrr1r2 resnetv2_50r7N)rsrjrkrKrKrLrs%rscK4tgdttddd}tdd|it|fi|S) z'ResNetV2-50d model (deep stem variant).rVrTrr1r2rr resnetv2_50dr7N)rwrjrkrKrKrLrw,  rwcKru) z)ResNetV2-50t model (tiered stem variant).rVrTrv resnetv2_50tr7N)ryrjrkrKrKrLry5rxrycKrq)zResNetV2-101 model.r^rr resnetv2_101r7N)rzrjrkrKrKrLrz>rtrzcKru) z(ResNetV2-101d model (deep stem variant).r^rTrv resnetv2_101dr7N)r{rjrkrKrKrLr{Erxr{cKrq)zResNetV2-152 model.rbrr resnetv2_152r7N)r|rjrkrKrKrLr|Nrtr|cKru) z(ResNetV2-152d model (deep stem variant).rbrTrv resnetv2_152dr7N)r}rjrkrKrKrLr}Urxr}cKru) z,ResNetV2-50d model with Group Normalization.rVrTrvresnetv2_50d_gnr7N)r~)rrrr;rkrKrKrLr~`rxr~cKru) z ResNetV2-50d model with EvoNorm.rVrTrvresnetv2_50d_evosr7N)r)rrrr;rkrKrKrLrirxrcKru) z6ResNetV2-50d model with Filter Response Normalization.rVrTrvresnetv2_50d_frnr7N)r)rrrr;rkrKrKrLrrrxr)resnetv2_50x1_bitmresnetv2_50x3_bitmresnetv2_101x1_bitmresnetv2_101x3_bitmresnetv2_152x2_bitmresnetv2_152x4_bitmresnetv2_50x1_bitm_in21kresnetv2_50x3_bitm_in21kresnetv2_101x1_bitm_in21kresnetv2_101x3_bitm_in21kresnetv2_152x2_bitm_in21kresnetv2_152x4_bitm_in21kresnetv2_50x1_bit_distilledresnetv2_152x2_bit_teacherresnetv2_152x2_bit_teacher_384)rTrr)r)Xr` collectionsr functoolsrtypingrrrrrr r rctorch.nnrC timm.datar r timm.layersrrrrrrrrrrrrr_builderr _featuresr _manipulaterrr _registryr!r"r#__all__r r&rfrkrsr|rrr{rrarrr$rno_gradrr;r=rG default_cfgsrUr\r]r`rardrirnrorprsrwryrzr{r|r}r~rrr]rKrKrKrLs  $ <  XaO$?  -%    !$'*-047; d