o پi @sl dZddlmZddlmZmZmZmZmZm Z m Z ddl Z ddl m Z ddlm mZddlmZmZmZmZddlmZmZmZmZmZmZddlmZmZdd l m!Z!dd l"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z*dd l+m,Z,m-Z-m.Z.dd l/m0Z0m1Z1dd l2m3Z3m4Z4m5Z5ddgZ6Gddde j7Z8Gddde j7Z9dde:de;de8fddZ< dde:de=de;de8fddZ>    dde:de=de=dee?de;de8f ddZ@dde:de=de;de8fd d!ZAdde:de=de;de8fd"d#ZB   dde:de=dee?de;de8f d$d%ZCdd'e:dee:effd(d)ZDe3id*eDd&d+d,eDd-d.d/d0d1eDd/d-eed2d3dd4d5eDd6d7d8d9d:d;d/d<d=eDd>d/d9d:d6d7d8d?d@dAeDd/eed3d2dBdCddDdEeDdFd/d-dGdHeDdId/d-dGdJeDdKd/d-dGdLeDdMd/d-dGdNeDdOd/eedPdQeDdRd/eedPdSeDdTd/eedPdUeDdVd/eedPdWeDdXd/eedPdYeDdZd/eedPd[eDd\d/d3d2d]id^eDd_d/d3d2d]d`eDdad/dbdcd2dBdddeeDdfeDdgd/d-dGdheDdid/d-dGdjeDdkd/d-dGdleDdmeDeed3d2d-dndoeDd/eed3d2d-dpdqeDd/d3d2d-drdseDd/d3d2d-drdteDd/eed3dBd2dCdd-du dveDd/d3dBd2dCdd-dwdxeDd/d2d3dd-dydzeDd/d{d|d2d-d}d~eDd/dd{d|dd-ddeDd/dd{d|dd-dideDd/dd{d|dd-ddeDd/d{d|d2ddd-dwdeDd/d3dBd2dCdd-dwdeDd/d3dBd2dCdd-dwdeDd/d3dBd2dCdd-dwdeDd/d{d|d2ddd-dwdeDd/d2d3dd-dydeDd/dd3dBd2dCdd-ddeDd/d{d|d2ddd-dwdeDd/d{d|d2ddd-dwdeDd3dBd2d-ddeDd/d2d3dd-dydeDd/ddd2ddd-dwdeDd/d{d|d2ddd-dwdeDd/d{d|d2ddd-dwdeDd/dd{d|d2ddd-ddeDd2d-ddeDd3dBd2d-diZEe4dde;de8fddZFe4dde;de8fddZGe4dde;de8fddZHe4dde;de8fddZIe4dde;de8fddZJe4dde;de8fddZKe4dde;de8fddZLe4dde;de8fddZMe4dde;de8fddZNe4dde;de8fddZOe4dde;de8fddZPe4dde;de8fddZQe4dde;de8fddZRe4dde;de8fddZSe4dde;de8fddZTe4dde;de8fddZUe4dde;de8fddZVe4dde;de8fddZWe4dde;de8fddZXe4dde;de8fdd„ZYe4dde;de8fddĄZZe4dde;de8fddƄZ[e4dde;de8fddȄZ\e4dde;de8fddʄZ]e4dde;de8fdd̄Z^e4dde;de8fdd΄Z_e4dde;de8fddЄZ`e4dde;de8fdd҄Zae4dde;de8fddԄZbe4dde;de8fddքZce4dde;de8fdd؄Zde4dde;de8fddڄZee4dde;de8fdd܄Zfe5egd5d=dݜdS)z MobileNet V3 A PyTorch impl of MobileNet-V3, compatible with TF weights from official impl. Paper: Searching for MobileNetV3 - https://arxiv.org/abs/1905.02244 Hacked together by / Copyright 2019, Ross Wightman )partial)AnyDictCallableListOptionalTupleUnionN)IMAGENET_DEFAULT_MEANIMAGENET_DEFAULT_STDIMAGENET_INCEPTION_MEANIMAGENET_INCEPTION_STD)SelectAdaptivePool2dLinear LayerTypePadType create_conv2dget_norm_act_layer)build_model_with_cfgpretrained_cfg_for_features) SqueezeExcite) BlockArgsEfficientNetBuilderdecode_arch_defefficientnet_init_weightsround_channelsresolve_bn_argsresolve_act_layerBN_EPS_TF_DEFAULT) FeatureInfo FeatureHooksfeature_take_indices)checkpoint_seq checkpoint)generate_default_cfgsregister_modelregister_model_deprecations MobileNetV3MobileNetV3Featuresc'seZdZdZddddddddd d d d ded d d d fd ed edededededededede e de e de e de e dede de de de e def&fdd Z d!ejfd"d#ZejjdDd$ed!eeeffd%d&ZejjdEd'ed!d fd(d)Zejjd!ejfd*d+ZdFd eded!d fd,d-Z   .  dGd/ejd0e eeeefd1ed2ed3ed4ed5ed!eeejeejeejfffd6d7Z 8   dHd0eeeefd9ed:ed5ed!eef d;d<Z d/ejd!ejfd=d>Z!dDd/ejd?ed!ejfd@dAZ"d/ejd!ejfdBdCZ#Z$S)Ir(aMobileNetV3. Based on my EfficientNet implementation and building blocks, this model utilizes the MobileNet-v3 specific 'efficient head', where global pooling is done before the head convolution without a final batch-norm layer before the classifier. Paper: `Searching for MobileNetV3` - https://arxiv.org/abs/1905.02244 Other architectures utilizing MobileNet-V3 efficient head that are supported by this impl include: * HardCoRe-NAS - https://arxiv.org/abs/2102.11646 (defn in hardcorenas.py uses this class) * FBNet-V3 - https://arxiv.org/abs/2006.02049 * LCNet - https://arxiv.org/abs/2109.15099 * MobileNet-V4 - https://arxiv.org/abs/2404.10518 FTNavg block_args num_classesin_chans stem_sizefix_stem num_features head_bias head_normpad_type act_layer norm_layeraa_layerse_layer se_from_exp round_chs_fn drop_ratedrop_path_ratelayer_scale_init_value global_poolc stt|| p tj} | ptj} t| | }| pt} ||_||_ d|_ |s)||}t ||dd| d|_ ||dd|_ td| ||| | | | ||d }tj||||_|j|_d d |jD|_|j|_||_t|d |_|j|j}|rt ||jd | d |_||j|_t|_nt ||jd | |d|_t|_| dd|_|rtd nt|_|dkrt |j|nt|_!t"|dS)a0Initialize MobileNetV3. Args: block_args: Arguments for blocks of the network. num_classes: Number of classes for classification head. in_chans: Number of input image channels. stem_size: Number of output channels of the initial stem convolution. fix_stem: If True, don't scale stem by round_chs_fn. num_features: Number of output channels of the conv head layer. head_bias: If True, add a learnable bias to the conv head layer. head_norm: If True, add normalization to the head layer. pad_type: Type of padding to use for convolution layers. act_layer: Type of activation layer. norm_layer: Type of normalization layer. aa_layer: Type of anti-aliasing layer. se_layer: Type of Squeeze-and-Excite layer. se_from_exp: If True, calculate SE channel reduction from expanded mid channels. round_chs_fn: Callable to round number of filters based on depth multiplier. drop_rate: Dropout rate. drop_path_rate: Stochastic depth rate. layer_scale_init_value: Enable layer scale on compatible blocks if not None. global_pool: Type of pooling to use for global pooling features of the FC head. Fr+stridepaddingTinplace ) output_strider9r?r>r:r;r<r=rArBcSsg|]}|dqS)stage.0frMrMK/home/ubuntu/.local/lib/python3.10/site-packages/timm/models/mobilenetv3.py xsz(MobileNetV3.__init__.. pool_typer)rG)rGbiasrN)#superr(__init__nnReLU BatchNorm2drrr2r@grad_checkpointingr conv_stembn1r Sequentialblocksfeatures feature_info stage_endsin_chsr6head_hidden_sizerrC feat_mult conv_head norm_headIdentityact2Flattenflattenr classifierr)selfr1r2r3r4r5r6r7r8r9r:r;r<r=r>r?r@rArBrCnorm_act_layerbuildernum_pooled_chs __class__rMrQrW-sR-          zMobileNetV3.__init__returncCsZ|j|jg}||j||j|j|j|jg|t t |j |j gtj |S)zqConvert model to sequential form. Returns: Sequential module containing all layers. )r\r]extendr_rCrfrgrirXrjDropoutr@rlr^)rmlayersrMrMrQ as_sequentials   zMobileNetV3.as_sequentialcoarsecCstd|rddSddS)z"Group parameters for optimization.z^conv_stem|bn1z^blocks\.(\d+)z^blocks\.(\d+)\.(\d+))stemr_)dict)rmrxrMrMrQ group_matchers zMobileNetV3.group_matcherenablecC ||_dSz)Enable or disable gradient checkpointing.Nr[rmr|rMrMrQset_grad_checkpointing z"MobileNetV3.set_grad_checkpointingcCs|jS)zGet the classifier head.)rlrmrMrMrQget_classifierszMobileNetV3.get_classifiercCsR||_t|d|_|rtdnt|_|dkr"t|j||_ dSt|_ dS)zReset the classifier head. Args: num_classes: Number of classes for new classifier. global_pool: Global pooling type. rSrrN) r2rrCrXrjrhrkrrdrl)rmr2rCrMrMrQreset_classifiers (zMobileNetV3.reset_classifierNCHWxindicesnorm stop_early output_fmtintermediates_only extra_blockscs|dvsJd|r|sJdg}|r!ttjd|\} } nttj|\} } fdd| D} j| } d} |}|}| | vrN||tj sU|sYj} njd| } t | dd D]\} } j rxtj sxt | |}n| |}| | vr||qf|r|S||fS) aa Forward features that returns intermediates. Args: x: Input image tensor indices: Take last n blocks if int, all if None, select matching indices if sequence norm: Apply norm layer to compatible intermediates stop_early: Stop iterating over blocks when last desired intermediate hit output_fmt: Shape of intermediate feature outputs intermediates_only: Only return intermediate features extra_blocks: Include outputs of all blocks and head conv in output, does not align with feature_info Returns: )rzOutput shape must be NCHW.z/Must use intermediates_only for early stopping.rcsg|]}j|qSrM)rb)rOirrMrQrRsz5MobileNetV3.forward_intermediates..rN)start) r"lenr_rbr\r]appendtorchjit is_scripting enumerater[r#)rmrrrrrrr intermediates take_indices max_indexfeat_idxr_blkrMrrQforward_intermediatess6       z!MobileNetV3.forward_intermediatesr prune_norm prune_headcCs|rtt|jd|\}}ntt|j|\}}|j|}|jd||_|t|jkr7t|_t|_|rIt|_t|_|dd|S)a~Prune layers not required for specified intermediates. Args: indices: Indices of intermediate layers to keep. prune_norm: Whether to prune normalization layer. prune_head: Whether to prune the classifier head. extra_blocks: Include outputs of all blocks. Returns: List of indices that were kept. rNrr.) r"rr_rbrXrhrfrgr)rmrrrrrrrMrMrQprune_intermediate_layerss      z%MobileNetV3.prune_intermediate_layerscCsF||}||}|jrtjst|j|dd}|S||}|S)zForward pass through feature extraction layers. Args: x: Input tensor. Returns: Feature tensor. T)rk)r\r]r[rrrr#r_rmrrMrMrQforward_featuress  zMobileNetV3.forward_features pre_logitscCsb||}||}||}||}||}|jdkr(tj||j|jd}|r,|S| |S)zForward pass through classifier head. Args: x: Input features. pre_logits: Return features before final linear layer. Returns: Classification logits or features. r/)ptraining) rCrfrgrirkr@Fdropoutrrl)rmrrrMrMrQ forward_head%s      zMobileNetV3.forward_headcCs||}||}|S)zoForward pass. Args: x: Input tensor. Returns: Output logits. )rrrrMrMrQforward:s zMobileNetV3.forwardFT)r0)NFFrFF)rFTF)%__name__ __module__ __qualname____doc__rrintboolstrrrrfloatrWrXr^rwrrignorerrr{rModulerrTensorr rrrrrrr __classcell__rMrMrqrQr(s     a  < !c$seZdZdZddddddded d d d d d d d fd ed eedfdededede dede de de de e de e de e de e dedede ef"fdd Zejjd'd e d!d fd"d#Zd$ejd!eejfd%d&ZZS)(r)zMobileNetV3 Feature Extractor. A work-in-progress feature extraction module for MobileNet-V3 to use as a backbone for segmentation and object detection models. )rrrDr+ bottleneckr+r,FrJr.TNr/r1 out_indices.feature_locationr3r4r5rKr9r?r>r:r;r<r=r@rArBc stt|| p tj} | ptj} |pt}||_d|_|s!| |}t ||dd|d|_ | ||_ | dd|_ t ||| | | | | ||||d }tj||||_t|j||_dd |jD|_t|d |_|d kr||jjd d }t|||_d Sd S)aInitialize MobileNetV3Features. Args: block_args: Arguments for blocks of the network. out_indices: Output from stages at indices. feature_location: Location of feature before/after each block, must be in ['bottleneck', 'expansion']. in_chans: Number of input image channels. stem_size: Number of output channels of the initial stem convolution. fix_stem: If True, don't scale stem by round_chs_fn. output_stride: Output stride of the network. pad_type: Type of padding to use for convolution layers. round_chs_fn: Callable to round number of filters based on depth multiplier. se_from_exp: If True, calculate SE channel reduction from expanded mid channels. act_layer: Type of activation layer. norm_layer: Type of normalization layer. aa_layer: Type of anti-aliasing layer. se_layer: Type of Squeeze-and-Excite layer. drop_rate: Dropout rate. drop_path_rate: Stochastic depth rate. layer_scale_init_value: Enable layer scale on compatible blocks if not None. Fr+rDrETrH) rKr9r?r>r:r;r<r=rArBrcSsi|] }|d|dqS)rLindexrMrNrMrMrQ sz0MobileNetV3Features.__init__..Nr)module hook_type)keys)rVr)rWrXrYrZrr@r[rr\r]act1rr^r_r r`ra get_dicts_stage_out_idxr feature_hooksr! named_modules)rmr1rrr3r4r5rKr9r?r>r:r;r<r=r@rArBrohooksrqrMrQrWOsB)     zMobileNetV3Features.__init__r|rscCr}r~rrrMrMrQrrz*MobileNetV3Features.set_grad_checkpointingrcCs||}||}||}|jdurJg}d|jvr ||t|jD]"\}}|jr7t j s7t ||}n||}|d|jvrG||q%|S|||j |j}t|S)zForward pass through feature extraction. Args: x: Input tensor. Returns: List of feature tensors. Nrr)r\r]rrrrrr_r[rrrr$ get_outputdevicelistvalues)rmrr`rboutrMrMrQrs$         zMobileNetV3Features.forwardr)rrrrrrrrrrrrrrrrWrrrrrrrrrMrMrqrQr)Hsp       Q"Fvariant pretrainedrscKsvd}t}d}|ddrd|vsd|vrd}nd}t}d }t|||f|dk|d k|d |}|d kr9t|j|_|S) zCreate a MobileNetV3 model. Args: variant: Model variant name. pretrained: Load pretrained weights. **kwargs: Additional model arguments. Returns: MobileNetV3 model instance. r.N features_onlyF feature_cfg feature_clscfg)r2r6 head_convr7r8rCcls)rpretrained_strict kwargs_filter)r(popr)rr default_cfg)rrkwargs features_mode model_clsrmodelrMrMrQ _create_mnv3s.   r?channel_multiplierc Ksdgddgdggddgdgdgg}tdt|d tt|d ttjfit|t|d ttd d d|}t ||fi|}|S)adCreates a MobileNet-V3 model. Ref impl: ? Paper: https://arxiv.org/abs/1905.02244 Args: variant: Model variant name. channel_multiplier: Multiplier to number of channels per layer. pretrained: Load pretrained weights. **kwargs: Additional model arguments. Returns: MobileNetV3 model instance. ds_r1_k3_s1_e1_c16_nre_noskipir_r1_k3_s2_e4_c24_nreir_r1_k3_s1_e3_c24_nreir_r3_k5_s2_e3_c40_se0.25_nreir_r1_k3_s2_e6_c80zir_r1_k3_s1_e2.5_c80zir_r2_k3_s1_e2.3_c80ir_r2_k3_s1_e6_c112_se0.25ir_r3_k5_s2_e6_c160_se0.25cn_r1_k1_s1_c960F multiplier hard_swish hard_sigmoid) gate_layer)r1r7r?r;r:r=NrM) rzrrrrXrZrrrrrrrrarch_def model_kwargsrrMrMrQ_gen_mobilenet_v3_rws(   rdepth_multiplier group_sizec KsFd|vr6d}d|vr t|d}dgddgdd gd gd gd gg}nKt|d }dgddgddgdgdgd gg}n5d}d|vrTt|d}dgddgdggddgdgdgg}nt|d }dgdd gd!ggdd"gd#gdgg}ttd$tjtd%} td,t|||d&|d'|d(ktt|d)ttjfit ||| d*|} t ||fi| } | S)-aCreates a MobileNet-V3 model. Ref impl: ? Paper: https://arxiv.org/abs/1905.02244 Args: variant: Model variant name. channel_multiplier: Multiplier to number of channels per layer. depth_multiplier: Depth multiplier for model scaling. group_size: Group size for grouped convolutions. pretrained: Load pretrained weights. **kwargs: Additional model arguments. Returns: MobileNetV3 model instance. smalliminimalreluds_r1_k3_s2_e1_c16zir_r1_k3_s2_e4.5_c24zir_r1_k3_s1_e3.67_c24ir_r1_k3_s2_e4_c40ir_r2_k3_s1_e6_c40ir_r2_k3_s1_e3_c48ir_r3_k3_s2_e6_c96cn_r1_k1_s1_c576rzds_r1_k3_s2_e1_c16_se0.25_nrezir_r1_k3_s2_e4.5_c24_nrezir_r1_k3_s1_e3.67_c24_nreir_r1_k5_s2_e4_c40_se0.25zir_r2_k5_s1_e6_c40_se0.25zir_r2_k5_s1_e3_c48_se0.25zir_r3_k5_s2_e6_c96_se0.25r-ds_r1_k3_s1_e1_c16ir_r1_k3_s2_e4_c24ir_r1_k3_s1_e3_c24ir_r3_k3_s2_e3_c40rir_r2_k3_s1_e6_c112ir_r3_k3_s2_e6_c160rds_r1_k3_s1_e1_c16_nrerrrrrr)rforce_act_layer rd_round_fn)rrr,?r)r1r6r4r5r?r;r:r=NrM) rrrrXrYrrzrrZrr) rrrrrrr6r:rr=rrrMrMrQ_gen_mobilenet_v3sl        rc Ks.|dd}|dvr#d}dgddgdd gd d gd d ggddgg}n:|dkr?d}dgddgddgdd gddggddgg}n|dkr[d}dgddgd d!gd"d#gd$d%ggd&d'gg}nttt|d(d)}ttd*|d+}t|d,} td1t|d-d.||d.ttj fit || |d/ |} t ||fi| } | S)2aFBNetV3 model generator. Paper: `FBNetV3: Joint Architecture-Recipe Search using Predictor Pretraining` - https://arxiv.org/abs/2006.02049 FIXME untested, this is a preliminary impl of some FBNet-V3 variants. Args: variant: Model variant name. channel_multiplier: Channel width multiplier. pretrained: Load pretrained weights. **kwargs: Additional model arguments. Returns: MobileNetV3 model instance. _)arr,ds_r2_k3_s1_e1_c16ir_r1_k5_s2_e4_c24ir_r3_k5_s1_e2_c24zir_r1_k5_s2_e5_c40_se0.25zir_r4_k5_s1_e3_c40_se0.25ir_r1_k5_s2_e5_c72ir_r4_k3_s1_e3_c72zir_r1_k3_s1_e5_c120_se0.25zir_r5_k5_s1_e3_c120_se0.25)zir_r1_k3_s2_e6_c184_se0.25zir_r5_k5_s1_e4_c184_se0.25zir_r1_k5_s1_e6_c224_se0.25cn_r1_k1_s1_c1344dir_r1_k3_s2_e5_c24ir_r5_k3_s1_e2_c24rzir_r4_k3_s1_e3_c40_se0.25ir_r1_k3_s2_e5_c72zir_r1_k3_s1_e5_c128_se0.25zir_r6_k5_s1_e3_c128_se0.25)zir_r1_k3_s2_e6_c208_se0.25zir_r5_k5_s1_e5_c208_se0.25zir_r1_k5_s1_e6_c240_se0.25cn_r1_k1_s1_c1440grJds_r3_k3_s1_e1_c24ir_r1_k5_s2_e4_c40ir_r4_k5_s1_e2_c40zir_r1_k5_s2_e4_c56_se0.25zir_r4_k5_s1_e3_c56_se0.25ir_r1_k5_s2_e5_c104ir_r4_k3_s1_e3_c104zir_r1_k3_s1_e5_c160_se0.25zir_r8_k5_s1_e3_c160_se0.25)zir_r1_k3_s2_e6_c264_se0.25zir_r6_k5_s1_e5_c264_se0.25zir_r2_k5_s1_e6_c288_se0.25cn_r1_k1_s1_c1728ffffff?)r round_limitr)rrriF) r1r6r7r4r?r>r;r:r=NrM) splitNotImplementedrrrrrzrrXrZrr) rrrrvlr4rr?r=r:rrrMrMrQ _gen_fbnetv3sd      r"c Ksdgdgdgddgdgdgg}tdt|dtt|d ttjfit|t|d ttd tj d d d|}t ||fi|}|S)aLCNet model generator. Essentially a MobileNet-V3 crossed with a MobileNet-V1 Paper: `PP-LCNet: A Lightweight CPU Convolutional Neural Network` - https://arxiv.org/abs/2109.15099 Args: variant: Model variant name. channel_multiplier: Multiplier to number of channels per layer. pretrained: Load pretrained weights. **kwargs: Additional model arguments. Returns: MobileNetV3 model instance. dsa_r1_k3_s1_c32dsa_r2_k3_s2_c64dsa_r2_k3_s2_c128dsa_r1_k3_s2_c256dsa_r1_k5_s1_c256dsa_r4_k5_s1_c256zdsa_r2_k5_s2_c512_se0.25r,rrr)rrr-)r1r4r?r;r:r=r6NrM) rzrrrrXrZrrrrYrrrMrMrQ _gen_lcnets(  r)c Ksd}d|vrFd}d|vr#d}t|d}dgdd ggd gd d gg} nd |vr>d}t|d}dgddggdgdd gg} ndJd|dd}d|vrdd}t|d}ddgddggdgdd gg} n>d|vrd}t|d}dgdd ggdgd d gg} n#d |vrd}t|d}dgddggd!gd"d gg} nJd|dtd(t| |d#dd$|||d%ktt|d&ttjfit|||d' |} t||fi| } | S))aCreates a MobileNet-V4 model. Paper: https://arxiv.org/abs/2404.10518 Args: variant: Model variant name. channel_multiplier: Multiplier to number of channels per layer. group_size: Group size for grouped convolutions. pretrained: Load pretrained weights. **kwargs: Additional model arguments. Returns: MobileNetV3 model instance. r-hybridgh㈵>mediumrJrer_r1_k3_s2_e4_c48uir_r1_a3_k5_s2_e4_c80uir_r1_a3_k3_s1_e2_c80) uir_r1_a3_k5_s2_e6_c160uir_r1_a0_k0_s1_e2_c160uir_r1_a3_k3_s1_e4_c160uir_r1_a3_k5_s1_e4_c160mqa_r1_k3_h4_s1_v2_d64_c160r1r3uir_r1_a3_k0_s1_e4_c160r3r1r3r4)uir_r1_a5_k5_s2_e6_c256uir_r1_a5_k5_s1_e4_c256uir_r2_a3_k5_s1_e4_c256uir_r1_a0_k0_s1_e2_c256uir_r1_a3_k5_s1_e2_c256r8uir_r1_a0_k0_s1_e4_c256mqa_r1_k3_h4_s1_d64_c256uir_r1_a3_k0_s1_e4_c256r;r6r;uir_r1_a5_k0_s1_e4_c256r;r=rlargergeluuir_r1_a3_k5_s2_e4_c96uir_r1_a3_k3_s1_e4_c96) uir_r1_a3_k5_s2_e4_c192uir_r3_a3_k3_s1_e4_c192uir_r1_a3_k5_s1_e4_c192uir_r2_a5_k3_s1_e4_c192mqa_r1_k3_h8_s1_v2_d48_c192uir_r1_a5_k3_s1_e4_c192rFrGrFrGrFuir_r1_a3_k0_s1_e4_c192)uir_r4_a5_k5_s2_e4_c512uir_r1_a5_k0_s1_e4_c512uir_r1_a5_k3_s1_e4_c512uir_r2_a5_k0_s1_e4_c512rKuir_r1_a5_k5_s1_e4_c512mqa_r1_k3_h8_s1_d64_c512rJrNrJrNrJrNrJFzUnknown variant .Nrcn_r1_k3_s2_e1_c32cn_r1_k1_s1_e1_c32cn_r1_k3_s2_e1_c96cn_r1_k1_s1_e1_c64)uir_r1_a5_k5_s2_e3_c96uir_r4_a0_k3_s1_e2_c96uir_r1_a3_k0_s1_e4_c96)uir_r1_a3_k3_s2_e6_c128uir_r1_a5_k5_s1_e4_c128uir_r1_a0_k5_s1_e4_c128uir_r1_a0_k5_s1_e3_c128uir_r2_a0_k3_s1_e4_c128)r/uir_r2_a3_k3_s1_e4_c160r2r1r4r0r4) r5r6r7r:r<r9r6uir_r2_a0_k0_s1_e4_c256uir_r1_a5_k0_s1_e2_c256)rBrCrDuir_r5_a5_k3_s1_e4_c192rH)rIrJrKrLrKrMuir_r3_a5_k0_s1_e4_c512)rTrr) r1r7r8r6r4r5r?r;r:rBrM) rrzrrrrXrZrr) rrrrrr6rBr4r:rrrrMrMrQ_gen_mobilenet_v4s 0 0     %  #    rar.urlc Ks|dddddttddd |S) zCreate default configuration dictionary. Args: url: Model weight URL. **kwargs: Additional configuration options. Returns: Configuration dictionary. r*)r+rc)rdg?bilinearr\rl) rbr2 input_size pool_sizecrop_pct interpolationmeanstd first_convrl)r r )rbrrMrMrQ_cfgs rmzmobilenetv3_large_075.untrained)rbzmobilenetv3_large_100.ra_in1kbicubiczvhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-weights/mobilenetv3_large_100_ra-f55367f5.pthztimm/)rirb hf_hub_idz)mobilenetv3_large_100.ra4_e3600_r224_in1kr)r+rp)rorirjrkrhtest_input_size test_crop_pctz(mobilenetv3_large_100.miil_in21k_ft_in1kre)r/r/r/)rrrz+https://github.com/Alibaba-MIIL/ImageNet21KzarXiv:2104.10972v4zhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-tresnet/mobilenetv3_large_100_1k_miil_78_0-66471c13.pth)rirjrk origin_url paper_idsrbroz mobilenetv3_large_100.miil_in21kz~https://github.com/rwightman/pytorch-image-models/releases/download/v0.1-tresnet/mobilenetv3_large_100_in21k_miil-d71cc17b.pthi+)rbrorsrtrirjrkr2z*mobilenetv3_large_150d.ra4_e3600_r256_in1k)ru)r+@rv)rorjrkrfrhrgrqrrzmobilenetv3_small_050.lamb_in1kzyhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-weights/mobilenetv3_small_050_lambc-4b7bbe87.pth)rbrorizmobilenetv3_small_075.lamb_in1kzyhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-weights/mobilenetv3_small_075_lambc-384766db.pthzmobilenetv3_small_100.lamb_in1kzxhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-weights/mobilenetv3_small_100_lamb-266a294c.pthzmobilenetv3_rw.rmsp_in1kzmhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-weights/mobilenetv3_100-35495452.pthztf_mobilenetv3_large_075.in1kzvhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-weights/tf_mobilenetv3_large_075-150ee8b0.pth)rbrorjrkztf_mobilenetv3_large_100.in1kzvhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-weights/tf_mobilenetv3_large_100-427764d5.pthz%tf_mobilenetv3_large_minimal_100.in1kz~https://github.com/rwightman/pytorch-image-models/releases/download/v0.1-weights/tf_mobilenetv3_large_minimal_100-8596ae28.pthztf_mobilenetv3_small_075.in1kzvhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-weights/tf_mobilenetv3_small_075-da427f52.pthztf_mobilenetv3_small_100.in1kzvhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-weights/tf_mobilenetv3_small_100-37f49e2b.pthz%tf_mobilenetv3_small_minimal_100.in1kz~https://github.com/rwightman/pytorch-image-models/releases/download/v0.1-weights/tf_mobilenetv3_small_minimal_100-922a7843.pthzfbnetv3_b.ra2_in1kzkhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-weights/fbnetv3_b_224-ead5d2a1.pth)rbrorqrhzfbnetv3_d.ra2_in1kzkhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-weights/fbnetv3_d_224-c98bce42.pthzfbnetv3_g.ra2_in1kzkhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-weights/fbnetv3_g_240-0b1df83b.pth)r+rw)r+ rx)rbrorfrqrhrgzlcnet_035.untrainedzlcnet_050.ra2_in1kzghttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-weights/lcnet_050-f447553b.pthzlcnet_075.ra2_in1kzghttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-weights/lcnet_075-318cad2c.pthzlcnet_100.ra2_in1kzghttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-weights/lcnet_100-a929038c.pthzlcnet_150.untrainedz$mobilenetv4_conv_small_035.untrained)rjrkrqrrriz*mobilenetv4_conv_small_050.e3000_r224_in1k)rorjrkrqrrriz&mobilenetv4_conv_small.e2400_r224_in1k)rorqrrriz&mobilenetv4_conv_small.e1200_r224_in1kz&mobilenetv4_conv_small.e3600_r256_in1k) rorjrkrfrgrhrqrrriz&mobilenetv4_conv_medium.e500_r256_in1k)rorfrgrhrqrrriz&mobilenetv4_conv_medium.e500_r224_in1k)rorhrqrrriz/mobilenetv4_conv_medium.e250_r384_in12k_ft_in1k)r+ry) rz)rorfrgrhriz'mobilenetv4_conv_medium.e180_r384_in12ki-.)ror2rfrgrhriz*mobilenetv4_conv_medium.e180_ad_r384_in12kz'mobilenetv4_conv_medium.e250_r384_in12kz%mobilenetv4_conv_large.e600_r384_in1k)r+r{z%mobilenetv4_conv_large.e500_r256_in1kz1mobilenetv4_hybrid_medium.e200_r256_in12k_ft_in1kz+mobilenetv4_hybrid_medium.ix_e550_r256_in1kz+mobilenetv4_hybrid_medium.ix_e550_r384_in1kz(mobilenetv4_hybrid_medium.e500_r224_in1kz)mobilenetv4_hybrid_medium.e200_r256_in12k)ror2rfrgrhrqrrriz*mobilenetv4_hybrid_large.ix_e600_r384_in1kz'mobilenetv4_hybrid_large.e600_r384_in1kz$mobilenetv4_conv_aa_medium.untrained)rfrgrhriz+mobilenetv4_conv_blur_medium.e500_r224_in1kz1mobilenetv4_conv_aa_large.e230_r448_in12k_ft_in1k)r|)r+ r}z1mobilenetv4_conv_aa_large.e230_r384_in12k_ft_in1k)r+r~z(mobilenetv4_conv_aa_large.e600_r384_in1kz)mobilenetv4_conv_aa_large.e230_r384_in12kz'mobilenetv4_hybrid_medium_075.untrained)rhriz&mobilenetv4_hybrid_large_075.untrainedcKtdd|i|}|S) MobileNet V3 mobilenetv3_large_075rrN)rrrrrrrMrMrQrrcKr)rmobilenetv3_large_100rrN)rrrrrMrMrQrrrcKstdd|d|}|S)rmobilenetv3_large_150d?g333333?)rrN)rrrrrMrMrQrrcKr)rmobilenetv3_small_050?rN)rrrrrMrMrQrrrcKr)rmobilenetv3_small_075rrN)rrrrrMrMrQrrrcKr)rmobilenetv3_small_100rrN)rrrrrMrMrQr rrcKs"|dttdd|i|}|S)rbn_epsmobilenetv3_rwrrN)rr) setdefaultrrrrMrMrQrs rcK.|dt|ddtdd|i|}|S) rrr9sametf_mobilenetv3_large_075rrN)rrrrrrrMrMrQr  rcKr) rrr9rtf_mobilenetv3_large_100rrN)rrrrrMrMrQr"rrcKr) rrr9r tf_mobilenetv3_large_minimal_100rrN)rrrrrMrMrQr+rrcKr) rrr9rtf_mobilenetv3_small_075rrN)rrrrrMrMrQr4rrcKr) rrr9rtf_mobilenetv3_small_100rrN)rrrrrMrMrQr=rrcKr) rrr9r tf_mobilenetv3_small_minimal_100rrN)rrrrrMrMrQrFrrcKtdd|i|}|S)z FBNetV3-B fbnetv3_brN)rr"rrMrMrQrOrrcKr)z FBNetV3-D fbnetv3_drN)rrrrMrMrQrVrrcKr)z FBNetV3-G fbnetv3_grN)rrrrMrMrQr]rrcKr)z PP-LCNet 0.35 lcnet_035ffffff?rN)rrr)rrMrMrQrdrrcKr)z PP-LCNet 0.5 lcnet_050rrN)rrrrrMrMrQrkrrcKr) PP-LCNet 1.0 lcnet_075rrN)rrrrrMrMrQrrrrcKr)r lcnet_100rrN)rrrrrMrMrQryrrcKr)z PP-LCNet 1.5 lcnet_150rrN)rrrrrMrMrQrrrcKr) MobileNet V4 mobilenetv4_conv_small_035rrN)rrrarrMrMrQrrrcKr)rmobilenetv4_conv_small_050rrN)rrrrrMrMrQrrrcKr)rmobilenetv4_conv_smallrrN)rrrrrMrMrQrrrcKr)rmobilenetv4_conv_mediumrrN)rrrrrMrMrQrrrcKr)rmobilenetv4_conv_largerrN)rrrrrMrMrQrrrcKr) MobileNet V4 Hybrid mobilenetv4_hybrid_mediumrrN)rrrrrMrMrQrrrcKr) MobileNet V4 Hybridmobilenetv4_hybrid_largerrN)rrrrrMrMrQrrrcKtd|dd|}|S) MobileNet V4 w/ AvgPool AA mobilenetv4_conv_aa_mediumrr0rr<N)rrrrrMrMrQrrrcKr)z MobileNet V4 Conv w/ Blur AA mobilenetv4_conv_blur_mediumrblurpcrN)rrrrrMrMrQrrrcKr)rmobilenetv4_conv_aa_largerr0rN)rrrrrMrMrQrrrcKr)rmobilenetv4_hybrid_medium_075rrN)rrrrrMrMrQrrrcKr)rmobilenetv4_hybrid_large_075rrN)rrrrrMrMrQrrr)mobilenetv3_large_100_miil mobilenetv3_large_100_miil_in21kr)rF)rrNF)rNF)r.)hr functoolsrtypingrrrrrrr rtorch.nnrXtorch.nn.functional functionalr timm.datar r r r timm.layersrrrrrr_builderrr_efficientnet_blocksr_efficientnet_builderrrrrrrrr _featuresr r!r" _manipulater#r$ _registryr%r&r'__all__rr(r)rrrrrrrr"r)rarm default_cfgsrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrMrMrMrQs $   (-{% 0 pG/    #(-159=AFJNSTY^cehlorw{ !%),15;>AEIMRU[