o پi| @sddlmZddlmZmZmZmZmZmZddl Z ddl m Z ddl m m ZddlmZmZddlmZmZmZmZmZmZmZmZmZddlmZddlmZm Z dd l!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'dd l(m)Z)dd l*m+Z+dd l,m-Z-m.Z.dd l/m0Z0m1Z1ddgZ2e+Gddde j3Z4Gddde j3Z5Gddde j3Z6d0de7de8de6fddZ9d0de7de8de6fddZ:    d1de7de;de8de8de6f dd Ze1d0de8de6fd*d+Z?e1d0de8de5fd,d-Z@e1d0de8de5fd.d/ZAdS)3)partial)CallableListOptionalSequenceTupleUnionNIMAGENET_INCEPTION_MEANIMAGENET_INCEPTION_STD) SelectAdaptivePool2dLinear LayerTypePadType RmsNorm2d ConvNormAct create_conv2dget_norm_act_layer to_2tuple)build_model_with_cfg) SqueezeExciteUniversalInvertedResidual) BlockArgsEfficientNetBuilderdecode_arch_defefficientnet_init_weightsround_channelsresolve_act_layer)feature_take_indices)register_notrace_module)checkpoint_seq checkpoint)generate_default_cfgsregister_model MobileNetV5MobileNetV5EncodercseZdZdZ      ddeeeefdeded ed ed e ed e d e e de e ffdd Z dee jde jfddZZS)"MobileNetV5MultiScaleFusionAdapteraMulti-layer fusion token adapter. Args: in_chs: List of input channel counts for each feature scale. out_chs: The number of output channels. output_resolution: The output resolution. expansion_ratio: The FFN expansion ratio. interpolation_mode: The upsampling interpolation mode. layer_scale_init_value: The initial value of the layer scale, no layer scale if None. @nearestNTin_chsout_chsoutput_resolutionexpansion_ratiointerpolation_modelayer_scale_init_valuenoskip act_layer norm_layerc stt|trt|n||_||_t||_||_ ||_ ||_ ||_ |p)t j}| p-t} t|j|jd|j || |j |j d|_| |j|_dS)Nr)r*r+dw_kernel_size_mid exp_ratior1r2r0r/)super__init__ isinstancersum in_channels out_channelsrr,r-r.r/r0nnGELUrrffnnorm) selfr*r+r,r-r.r/r0r1r2 __class__K/home/ubuntu/.local/lib/python3.10/site-packages/timm/models/mobilenetv5.pyr6&s*    z+MobileNetV5MultiScaleFusionAdapter.__init__inputsreturnc Cs8|djdd}g}t|D])\}}|jdd}|d|dks*|d|dkr3tj|||jd}||qtj|dd}||}|d|j dksW|d|j dkr|d|j ddksm|d|j ddkrwtj||j dd}n|d|j d}|d|j d} tj ||| f|| fd}| |}|S)Nrr)sizemode)dimbilinear) kernel_sizestride) shape enumerateF interpolater.appendtorchcatr=r, avg_pool2dr>) r?rDhigh_resolutionresized_inputs_img feat_sizechannel_cat_imgs h_strides w_stridesrBrBrCforwardJs,   $ z*MobileNetV5MultiScaleFusionAdapter.forward)r(r)NTNN)__name__ __module__ __qualname____doc__rintrfloatstrrboolrr6rRTensorr] __classcell__rBrBr@rCr's8  "$r'c)seZdZdZdddddddd dd d d d ded d d d fd edededededededede edede e de e de e de e dede de de de e d ef(fd!d" Zd#d$ZejjdFd%efd&d'ZejjdGd(efd)d*Zejjd+ejfd,d-ZdHded efd.d/Z   0  dId1ejd2e eee efd3ed4ed5ed6ed7ed+ee ejeeje ejfffd8d9Z :   dJd2eee efd;edZd1ejd+ejfd?d@ZdFd1ejdAed+ejfdBdCZd1ejd+ejfdDdEZ Z!S)Kr%z MobiletNet-V5 FT)rFNavg block_args num_classesin_chans stem_sizefix_stem num_featurespad_typeuse_msfa msfa_indicesmsfa_output_resolutionr1r2aa_layerse_layer se_from_exp round_chs_fn drop_ratedrop_path_rater/ global_poolc st| p tj} | p t} t| | }|pt}|_|_d_ | _ | _ |s,||}t ||dd|| | d_ td|||| | | |||d }tj|||_|j_ddjD_|j_|r|__ttjj d _ tfd dj D_tj|j | | d _t|d _d _d _ n(|j_|_d _t|d _jj!}t"|jd|d_|j_ |rt#dnt$_%|d krt&j|nt$_'t(d S)a Args: block_args: Arguments for blocks of the network. num_classes: Number of classes for classification head. in_chans: Number of input image channels. stem_size: Number of output channels of the initial stem convolution. fix_stem: If True, don't scale stem by round_chs_fn. num_features: Number of output channels of the conv head layer. head_bias: If True, add a learnable bias to the conv head layer. pad_type: Type of padding to use for convolution layers. act_layer: Type of activation layer. norm_layer: Type of normalization layer. aa_layer: Type of anti-aliasing layer. se_layer: Type of Squeeze-and-Excite layer. se_from_exp: If True, calculate SE channel reduction from expanded mid channels. round_chs_fn: Callable to round number of filters based on depth multiplier. drop_rate: Dropout rate. drop_path_rate: Stochastic depth rate. layer_scale_init_value: Enable layer scale on compatible blocks if not None. global_pool: Type of pooling to use for global pooling features of the FC head. FrirKrLpaddingr2r1 output_striderwr~r}r1r2r{r|rr/cSg|]}|dqSstagerB.0frBrBrC z(MobileNetV5.__init__..rcg|] }j|dqSnum_chs feature_informir?rBrCrr*r+r,r2r1 pool_typeNr)r))r5r6r;r<rrrrrrgrad_checkpointingryrzr conv_stemr Sequentialblocksfeaturesr stage_endsr*rvhead_hidden_sizerlenr8 msfa_in_chsr'msfar r conv_head norm_head feat_multrFlattenIdentityflattenr classifierr)r?rqrrrsrtrurvrwrxryrzr1r2r{r|r}r~rrr/rnorm_act_layerbuildernum_pooled_chsr@rrCr6psz ,         zMobileNetV5.__init__cCsx|j|jg}||j||j|jdur||j|jdur(||j|t t |j |j gtj |SN)rbn1extendrrQrrrr;rDropoutrrr)r?layersrBrBrC as_sequentials        zMobileNetV5.as_sequentialcoarsecCstd|rddSddS)Nz^conv_stem|bn1z^blocks\.(\d+)z^blocks\.(\d+)\.(\d+))stemr)dict)r?rrBrBrC group_matchers zMobileNetV5.group_matcherenablecCs ||_dSr)r)r?rrBrBrCset_grad_checkpointings z"MobileNetV5.set_grad_checkpointingrEcCs|jSr)rrrBrBrCget_classifierszMobileNetV5.get_classifiercCsR||_t|d|_|rtdnt|_|dkr"t|j||_ dSt|_ dS)Nrrr) rrr rr;rrrr rr)r?rrrrBrBrCreset_classifiers (zMobileNetV5.reset_classifierNCHWxindicesr> stop_early output_fmtintermediates_only extra_blockscs|dvsJd|r|sJdg}|r!ttjd|\} } nttj|\} } fdd| D} j| } d} |}| | vrI||tjsP|sTj} njd| } | D]} | d7} | |}| | vrp||q]|ru|S||fS) aa Forward features that returns intermediates. Args: x: Input image tensor indices: Take last n blocks if int, all if None, select matching indices if sequence norm: Apply norm layer to compatible intermediates stop_early: Stop iterating over blocks when last desired intermediate hit output_fmt: Shape of intermediate feature outputs intermediates_only: Only return intermediate features extra_blocks: Include outputs of all blocks and head conv in output, does not align with feature_info Returns: rOutput shape must be NCHW./Must use intermediates_only for early stopping.rcg|]}j|qSrBrrirrBrCr&z5MobileNetV5.forward_intermediates..rN) rrrrrrQrRjit is_scripting)r?rrr>rrrr intermediates take_indices max_indexfeat_idxrblkrBrrCforward_intermediatess2     z!MobileNetV5.forward_intermediatesr prune_norm prune_headcCs|rtt|jd|\}}ntt|j|\}}|j|}|jd||_|t|jkr3d|_d|_|rAd|_d|_|dd|S)z@ Prune layers not required for specified intermediates. rNrrl)rrrrrrr)r?rrrrrrrBrBrCprune_intermediate_layersEs   z%MobileNetV5.prune_intermediate_layerscCs|jdur7d}g}||}||jvr|||jD]}|d7}||}||jvr/||q||}|S||}|jrNtjsNt |j|dd}|S||}|S)NrrT)r) rrryrQrrrRrrr!r?rrrrrBrBrCforward_features]s(          zMobileNetV5.forward_features pre_logitscCsl||}|jdur||}|jdur||}||}|jdkr-tj||j|jd}|r1|S||S)Nro)ptraining) rrrrrrOdropoutrr)r?rrrBrBrC forward_headts        zMobileNetV5.forward_headcCs||}||}|Sr)rrr?rrBrBrCr]s  zMobileNetV5.forwardF)T)rpNFFrFF)rFTF)"r^r_r`rarrrbrerdrrrrrcr6rrRrignorerrr;Modulerrrfrrrrrrr]rgrBrBr@rCr%ls     v   @  c"sDeZdZdZddddddddddd ed d dfd ed ed ededede edede e de e de e de e dede de de de e f fdd Z      d.dejde eeeefd ed!ed"ed#ed$ed%eeejeejeejfffd&d'Zdejd%ejfd(d)Zdejd%ejfd*d+Zdejd%ejfd,d-ZZS)/r&zMobileNetV5 Vision Encoderri@Frl)rFrnrjNTrorqrsrtrurwryrzr1r2r{r|r}r~rrr/c s t|p tj}| p t} | pt} d_|_d_|s!| |}t ||dd|| |d_ t d|| | || | | ||d }tj |||_ |j_dd jD_d __ttj|d_tfd d jD_|_tjjj| |d _tdS) NrFrirrrrcSrrrBrrBrBrCrrz/MobileNetV5Encoder.__init__..rkcrrrrrrBrCrrr)r5r6r;r<rrrrrrrrrrrrrrrvrrrryr8rrzr'rr)r?rqrsrtrurwryrzr1r2r{r|r}r~rrr/rr@rrCr6sZ      zMobileNetV5Encoder.__init__rrrr>rrrrrEcs&~|dvs Jd|r|sJdg}g} |r$ttjd|\} } nttj|\} } fdd| D} j| } d} |}| | vrL||| jvrV| |tj s]|saj} njd| } | D]}| d7} ||}| | vr}||| jvr| |qj|r|S | |fS) al Forward features that returns intermediates. Args: x: Input image tensor indices: Take last n blocks if int, all if None, select matching indices if sequence norm: (Unused) Applies norm layer to compatible intermediates stop_early: Stop iterating over blocks when last desired intermediate hit output_fmt: Shape of intermediate feature outputs intermediates_only: Only return intermediate features extra_blocks: Include outputs of all blocks and head conv in output, does not align with feature_info Returns: rrrrcrrBrrrrBrCrrz.rN) rrrrrrQryrRrrr)r?rrr>rrrrrmsfa_intermediatesrrrrrrBrrCrs>         z(MobileNetV5Encoder.forward_intermediatescCs`d}g}||}||jvr|||jD]}|d7}||}||jvr*||q||S)Nrr)rryrQrrrrBrBrCrs       z#MobileNetV5Encoder.forward_featurescCstd)Nz=MobileNetV5Encoder does not support classification use cases.)NotImplementedErrorrrBrBrCr(szMobileNetV5Encoder.forward_headcCs ||Sr)rrrBrBrCr]+s zMobileNetV5Encoder.forwardr)r^r_r`rarrrbrerdrrrrrcr6rRrfrrrrrrr]rgrBrBr@rCr&s     K EFvariant pretrainedrEcKs<|dd}t|dd}d}tt||fd||d|}|S)N out_indicesrrrrigetterr feature_cls)rrrv head_conv head_bias head_normrF)pretrained_strict feature_cfg kwargs_filter)poprrr&)rrkwargsrrrmodelrBrBrC_create_mnv5_encoder/s   rcKs6|dd}t|dd}tt||fd|d|}|S)NrrrrF)rr)rrrr%)rrrrrrrBrBrC _create_mnv5Fs  r?channel_multiplierencoderc Ksd|vrgdgdgdgdg}ngdgdgdgdg}tt||dd |d ktt|d ttjd d }t|fi|}|rLt||fi|}|St||fi|}|S)Nmobilenetv5_base)er_r1_k3_s2_e4_c128er_r1_k3_s1_e4_c128r)uir_r1_a3_k5_s2_e6_c256uir_r1_a5_k0_s1_e4_c256uir_r1_a3_k0_s1_e4_c256rr)uir_r1_a5_k5_s2_e6_c512uir_r1_a5_k0_s1_e4_c512ruir_r1_a0_k0_s1_e1_c512mqa_r1_k3_h8_s2_d64_c512uir_r1_a0_k0_s1_e2_c512rrrrrrrrrr)uir_r1_a5_k5_s2_e6_c1024mqa_r1_k3_h16_s1_d64_c1024uir_r1_a0_k0_s1_e2_c1024r r r r r r r r r r r r )%uir_r1_a5_k5_s2_e6_c640uir_r1_a5_k0_s1_e4_c640r r r r r r uir_r1_a0_k0_s1_e1_c640mqa_r1_k3_h12_v2_s1_d64_c640uir_r1_a0_k0_s1_e2_c640rrrrrrrrrrrrrrrrrrrrrrrrrr)'uir_r1_a5_k5_s2_e6_c1280mqa_r1_k3_h16_s1_d96_c1280uir_r1_a0_k0_s1_e2_c1280rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr) group_sizerr) multipliergh㈵>)rqrtrur~r2r1r/) rrrrrr;r<rr) rrrrrrarch_def model_kwargsrrBrBrC_gen_mobilenet_v5Ts48(c   rrlurlc Ks|dddddttddd |S) Nrh)rir)rjrjrbicubiczconv_stem.convr) rrr input_size pool_sizecrop_pct interpolationmeanstd first_convrr )rrrBrBrC_cfgsr")rir#)rrrrh)rr)mobilenetv5_300m_encmobilenetv5_300mzmobilenetv5_base.untrainedcKs(|dd}t d|d|d|}|S)zMobileNet V5 Vision Encoderrwsamer$T)rrrwN)r$)rr)rrrwrrBrBrCr$#s r$cKtdd|i|}|S)Nr%r)r%rrrrrBrBrCr%1r%cKr')Nrr)rr(r)rBrBrCr7r*rr)rNFF)rl)B functoolsrtypingrrrrrrrRtorch.nnr;torch.nn.functional functionalrO timm.datar r timm.layersr r rrrrrrr_builderr_efficientnet_blocksrr_efficientnet_builderrrrrrr _featuresr _features_fxr _manipulater!r" _registryr#r$__all__rr'r%r&rdrerrrcrr" default_cfgsr$r%rrBrBrBrCsx  ,    R) 4