o پiM@sLdZddlmZddlmZddlmZmZmZm Z m Z m Z ddl Z ddl mZddlmZmZddlmZmZmZmZmZmZdd lmZdd lmZdd lmZdd lm Z m!Z!dd l"m#Z#m$Z$dgZ%eeej&dZ'Gdddej(Z)      dQde*de*de+de+de*de+dee e+e*e+e*ffddZ,   ! dRd"ee e+e*e+e*fd#e+de*de+d$e+d%e-d&e-d'e*de eej(eee-efffd(d)Z.Gd*ddej(Z/d+e-d,e0de/fd-d.Z1dSd0e-dee-effd1d2Z2e#e2d3d4e2d3d4e2d3d4e2d3d4e2d3d4e2e2e2e2d3d5dd6d7d8e2d3d5dd6d7d8e2d3d9d5dd6d7d:e2d3d9d5dd6d7d:d; Z3e$dTd,e0de/fd=d>Z4e$dTd,e0de/fd?d@Z5e$dTd,e0de/fdAdBZ6e$dTd,e0de/fdCdDZ7e$dTd,e0de/fdEdFZ8e$dTd,e0de/fdGdHZ9e$dTd,e0de/fdIdJZ:e$dTd,e0de/fdKdLZ;e$dTd,e0de/fdMdNZsz_block_cfg..cs(g|]\}}|gdg|dqS)rrG)rfidxrg)layersrGrHrhs(rrNr1r r.r!rd)sum enumeraterangeappendrr;listzip) r`rarbrcr)r*strides exp_ratiosdepthbase_chs out_chs_listi se_ratiosrG)rarjrH _block_cfgs$,ry r"r# block_cfgprev_chs output_strider+r,drop_path_ratecCsH|g}g} d} d} g} t|} t|D]g\}\}}}}| }|dkrD|dkr'dnd|d}| t|d| |dg7} | |krD| |}d}||| d}|dkrTt|nd }| t||||| |f|||||d | |9} |} |}|| dg7}qtd ||d }| t|d| dt| ddg7} | t|||d | | fS)aBuild ReXNet blocks from configuration. Args: block_cfg: Block configuration list. prev_chs: Previous channel count. width_mult: Width multiplier. ch_div: Channel divisor. output_stride: Target output stride. act_layer: Activation layer name. dw_act_layer: Depthwise activation layer name. drop_path_rate: Drop path rate. Returns: Tuple of (features list, feature_info list). rdrrstemz features.)num_chs reductionmoduler!N) r$r%r(r&r'r)r*r+r,r-ir.r0) lenrmdictrrorrKrr)r{r|r`r*r}r+r,r~feat_chs feature_info curr_strider'features num_blocks block_idxchsr(r&r) next_dilationfname block_dprr-pen_chsrGrGrH _build_blockssH &rcseZdZdZ         dAdedededededededededededededeffdd Zej j dBd e d!e ee ffd"d#Zej j dCd%e d!d&fd'd(Zej j d!ejfd)d*ZdDdedeed!d&fd+d,Z &   - dEd.ejd/eeeeefd0e d1e d2ed3e d!eeejeejeejfffd4d5Z  $dFd/eeeefd6e d7e d!eefd8d9Zd.ejd!ejfd:d;ZdBd.ejdZd.ejd!ejfd?d@ZZS)GrzReXNet model architecture. Based on `ReXNet: Diminishing Representational Bottleneck on Convolutional Neural Network` - https://arxiv.org/abs/2007.00992 r1avgrzr^r_r UUUUUU?rr"r#皙?r!in_chans num_classes global_poolr}rbrcr`rar)r*r+r, drop_rater~c stt|||_| |_d|_|dvsJ|dkrd|nd}tt||| d}t||dd| d|_ t ||||| | }t |||| || | |\}|_ |d j |_|_tj||_t|j||| |_t|d S) aInitialize ReXNet. Args: in_chans: Number of input channels. num_classes: Number of classes for classification. global_pool: Global pooling type. output_stride: Output stride. initial_chs: Initial channel count. final_chs: Final channel count. width_mult: Width multiplier. depth_mult: Depth multiplier. se_ratio: Squeeze-excitation ratio. ch_div: Channel divisor. act_layer: Activation layer name. dw_act_layer: Depthwise activation layer name. drop_rate: Dropout rate. drop_path_rate: Drop path rate. F)rzr^r rzr.r1rd)r&r+rN)r6rr7rrgrad_checkpointingrr;rrryrrr: num_featureshead_hidden_sizerY Sequentialrrheadr)rCrrrr}rbrcr`rar)r*r+r,rr~ stem_base_chsstem_chsr{rrErGrHr7s.#    zRexNet.__init__FcoarserJcCstddd}|S)zGroup matcher for parameter groups. Args: coarse: Whether to use coarse grouping. Returns: Dictionary of grouped parameters. z^stemz^features\.(\d+))rblocks)r)rCrmatcherrGrGrH group_matcher1s  zRexNet.group_matcherTenableNcCs ||_dS)zEnable or disable gradient checkpointing. Args: enable: Whether to enable gradient checkpointing. N)r)rCrrGrGrHset_grad_checkpointingAs zRexNet.set_grad_checkpointingcCs|jjS)zTGet the classifier module. Returns: Classifier module. )rfc)rCrGrGrHget_classifierJszRexNet.get_classifiercCs||_|j||dS)zReset the classifier. Args: num_classes: Number of classes for new classifier. global_pool: Global pooling type. N)rrreset)rCrrrGrGrHreset_classifierSszRexNet.reset_classifierNCHWrLindicesnorm stop_early output_fmtintermediates_onlyc s|dvsJdg}dd|jDtt|\}} fdd|D}| } ||}tjs4|s8|j} n |jd| d} t| D]\} } |j rWtjsWt | |}n| |}| |vrd| |qE|ri|S||fS)a Forward features that returns intermediates. Args: x: Input image tensor indices: Take last n blocks if int, all if None, select matching indices if sequence norm: Apply norm layer to compatible intermediates stop_early: Stop iterating over blocks when last desired intermediate hit output_fmt: Shape of intermediate feature outputs intermediates_only: Only return intermediate features Returns: )rzOutput shape must be NCHW.cS"g|] }t|dddqSr.rr?splitrfinforGrGrHrht"z0RexNet.forward_intermediates..csg|]}|qSrGrG)rfrw stage_endsrGrHrhvsNr) rrrrrNjit is_scriptingrrmrrro) rCrLrrrrr intermediates take_indices max_indexstagesfeat_idxstagerGrrHforward_intermediates]s(   zRexNet.forward_intermediates prune_norm prune_headcCsRdd|jD}tt||\}}||}|jd|d|_|r'|dd|S)aEPrune layers not required for specified intermediates. Args: indices: Indices of intermediate layers to keep. prune_norm: Whether to prune normalization layer. prune_head: Whether to prune the classifier head. Returns: List of indices that were kept. cSrrrrrGrGrHrhrz4RexNet.prune_intermediate_layers..Nrr)rrrrr)rCrrrrrrrGrGrHprune_intermediate_layerss z RexNet.prune_intermediate_layerscCs8||}|jrtjst|j|}|S||}|S)zForward pass through feature extraction layers. Args: x: Input tensor. Returns: Feature tensor. )rrrNrrrrrCrLrGrGrHforward_featuress   zRexNet.forward_features pre_logitscCs|r |j||dS||S)zForward pass through head. Args: x: Input features. pre_logits: Return features before final linear layer. Returns: Classification logits or features. )r)r)rCrLrrGrGrH forward_heads zRexNet.forward_headcCs||}||}|S)zoForward pass. Args: x: Input tensor. Returns: Output logits. )rrrrGrGrHrQs zRexNet.forward)r1rrrzr^r_r r rrr"r#rr!rR)T)N)NFFrF)rFT)rSrTrUrVr?rXrWr7rNrignorer[rrrrrYrZrrrr\r rr rrrrrQr]rGrGrErHrs     ?   2  variant pretrainedcKs"tdd}tt||fd|i|S)zCreate a ReXNet model. Args: variant: Model variant name. pretrained: Load pretrained weights. **kwargs: Additional model arguments. Returns: ReXNet model instance. T)flatten_sequential feature_cfg)rrr)rrkwargsrrGrGrH_create_rexnets rrurlc Ks |dddddttdddd |S) zCreate default configuration dictionary. Args: url: Model weight URL. **kwargs: Additional configuration options. Returns: Configuration dictionary. r)r1r)rg?bicubicz stem.convzhead.fcmit) rr input_size pool_sizecrop_pct interpolationmeanstd first_conv classifierlicenser )rrrGrGrH_cfgs rztimm/) hf_hub_idgffffff?)r1 rz apache-2.0)rr test_crop_pcttest_input_sizeri-.)rrrrrr) zrexnet_100.nav_in1kzrexnet_130.nav_in1kzrexnet_150.nav_in1kzrexnet_200.nav_in1kzrexnet_300.nav_in1kzrexnetr_100.untrainedzrexnetr_130.untrainedzrexnetr_150.untrainedzrexnetr_200.sw_in12k_ft_in1kzrexnetr_300.sw_in12k_ft_in1kzrexnetr_200.sw_in12kzrexnetr_300.sw_in12kFcKstd|fi|S)zReXNet V1 1.0x rexnet_100rrrrGrGrHrsrcKtd|fddi|S)zReXNet V1 1.3x rexnet_130r`?rrrGrGrHrrcKr)zReXNet V1 1.5x rexnet_150r`?rrrGrGrHrrrcKr)zReXNet V1 2.0x rexnet_200r`@rrrGrGrHr#rrcKr)zReXNet V1 3.0x rexnet_300r`@rrrGrGrHr)rrcKr)z*ReXNet V1 1.0x w/ rounded (mod 8) channels rexnetr_100r*rrrrGrGrHr/rrcKtd|fddd|S)z*ReXNet V1 1.3x w/ rounded (mod 8) channels rexnetr_130rrr`r*rrrGrGrHr5rcKr)z*ReXNet V1 1.5x w/ rounded (mod 8) channels rexnetr_150rrrrrrGrGrHr;rrcKr)z*ReXNet V1 2.0x w/ rounded (mod 8) channels rexnetr_200rrrrrrGrGrHrArrcKr)z+ReXNet V1 3.0x w/ rounded (mod 16) channels rexnetr_300rr^rrrrGrGrHrGrr)r r r^r_r!r)rrzr"r#r!)rrR)>rV functoolsrmathrtypingrrrrr r rNtorch.nnrY timm.datar r timm.layersrrrrrr_builderr_efficientnet_builderr _featuresr _manipulaterr _registryrr__all__ BatchNorm2dr>rZrrWr?ryrXrrr[rr default_cfgsrrrrrrrrrrrGrGrGrHs       c , ?e