o پi*M@s< dZddlmZmZmZddlmZmZddlm Z ddl m Z m Z ddl mZmZmZmZgZeeedd d ddd d edd d d dd d edd dd dd dedd dd dd d fdddddedeedd d ddd d edd gdd d dd dedd gddd dd dedd dd dddfddddd d!ded" eeddd ddd d edd#d#d d dd dedd$dd dd dedddd dd dfdddddded%eedd d dd&d d edd d d d&d d edd dd d&d dedd dd d&d d fdddddd'ded&d(d) eedddddd*d eddd+d dd*d edd,d d dd*d eddd d dd*d fdd-dd.ed/dd0d1eedd d ddd d edd d d dd d edd dd dd dedd dd dd d fdddd.ed/d d0d1eedd d ddd d edd gdd d dd dedd gddd dd dedd dd dddfddddd d!d.ed/dd0d" eeddd ddd d edd#d#d d dd d.ed/dd#d2d3 edd$dd dd dedddd dd dfddddd.ed/dd0d4eedd d dd&d d edd d d d&d d edd dd d&d dedd dd d&d d fddddd'd.ed/d d&d5d6eedd d ddd d edd d d dd d edd dd dd dedd dd dd d fdddd7ed8d9d1eeddd ddd d edd#d#d d dd dedd$dd dd dedddd dd dfddddd7ed8d9d4eedd d ddd d edd d d dd d edd dd dd dedd dd dd d fdddd7ed:d9d1eedd d;d d&dd edd$dd d&ddedd d?d d&ddfd@ddddd!ed dAedddBd.edCd dDdEdF eeddd ddd d edd#d d dd d7edGd9dHedd$dd dd d.eddIdHedddd dd dedHfdddddJeeddd ddd d edd#d d dd d.eddIdHedd$dd dd d.eddIdHedddd dd dedHfdddddJdKZddMedNeedOedPefdQdRZddSedPeeeffdTdUZe edVdWddXdYdZed[dWddXdYd\d]eddXdYd^ed_dWddXdYdZedXdYdXd`edadWdXdYdXdbedcdWdXdYdXd\ddededWdXdYdXd\ddedfdWdXdYdXd\ddedgdWdhdXdYd\diedjdWdhdXdYdkedldWddXdYd\d]edmdWdndndodpdqdpd\dr edsdWddXdYdZedtdWddXdYdZduZe ddOedPefdvdwZe ddOedPefdxdyZe ddOedPefdzd{Ze ddOedPefd|d}Ze ddOedPefd~dZe ddOedPefddZe ddOedPefddZ e ddOedPefddZ!e ddOedPefddZ"e ddOedPefddZ#e ddOedPefddZ$e ddOedPefddZ%e ddOedPefddZ&e ddOedPefddZ'e ddOedPefddZ(d:S)a Bring-Your-Own-Attention Network A flexible network w/ dataclass based config for stacking NN blocks including self-attention (or similar) layers. Currently used to implement experimental variants of: * Bottleneck Transformers * Lambda ResNets * HaloNets Consider all of the models definitions here as experimental WIP and likely to change. Hacked together by / copyright Ross Wightman, 2021. )AnyDictOptionalIMAGENET_DEFAULT_MEANIMAGENET_DEFAULT_STD)build_model_with_cfg)register_modelgenerate_default_cfgs) ByoBlockCfg ByoModelCfgByobNetinterleave_blocksbottleg?)typedcsgsbri)r self_attni)typesrrrrrri@tieredmaxpoolT bottleneck)blocksstem_chs stem_type stem_poolfixed_input_sizeself_attn_layerself_attn_kwargs)reveryrrrrrigZd;O?)rrrrrsiluise) rr r!r" act_layer num_features attn_layerr$r%)rr r!r"r+r#r$r%eca)dim_head) rr r!r"r#r+r-r$r%g? 7x7halo) block_size halo_size)rr r!r"r$r%)r8r9 num_heads) rr'rrrrrr$r%)rr r!r"r+r$r%)r8r9r2)rr r!r"r+r-r$r%lambda )rN0` i  )rd_ratio) bottle_in linear_outgQ?)r8r9qk_ratio) rr r" downsampler,r+r- attn_kwargs block_kwargsr$r% )rrrrrrr$r%)r9)rr r!r"r+) botnet26t sebotnet33ts botnet50tseca_botnext26ts halonet_h1 halonet26t sehalonet33ts halonet50tseca_halonext26tslambda_resnet26tlambda_resnet50tslambda_resnet26rpt_256 haloregnetz_blamhalobotnet50tshalo2botnet50tsFvariant cfg_variant pretrainedreturncKs0tt||f|s t|nt|tddd|S)a<Create a Bring-Your-Own-Attention network model. Args: variant: Model variant name. cfg_variant: Config variant name if different from model variant. pretrained: Load pretrained weights. **kwargs: Additional model arguments. Returns: Instantiated ByobNet model. T)flatten_sequential) model_cfg feature_cfg)r r model_cfgsdict)r[r\r]kwargsreG/home/ubuntu/.local/lib/python3.10/site-packages/timm/models/byoanet.py_create_byoanet s rgurlc Ks"|dddddttddddd |S) zGenerate default model configuration. Args: url: URL for pretrained weights. **kwargs: Override default configuration values. Returns: Model configuration dictionary. ir&rjrFrFgffffff?bicubiczstem.conv1.convzhead.fcF) rh num_classes input_size pool_sizecrop_pct interpolationmeanstd first_conv classifierr#min_input_sizer)rhrdrererf_cfgs rwzshttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/botnet26t_c1_256-167a0e9f.pthztimm/)r&rr)r7r7)rh hf_hub_idr#rnrozxhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/sebotnet33ts_a1h2_256-957e3c3e.pthgGz?)rhrxr#rnrorp)r#rnrozxhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/eca_botnext26ts_c_256-95a898f6.pth)rnrorvzuhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/halonet26t_a1h_256-3083328c.pth)rhrxrnrorvzthttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/sehalonet33ts_256-87e053f9.pth)rhrxrnrorvrpzwhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/halonet50ts_a1h2_256-f3a3daee.pthzyhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/eca_halonext26ts_c_256-06906299.pthzyhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/lambda_resnet26t_c_256-e5a5c857.pth)r&r3r3)rhrxrvrnrorpz|https://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/lambda_resnet50ts_a1h_256-b87370f7.pth)rhrxrvrnroz{https://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/lambda_resnet26rpt_c_256-ab00292d.pthzxhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/haloregnetz_c_raa_256-c8ad7616.pth)?ryryz stem.convrirk) rhrxrrrsrtrnrorvrpz}https://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/lamhalobotnet50ts_a1h2_256-fe3d9445.pthz{https://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/halo2botnet50ts_a1h2_256-fd9c11a3.pth)zbotnet26t_256.c1_in1kzsebotnet33ts_256.a1h_in1kzbotnet50ts_256.untrainedzeca_botnext26ts_256.c1_in1kzhalonet_h1.untrainedzhalonet26t.a1h_in1kzsehalonet33ts.ra2_in1kzhalonet50ts.a1h_in1kzeca_halonext26ts.c1_in1kzlambda_resnet26t.c1_in1kzlambda_resnet50ts.a1h_in1kzlambda_resnet26rpt_256.c1_in1kzhaloregnetz_b.ra3_in1kzlamhalobotnet50ts_256.a1h_in1kzhalo2botnet50ts_256.a1h_in1kcK|ddtdd|i|S)z4 Bottleneck Transformer w/ ResNet26-T backbone. img_sizer botnet26t_256rLr]N)r|rL setdefaultrgr]rdrererfr|r r|cKtdd|i|S)zY Bottleneck Transformer w/ a ResNet33-t backbone, SE attn for non Halo blocks, SiLU, sebotnet33ts_256rMr]N)rrMrgrrererfrzrcKrz)z> Bottleneck Transformer w/ ResNet50-T backbone, silu act. r{rbotnet50ts_256rNr]N)rrNr}rrererfrrrcKrz)z> Bottleneck Transformer w/ ResNet26-T backbone, silu act. r{reca_botnext26ts_256rOr]N)rrOr}rrererfrrrcKtdd|i|S)za HaloNet-H1. Halo attention in all stages as per the paper. NOTE: This runs very slowly! rPr]N)rPrrrererfrPsrPcKr)zJ HaloNet w/ a ResNet26-t backbone. Halo attention in final two stages rQr]N)rQrrrererfrQrrQcKr)zc HaloNet w/ a ResNet33-t backbone, SE attn for non Halo blocks, SiLU, 1-2 Halo in stage 2,3,4. rRr]N)rRrrrererfrRrrRcKr)zT HaloNet w/ a ResNet50-t backbone, silu act. Halo attention in final two stages rSr]N)rSrrrererfrSrrScKr)zT HaloNet w/ a ResNet26-t backbone, silu act. Halo attention in final two stages rTr]N)rTrrrererfrTrrTcKr)zG Lambda-ResNet-26-T. Lambda layers w/ conv pos in last two stages. rUr]N)rUrrrererfrUrrUcKr)zR Lambda-ResNet-50-TS. SiLU act. Lambda layers w/ conv pos in last two stages. rVr]N)rVrrrererfrVrrVcKs|ddtdd|i|S)zN Lambda-ResNet-26-R-T. Lambda layers w/ rel pos embed in last two stages. r{rrWr]N)rWr}rrererfrWrrWcKr)z Halo + RegNetZ rXr]N)rXrrrererfrXrrXcKr)z3 Combo Attention (Lambda + Halo + Bot) Network lamhalobotnet50ts_256rYr]N)rrYrrrererfrrrcKr)z1 Combo Attention (Halo + Halo + Bot) Network halo2botnet50ts_256rZr]N)rrZrrrererfrrr)NF)r()F))__doc__typingrrr timm.datarr_builderr _registryr r byobnetr r rr__all__rcrbstrboolrgrw default_cfgsr|rrrrPrQrRrSrTrUrVrWrXrrrerererfs                       q  A