o پi\G @sdZddlZddlmZddlZddlmZddlmmZ ddl m Z m Z ddl mZddlmZddlmZmZd gZd d ZGd d d ejZGdddejZGdddeZGdddeZGdddeZGdddejZGdd d ejZd?ddZd@ddZeedd ed!d"d#ed$d ed%d ed&d ed'd ed(d"d#ed)d ed*d d+ Z ed?d,efd-d.Z!ed?d,efd/d0Z"ed?d,efd1d2Z#ed?d,efd3d4Z$ed?d,efd5d6Z%ed?d,efd7d8Z&ed?d,efd9d:Z'ed?d,efd;d<Z(ed?d,efd=d>Z)dS)Aa: SEResNet implementation from Cadene's pretrained models https://github.com/Cadene/pretrained-models.pytorch/blob/master/pretrainedmodels/models/senet.py Additional credit to https://github.com/creafz Original model: https://github.com/hujie-frank/SENet ResNet code gently borrowed from https://github.com/pytorch/vision/blob/master/torchvision/models/resnet.py FIXME I'm deprecating this model and moving them to ResNet as I don't want to maintain duplicate support for extras like dilation, switchable BN/activations, feature extraction, etc that don't exist here. N) OrderedDictIMAGENET_DEFAULT_MEANIMAGENET_DEFAULT_STD)create_classifier)build_model_with_cfg)register_modelgenerate_default_cfgsSENetcCsXt|tjrtjj|jddddSt|tjr*tj|jdtj|jddSdS)Nfan_outrelu)mode nonlinearityg?) isinstancennConv2dinitkaiming_normal_weight BatchNorm2d constant_bias)mrE/home/ubuntu/.local/lib/python3.10/site-packages/timm/models/senet.py _weight_inits  rcs$eZdZfddZddZZS)SEModulecsVtt|tj|||dd|_tjdd|_tj|||dd|_t |_ dS)Nr) kernel_sizeTinplace) superr__init__rrfc1ReLUr fc2Sigmoidsigmoid)selfchannels reduction __class__rrr#'s zSEModule.__init__cCsB|}|jddd}||}||}||}||}||S)N)T)keepdim)meanr$r r&r()r)x module_inputrrrforward.s    zSEModule.forward)__name__ __module__ __qualname__r#r4 __classcell__rrr,rr%s rc@seZdZdZddZdS) BottleneckzH Base class for bottlenecks that implements `forward()` method. cCs|}||}||}||}||}||}||}||}||}|jdur4||}|||}||}|SN) conv1bn1r conv2bn2conv3bn3 downsample se_moduler)r2shortcutoutrrrr4=s           zBottleneck.forwardN)r5r6r7__doc__r4rrrrr98s r9c&eZdZdZdZdfdd ZZS) SEBottleneckz" Bottleneck for SENet154. rNc stt|tj||dddd|_t|d|_tj|d|dd|d|dd|_t|d|_ tj|d|dddd|_ t|d|_ tj dd |_ t|d|d |_||_||_dS) Nr.rFrrrIr/rstridepaddinggroupsrTr r+)r"rHr#rrr;rr<r=r>r?r@r%r rrBrArLr)inplanesplanesrNr+rLrAr,rrr#Zs zSEBottleneck.__init__rNr5r6r7rF expansionr#r8rrr,rrHTrHcrG)SEResNetBottleneckz ResNet bottleneck with a Squeeze-and-Excitation module. It follows Caffe implementation and uses `stride=stride` in `conv1` and not in `conv2` (the latter is used in the torchvision implementation of ResNet). rIrNcstt|tj||dd|d|_t||_tj||dd|dd|_t||_ tj||dddd|_ t|d|_ tj dd |_ t|d|d |_||_||_dS) NrFrrrLr/rrMrNrrIrJTr rO)r"rWr#rrr;rr<r=r>r?r@r%r rrBrArLrPr,rrr#rs   zSEResNetBottleneck.__init__rSrTrrr,rrWjsrWcrG)SEResNeXtBottleneckzI ResNeXt bottleneck type C with a Squeeze-and-Excitation module. rIrNc stt|t||d|}tj||dddd|_t||_ tj||d|d|dd|_ t||_ tj||dddd|_ t|d|_ tjd d |_t|d|d |_||_||_dS) N@rFrXr/rKrIrJTr rO)r"rZr#mathfloorrrr;rr<r=r>r?r@r%r rrBrArL) r)rQrRrNr+rLrA base_widthwidthr,rrr#s   zSEResNeXtBottleneck.__init__)rNrIrTrrr,rrZrVrZcs*eZdZdZdfdd ZddZZS) SEResNetBlockrNcstt|tj||dd|dd|_t||_tj||dd|dd|_t||_ tj dd|_ t ||d|_ ||_||_dS) Nr/rF)rrMrLrrYTr rO)r"r`r#rrr;rr<r=r>r%r rrBrArLrPr,rrr#s   zSEResNetBlock.__init__cCsp|}||}||}||}||}||}||}|jdur*||}|||}||}|Sr:)r;r<r r=r>rArBrCrrrr4s         zSEResNetBlock.forwardrS)r5r6r7rUr#r4r8rrr,rr`s r`cseZdZ   d fd d Z  d!d d Zejjd"d dZejjd#ddZ ejjde j fddZ d$de defddZddZd"defddZddZZS)%r 皙?r/r[Frravgc sptt|||_| |_||_|r`dtj|ddddddfdtdfd tj d d fd tjdddddddfd tdfdtj d d fdtjd|dddddfdt|fdtj d d fg } ndtj||dddddfdt|fd tj d d fg} t t | |_ tj ddd d|_t|dddg|_|j|d|d||ddd|_|jtd|jdddg7_|j|d|dd||| | d|_|jtd|jdddg7_|j|d|dd||| | d|_|jtd|jd d!dg7_|j|d"|dd||| | d|_|jtd"|jd#d$dg7_d"|j|_|_t|j|j| d%\|_|_|D]}t|q.d&S)'af Parameters ---------- block (nn.Module): Bottleneck class. - For SENet154: SEBottleneck - For SE-ResNet models: SEResNetBottleneck - For SE-ResNeXt models: SEResNeXtBottleneck layers (list of ints): Number of residual blocks for 4 layers of the network (layer1...layer4). groups (int): Number of groups for the 3x3 convolution in each bottleneck block. - For SENet154: 64 - For SE-ResNet models: 1 - For SE-ResNeXt models: 32 reduction (int): Reduction ratio for Squeeze-and-Excitation modules. - For all models: 16 dropout_p (float or None): Drop probability for the Dropout layer. If `None` the Dropout layer is not used. - For SENet154: 0.2 - For SE-ResNet models: None - For SE-ResNeXt models: None inplanes (int): Number of input channels for layer1. - For SENet154: 128 - For SE-ResNet models: 64 - For SE-ResNeXt models: 64 input_3x3 (bool): If `True`, use three 3x3 convolutions instead of a single 7x7 convolution in layer0. - For SENet154: True - For SE-ResNet models: False - For SE-ResNeXt models: False downsample_kernel_size (int): Kernel size for downsampling convolutions in layer2, layer3 and layer4. - For SENet154: 3 - For SE-ResNet models: 1 - For SE-ResNeXt models: 1 downsample_padding (int): Padding for downsampling convolutions in layer2, layer3 and layer4. - For SENet154: 1 - For SE-ResNet models: 0 - For SE-ResNeXt models: 0 num_classes (int): Number of outputs in `last_linear` layer. - For all models: 1000 r;r[r/r.rF)rLrMrr<relu1Tr r=r>relu2r?r@relu3rrLrMr)rL ceil_modelayer0)num_chsr+moduler)rRblocksrNr+downsample_kernel_sizedownsample_paddingrIlayer1)rRrmrLrNr+rnrolayer2layer3i layer4 pool_typeN)r"r r#rQ num_classes drop_raterrrr% Sequentialrrj MaxPool2dpool0dict feature_info _make_layerrprUrsrvrx num_featureshead_hidden_sizer global_pool last_linearmodulesr)r)blocklayersrNr+r|in_chansrQ input_3x3rnror{rlayer0_modulesrr,rrr#s/            zSENet.__init__c Csd} |dks|j||jkr'ttj|j||j|||ddt||j} ||j||||| g} ||j|_td|D] } | ||j|||q=tj| S)NrFrh)rQrUrr}rrrangeappend) r)rrRrmrNr+rLrnrorArirrrr5s  zSENet._make_layercCstd|rdndd}|S)Nz^layer0z ^layer(\d+)z^layer(\d+)\.(\d+))stemrm)r)r)coarsematcherrrr group_matcherGszSENet.group_matcherTcCs|rJddS)Nz$gradient checkpointing not supportedr)r)enablerrrset_grad_checkpointingLszSENet.set_grad_checkpointingreturncCs|jSr:)r)r)rrrget_classifierPszSENet.get_classifierr{rcCs$||_t|j|j|d\|_|_dS)Nry)r{rrrr)r)r{rrrrreset_classifierTs zSENet.reset_classifiercCs@||}||}||}||}||}||}|Sr:)rjrrprsrvrxr)r2rrrforward_featuresYs      zSENet.forward_features pre_logitscCs:||}|jdkrtj||j|jd}|r|S||S)Nr)ptraining)rr|Fdropoutrr)r)r2rrrr forward_headbs  zSENet.forward_headcCs||}||}|Sr:)rrrrrrr4hs  z SENet.forward)rar/r[Frrrbrc)rrrF)T)rc)r5r6r7r#rtorchjitignorerrrModulerintstrrrboolrr4r8rrr,rr s$|    FcKstt||fi|Sr:)rr )variant pretrainedkwargsrrr _create_senetnsrc Ks|dddddttddd |S) Nrb)r/r)rgrgg?bilinearz layer0.conv1r) urlr{ input_size pool_sizecrop_pct interpolationr1std first_conv classifierr)rrrrr_cfgrsrzmhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-weights/legacy_senet154-e9eb9fe6.pth)rzhhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-weights/seresnet18-4bb0ce65.pthbicubic)rrzhhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-weights/seresnet34-a4004e63.pthzhhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-cadene/se_resnet50-ce0d4300.pthzihttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-cadene/se_resnet101-7e38fcc6.pthzihttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-cadene/se_resnet152-d17c99b7.pthzphttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-weights/seresnext26_32x4d-65ebdb501.pthzwhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-weights/legacy_se_resnext50_32x4d-f3651bad.pthzxhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-weights/legacy_se_resnext101_32x4d-37725eac.pth) zlegacy_senet154.in1kzlegacy_seresnet18.in1kzlegacy_seresnet34.in1kzlegacy_seresnet50.in1kzlegacy_seresnet101.in1kzlegacy_seresnet152.in1kzlegacy_seresnext26_32x4d.in1kzlegacy_seresnext50_32x4d.in1kzlegacy_seresnext101_32x4d.in1krcK2ttgdddd}td|fit|fi|S)Nr.r.r.r.rrurrrNr+legacy_seresnet18rr`rrr model_argsrrrr rcKr)Nr/rIr/rrurlegacy_seresnet34rrrrrrrrcKr)Nrrrurlegacy_seresnet50rrWrrrrrrrrcKr)Nr/rIr/rrurlegacy_seresnet101rrrrrrrrcKr)Nr/rr$r/rrurlegacy_seresnet152rrrrrrrrc Ks:ttgdddddddd}td |fit|fi|S) Nrr[rur/rrqT)rrrNr+rnrorQrlegacy_senet154)rrHrrrrrrs  rcKr)Nrrwrurlegacy_seresnext26_32x4drrZrrrrrrrrcKr)Nrrwrurlegacy_seresnext50_32x4drrrrrrrrcKr)Nrrwrurlegacy_seresnext101_32x4drrrrrrrrr)r)*rFr\ collectionsrrtorch.nnrtorch.nn.functional functionalr timm.datarr timm.layersr_builderr _registryr r __all__rrrr9rHrWrZr`r rr default_cfgsrrrrrrrrrrrrrs    " 8