o پi= @sdZddlZddlmZddlmmZddlmZ      dd e d e d e d e d e de f ddZ     ddej d e d e d e d e d e f ddZGdddejZd d e de de fddZGdddejZdS)!a> DropBlock, DropPath PyTorch implementations of DropBlock and DropPath (Stochastic Depth) regularization layers. Papers: DropBlock: A regularization method for convolutional networks (https://arxiv.org/abs/1810.12890) Deep Networks with Stochastic Depth (https://arxiv.org/abs/1603.09382) Code: DropBlock impl inspired by two Tensorflow impl that I liked: - https://github.com/tensorflow/tpu/blob/master/models/official/resnet/resnet_model.py#L74 - https://github.com/clovaai/assembled-cnn/blob/master/nets/blocks.py Hacked together by / Copyright 2020 Ross Wightman N)ndgrid皙??F drop_prob block_size gamma_scale with_noiseinplace batchwisecCs|j\}}} } | | } t|t| | } ||| | d| |d| |d} ttj| |jdtj| |jd\}}|| dk|| | ddk@|| dk|| | ddk@@}t|dd| | fj|jd}|r|tj d|| | f|j|jd}nt |}d| ||dkj|jd}t j | | d| dd }|r|rtj d|| | f|j|jdnt|}|r|||d||S|||d|}|S||jtjdd|j}|r||||S|||}|S)a  DropBlock. See https://arxiv.org/pdf/1810.12890.pdf DropBlock with an experimental gaussian noise option. This layer has been tested on a few training runs with success, but needs further validation and possibly optimization for lower runtime impact. r)devicedtype)rr kernel_sizestridepaddinggHz>)shapeminrtorcharangerreshapetorrand rand_likeF max_pool2drandn randn_likemul_add_numelfloat32sumadd)xrrr r r r BCHW total_sizeclipped_block_sizegammaw_ih_i valid_block uniform_noise block_mask normal_noisenormalize_scaler6D/home/ubuntu/.local/lib/python3.10/site-packages/timm/layers/drop.py drop_block_2dsD& * ( r8r'cCs"|j\}}}} | |} t|t| |} ||| | d| |d||d} t|| } tj| |j| d| dd} |ret| }|rY| d|  || |S|d| || }|Sd| } | | jtj ddj|jd}|r| | ||S|| |}|S)z DropBlock. See https://arxiv.org/pdf/1810.12890.pdf DropBlock with an experimental gaussian noise option. Simplied from above without concern for valid block mask at edges. r rrrrgư>)rrr empty_like bernoulli_rrrrnormal_r!r"r#r$r%r&)r'rrr r r r(r)r*r+r,r-r.r3r4r5r6r6r7drop_block_fast_2dNs0  * r<csVeZdZdZ       ddeded ed ed ed ed effdd ZddZZ S) DropBlock2dz9 DropBlock. See https://arxiv.org/pdf/1810.12890.pdf rrrFTrrr r r r fastcs<tt|||_||_||_||_||_||_||_ dSN) superr=__init__rr rr r r r>)selfrrr r r r r> __class__r6r7rAys  zDropBlock2d.__init__cCsR|jr|js|S|jrt||j|j|j|j|jSt||j|j|j|j|j|j Sr?) trainingrr>r<rr r r r8r rBr'r6r6r7forwards zDropBlock2d.forward)rrrFFFT) __name__ __module__ __qualname____doc__floatintboolrArG __classcell__r6r6rCr7r=us2r=TrE scale_by_keepcCs`|dks|s|Sd|}|jdfd|jd}|||}|dkr,|r,||||S)a(Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks). This is the same as the DropConnect impl I created for EfficientNet, etc networks, however, the original name is misleading as 'Drop Connect' is a different form of dropout in a separate paper... See discussion: https://github.com/tensorflow/tpu/issues/494#issuecomment-532968956 ... I've opted for changing the layer and argument names to 'drop path' rather than mix DropConnect as a layer name and use 'survival rate' as the argument. rPrr)r)rndim new_emptyr:div_)r'rrErQ keep_probr random_tensorr6r6r7 drop_paths   rWcs<eZdZdZd dedeffdd Zdd Zd d ZZ S) DropPathz^Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks). rPTrrQcstt|||_||_dSr?)r@rXrArrQ)rBrrQrCr6r7rAs zDropPath.__init__cCst||j|j|jSr?)rWrrErQrFr6r6r7rGzDropPath.forwardcCsdt|jddS)Nz drop_prob=z0.3f)roundr)rBr6r6r7 extra_reprrYzDropPath.extra_repr)rPT) rHrIrJrKrLrNrArGr\rOr6r6rCr7rXs rX)rrrFFF)rrrFF)rPFT)rKrtorch.nnnntorch.nn.functional functionalrgridrrLrMrNr8Tensorr<Moduler=rWrXr6r6r6r7sZ   8 '!