o ߥiU@slddlmZddlZddlmZ ddejdejdeefddZdd e d e fd d Z Gdddej Z dS))OptionalNmaskdtypetgt_lencCs^|\}}|dur |n|}|ddddddf|d|||}||t|jS)z_ Expands attention_mask from `[bsz, seq_len]` to `[bsz, 1, tgt_seq_len, src_seq_len]`. N)sizeexpandto masked_fillbooltorchfinfomin)rrrbszsrc_len expanded_maskra/home/ubuntu/.local/lib/python3.10/site-packages/modelscope/models/multi_modal/ofa/utils/utils.py expand_masks   rF drop_probtrainingcCsZ|dks|s|Sd|}d|jddf}|tj||j|jd}||||}|S)a Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks). Args: x (`nn.Modules`): input nn layers. drop_prob (`float`): drop path ratio. training (`bool`): whether is training or inference. rr)rdevice)shaper randrrfloor_div)xrr keep_probr random_tensoroutputrrr drop_paths  r!cs8eZdZdZd fdd ZddZdefdd ZZS) DropPathz Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks). See more details about drop path from https://arxiv.org/pdf/1605.07648v4.pdf. Args: drop_prob: drop path ratio. Ncst||_dSN)super__init__r)selfr __class__rrr%4s  zDropPath.__init__cCst||j|jSr#)r!rr)r&rrrrforward8szDropPath.forwardreturncCs d|jS)Nzp={})formatr)r&rrr extra_repr;s zDropPath.extra_reprr#) __name__ __module__ __qualname____doc__r%r)strr, __classcell__rrr'rr"+s r"r#)rF)typingrr torch.nnnnTensorrintrfloatr r!Moduler"rrrrs