o iU @s2ddlmZmZmZmZmZddlZddlmZdeej de deeee ej fdffd d Z deeej dfeej fde deej fd d Z deeej dfeej fde dej fd dZGdddejjZGdddejjZdej de deej dffddZdeej de dej fddZdS))ListOptionalSequenceTupleUnionN)_get_storage_basetensorsdimreturn.cCs,t|dks ||djkrdSg}t|djdD](}||kr0||d|dq||kr8|d8}||d|qd}t|ddD]D\}}|j|djkr\dS||dkridS||d|d||kr~dS|durt|d}t||krdSqMt |S)z If the tensors are already stacked on dimension :code:`dim`, returns the strides of the stacked tensors. Otherwise returns :code:`None`. rrN) lenndimrangeappendstorage_offsetstride enumerateshapertuple)r r final_strideistorage_data_ptrxrG/home/ubuntu/.local/lib/python3.10/site-packages/xformers/ops/unbind.pyget_stack_strides s8  rcCsDt||}|dur t|dj}||t||d||SdS)Nr)rlistrinsertr as_strided)r r strides input_shaperrr_stack_or_none_fw;s r!cCs$t||}|durtj||d}|SN)r )r!torchstack)r r outrrr _stack_fwGs r&c@s<eZdZdZedejdefddZe dejfddZ d S) _Unbindz See function `unbind` rr cCs||_||SN)r unbind)ctxrr rrrforwardVs z_Unbind.forwardr cGst||jdfSr()r&r )clsr*r rrrbackward\sz_Unbind.backwardN) __name__ __module__ __qualname____doc__ staticmethodr#Tensorintr+ classmethodr-rrrrr'Q r'c@s<eZdZdZededejfddZe dejfddZ d S) _StackOrNonez& See function `stack_or_none` r r cGs||_t||dSr")r r!)r*r r rrrr+gs z_StackOrNone.forwardgradcCsdg|j|jdRSr")r)r )r,r*r8rrrr-msz_StackOrNone.backwardN) r.r/r0r1r2r4r#r3r+r5r-rrrrr7br6r7rcCs t||S)z Does exactly the same as :attr:`torch.unbind` for the forward. In backward, avoids a :attr:`torch.cat` if the gradients are already multiple views of the same storage )r'apply)rr rrrr)ss r)cCstj|g|RS)z Does exactly the same as :attr:`torch.stack` if the tensors can be concatenated without any memory operation. Otherwise returns None. )r7r9)r r rrr stack_or_none|sr:)typingrrrrrr#commonrr3r4SymIntrr!r&autogradFunctionr'r7r)r:rrrrs8  .    ""