o i @sddlmZddlZddlmZmZmZejeejeedgdZ Gdddej j Z Gdd d ej j Z ejj dd ejd ededefddZejje jfd ejdede defddZdS))EnumN)SparseSemiStructuredTensor$SparseSemiStructuredTensorCUSPARSELT!SparseSemiStructuredTensorCUTLASS GRADIENT_TYPE)DENSESPARSESTEc@s<eZdZedejdedefddZedejfddZ d S) _SparsifyFuncxalgobackendc Csb|dk}t|ts+tj|||d\}}}}} |rtnt} | |j||||| ddd} | S|} | S)Ncutlass) algorithm use_cutlassFT)packedmetapacked_tmeta_tcompressed_swizzled_bitmask requires_gradfuse_transpose_cusparselt) isinstancertorch_sparse_semi_structured_tilerrshapedetach) ctxr r r rrrrrbitmaskclsoutr!V/home/ubuntu/.local/lib/python3.10/site-packages/torchao/sparsity/training/autograd.pyforwards.   z_SparsifyFunc.forwardgrad_outcCs |ddfS)Nr!)rr$r!r!r"backward4s z_SparsifyFunc.backwardN) __name__ __module__ __qualname__ staticmethodrTensorstrrr#r%r!r!r!r"r s r c@s>eZdZeejfdejdefddZ edejfddZ dS) _SparsifyLikeFuncr patternc Cst|tsJt|tstd|jstdt||j\}}|j|_|j |_ |j|_ ||_ |j |j ||j||j |j|jdS)NzC`sparsify_like(x, pattern)` is only implemented for CUTLASS backendzK`sparsify_like(x, pattern)` is not implemented when `bitmask` is transposedr)rrrNotImplementedErrorr is_contiguousr_sparse_semi_structured_applyrrrgradient __class__rr)rr r-r2rrr!r!r"r#;s2  z_SparsifyLikeFunc.forwardr$c Cs|jtjks t|tr|dddfSt|trJ|j|jks J|jtjkr8|js-Jt ||j dddfS|jtj ks@Jt j ||j dd\}}}}t|j||j||j|j |jddddfS)Nr)r r.)r2rr rrdtyper threads_masksr0r#_sparse_semi_structured_apply_denserrrrrrrr)rr$r_rr!r!r"r%as<    z_SparsifyLikeFunc.backwardN) r&r'r(r)rrrr*rr#r%r!r!r!r"r,:s %r,rr r r returncCt|||S)zl Sparsifies a dense tensor into a semi-structured tensor, according to the algo and backend passed. )r apply)r r r r!r!r"semi_structured_sparsify r<r-r2cCr:)zj Sparsifies a dense tensor into a semi-structured tensor, using the mask of the provided pattern. )r,r;)r r-r2r!r!r"semi_structured_sparsify_liker=r>)r8r)enumrr torch.sparserrr_dynamoallow_in_graphrautogradFunctionr r,r*r+r<rr>r!r!r!r"s>   $N