o i @sdddlZddlmZgdZddZddZGdd d eZ dd ejd ededejfddZ dS)N)UniformQuantizationObserverBase)create_block_sparse_tensorcreate_semi_structured_tensorPerChannelNormObserver mask_creatorcCsp|dkr|dks Jdttj||||fd||d}tj||dd}tj||dd}||S)N?gz*sparsity should be a value between 0 and 1)dtyperdim)torch bernoullifullrepeat_interleaveto contiguouscuda)MN blocksizesparsityr ArJ/home/ubuntu/.local/lib/python3.10/site-packages/torchao/sparsity/utils.pyrsrcCsdtdd||df}tjjj|dd||tj }|t |||}||S)z This function returns a 1:2 sparse matrix of size (r, c). Note that this means this matrix will also be 2:4 and 4:8 sparse as well. r) num_classes) r randintrnn functionalone_hotreshaperrint32rand)rcr choice_indicesmask sparse_weightrrrrs  rcs2eZdZdZd fdd ZddZdd ZZS) rz` A custom observer that computes the L2 norm of each channel and stores it in a buffer. returnNc sJtjdtjtjdddttjjd|d|_| dt gdS)NF)r qscheme reduce_range quant_min quant_maxepsrnormr) super__init__r quint8per_channel_affinefinfofloat32r-averaging_constantregister_buffertensor)selfkwargs __class__rrr08s  zPerChannelNormObserver.__init__cCs|dkr|S|}ddt|D}|d|d|d<|d<||}tj|dd}tjj|ddd}|j dkrP|j |j |j ||S|j |7_ |S) NrcSsg|]}|qSrr).0irrr Osz2PerChannelNormObserver.forward..r) start_dimr r) numeldetachranger permuter flattenlinalg vector_normr.resize_shapecopy_)r8x_origx new_axis_listyr.rrrforwardIs   zPerChannelNormObserver.forwardcCstd)Nz>PerChannelNormObserver is designed to store activations only. )NotImplementedError)r8rrrcalculate_qparams_sz(PerChannelNormObserver.calculate_qparams)r(N)__name__ __module__ __qualname____doc__r0rOrQ __classcell__rrr:rr3s rrr7rrr(cCsd}||dkrtd|jd|d||}|||}tj|dddddt||f}tj |j|j d}|j d|dd |j}|S) a Class for creating N:M sparsity masks. Masks will be created using the N:M ratio, where for every block of M weights, N will be pruned based on ranked weight value. Each mask will correspond to the given tensor. :param tensor: The input tensor to create a mask for :param N: The number of weights in a group to keep :param M: The size of a weight group :return: A mask tensor with the same shape as the input tensor NrzTensor of size z can't be evenly divided into z groupsrr )device)r indexvalue) rA ValueErrorrIrBabsr r argsortintonesrXscatter_)r7rrr& num_groups tensor_temprYw_brrrres &r)rrW) r torch.ao.quantization.observerr__all__rrrTensorr^rrrrrs"  4