o i@sddlmZddlZeddGdddZeddGdddeZeddGd d d eZeddGd d d eZeddGd ddeZeddGdddeZeddGdddeZ ej e eegdS)) dataclassNT)frozenc@eZdZdZdS) Granularityz Base class for representing the granularity of quantization. This class serves as a parent for specific granularity types used in quantization operations, such as per-tensor or per-axis quantization. N__name__ __module__ __qualname____doc__r r T/home/ubuntu/.local/lib/python3.10/site-packages/torchao/quantization/granularity.pyr rc@r) PerTensorz Represents per-tensor granularity in quantization. This granularity type calculates the quantization parameters based off the entire tensor. Nrr r r r rr rc@eZdZUdZeed<dS)PerAxisa Represents per-axis granularity in quantization. This granularity type calculates different quantization parameters along a specified axis of the tensor. Examples: * input_tensor shape [A, B], axis 0 -> scale_shape [A, 1] * input_tensor shape [A, B], axis 1 -> scale_shape [1, B] * input_tensor shape [A, B, C], axis 1 -> scale_shape [1, B, 1] Attributes: axis (int): The axis which is kept, reduction is performed across all the other axes axisNrrr r int__annotations__r r r r r$  rc@r)PerGroupa Represents per-channel group granularity in quantization. This granularity type calculates different quantization parameters for each group of elements. For example if the input tensor is shape [8, 16], and the group size is 4, then the input tensor is reshaped to [64, 4] quantization parameters are calculated for each group of 4 elements, giving a total of 64 quantization parameters. Attributes: group_size (int): The size of each quantization group group_sizeNrr r r r r9rrc@seZdZUdZdZeed<dS)PerRowa Represents row-wise granularity in quantization. Examples: * input_tensor shape [A, B], dim 0 -> scale_shape [1, B] * input_tensor shape [A, B], dim 1 -> scale_shape [A, 1] * input_tensor shape [A, B], dim -1 -> scale_shape [A, 1] * input_tensor shape [A, B, C], dim 1 -> scale_shape [A, 1, C] Attributes: dim (int): The dim which is reduced across, all other dims are kept dimN)rrr r rrrr r r r rNs  rc@r)PerTokena: Represents per-token granularity in quantization. This granularity type calculates a different set of quantization parameters for each token, which is represented as the last dimension of the tensor. For example, if the input tensor has shape [2, 3, 4], then there are 6 tokens with 4 elements each, and we will calculate 6 sets of quantization parameters, one for each token. If the input tensor has only two dimensions, e.g. [8, 16], then this is equivalent to `PerAxis(axis=0)`, which yields 8 sets of quantization parameters. Nrr r r r r`src@s"eZdZUdZeedfed<dS)PerBlocka' Represents multidimensional per-block granularity in quantization. Example: * block_size has shape [X, Y] * input_tensor shape [A] -> scaling undefined * input_tensor shape [A, B] -> scale shape [A // X, B // Y] * input_tensor shape [A, B, C] -> scale shape [A, B // X, C // Y] * input_tensor shape [A, B, C, D] -> scale shape [A, B, C // X, D // Y], and so on Note that `PerBlock((1, Y))` is equivalent to `PerGroup(Y)` Attributes: block_size (tuple[int, ...]): The size of each quantization group . block_sizeN)rrr r tuplerrr r r r rss r) dataclassesrtorchrrrrrrr serializationadd_safe_globalsr r r r s"