o ܹi-@s,dZddlZGdddZGdddZdS)aN Helper classes for working with low precision floating point types that align with the opencompute (OCP) microscaling (MX) specification. * MXFP4Tensor: 4-bit E2M1 floating point data * MXScaleTensor: 8-bit E8M0 floating point data Reference: https://www.opencompute.org/documents/ocp-microscaling-formats-mx-v1-0-spec-final-pdf Nc@s>eZdZdddZddZddZdd Zd d Zd d ZdS) MXFP4TensorNcCj||_|durt|tjsJd|j|_|||_dS|dur1t|tr+||_dS|f|_dStd)at Tensor class for working with four bit E2M1 floating point data as defined by the opencompute microscaling specification. Parameters: - data: A torch tensor of float32 numbers to convert to fp4e2m1 microscaling format. - size: The size of the tensor to create. - device: The device on which to create the tensor. N%Parameter data must be a torch tensor.Either parameter data or size must be provided device isinstancetorchTensor _from_floatdatatuplesize ValueErrorselfr rrrN/home/ubuntu/veenaModal/venv/lib/python3.10/site-packages/triton/tools/mxfp.py__init__s  zMXFP4Tensor.__init__cCsptjdd|jtj|jd}tjdd|jtj|jd}tjdd|jtj|jd}|d>|d>B|Btj|_|S)Nrrdtyper)r randintruint8rtyper )rSEMrrrrandom#s zMXFP4Tensor.randomcCs |tjks Jd|j}|d?d@|}|d?d@|}|d@|}t|}|dk|dk@}|}|rs||} ||} ||} td| } t| dk| | d} t| dk| dd| d}| td| |}|||<|||dk@d9<|tjS) z Convert fp4e2m1 data to float32. Returns: - A torch tensor of type dtype representing the fp4e2m1 data. zCCurrently only float32 is supported for fp4e2m1 to float conversionrrr??r)r float32r r zeros_likeanypowwhere)rrr rrr valueis_zero non_zero_maskS_nzE_nzM_nzsignexponentmantissavalue_nzrrrto+s&   zMXFP4Tensor.tocCsTt|tj}t|}|dk}t|t|B}tjgdtj|jd}tjddgtj|jd}g}g} g} |D]M} | dkrcd} |D]} | d}|d| }| || | | | qFq<| d} |D]} d| d}|d| }| || | | | qkq| d>B| BtjS)a5 Convert float32 numbers to mxf4 e2m1 format. * No encodings are reserved for Inf or NaN in mxf4. * Conversion from float supports roundTiesToEven rounding mode. * If a value exceeds the mxf4 representable range after rounding, clamps to the maximum mxf4 magnitude, preserving the sign. * If a value has magnitude less than the minimum subnormal magnitude in mxf4 after rounding, converts to zero. Parameters: - values: A torch tensor of float32 numbers to convert to fp4 format. r)rrrrrrrr#rr$r"T)dimkeepdimgư>r6r)r signbitrrabsisnanisinftensorrappenditemr%viewshape unsqueezemaxminsumexpandint32argmin)rvaluesr abs_valuesr+ is_invalidE_bitsM_bitscandidate_values candidate_E candidate_Mrr1r significandr* candidatesabs_values_flatNabs_values_expandedmax_candidate_valueerrors min_errors_is_tieM_bits_expanded tie_breaker best_indices E_selected M_selectedrrrr Nsd                   zMXFP4Tensor._from_floatc Cs|j}d|kr|jksJdJd||}|dd}|ddkrIdgd|j}|j|ddd}d||<tjjj||ddd}t|j}|||<| |dd|j |}| |dd}| |dd} | d>|B} | S)a Packs two e2m1 elements into a single uint8 along the specified dimension. Parameters: - dim: The dimension along which to pack the elements. Returns: - A torch tensor of dtype uint8 with two e2m1 elements packed into one uint8. rzHThe dimension to pack along is not within the range of tensor dimensionsrrconstant)moder*r) r ndimrr nn functionalpadlistrAinsertreshapeselect) rr6r size_along_dimnew_size_along_dim pad_sizes pad_index new_shapelowhighpackedrrrto_packed_tensors*       zMXFP4Tensor.to_packed_tensorc Cs|d?d@}|d@}tj||f|dd}t|j}|d|||dg||dd}|j|} ||ddkrStdg| j} td||| |<| t| } | tj S)a Unpacks a tensor where two fp4 elements are packed into a single uint8. Parameters: - packed_tensor: The packed tensor - dim: The dimension along which the tensor was packed. - original_shape: The shape of the original tensor before packing. Returns: - A tensor with the original data unpacked into uint8 elements containing one fp4e2m1 element in the least significant bits. rrr8Nrr) r stackrfrArhslicerbr rr) r packed_tensorr6original_shaperprostackedrArnr indicesrrrunpack_packed_tensors  *   z MXFP4Tensor.unpack_packed_tensorNNN) __name__ __module__ __qualname__rr!r4r rrrzrrrrr s #X #rc@s0eZdZd ddZd ddZddZdd ZdS) MXScaleTensorNcCr)a6 Tensor class for working with microscaling E8M0 block scale factors. Parameters: - data: A torch tensor of float32 numbers to convert to fp8e8m0 microscaling format. - size: The size of the tensor to create. - device: The device on which to create the tensor. Nrrrrrrrrs  zMXScaleTensor.__init__c Csd}|durdntdttt||}|durdntdtdttt||}||ks7Jdtj||d|jtj|j d}||_ |S)zp Generate random E8M0 data within a specified range. * Excludes the NaN encoding (255). Nrz&Low must be less than or equal to highrr) rCintr log2r=rDrrrrr )rrorpbias min_exponent max_exponentrrrrr!s*0zMXScaleTensor.randomcCs^|tjks Jd|j|}|dk}|}d||<|d}td|}tj||<||S)NzBCurrently only float32 is supported for f8e8m0 to float conversionrrg@)r r%r rcloner(nan)rrr is_nane_biaseder*rrrr4 s    zMXScaleTensor.toc Cstj|tj|jd}t|t|B|dkB}d||<||}tt|}|d}|tj }t |dd}|tj||<|S)aO Convert float32 numbers to E8M0 format. * Values <= 0, NaNs, and Infs are converted to the NaN encoding (255). * Positive values are converted by computing the floor of log2(value) to get the exponent. Parameters: - values: A torch tensor of float32 numbers to convert to E8M0 format. r5rrrr) r empty_likerrr;r<floorrrrGclamp) rrIresultrK valid_valuesrr e_biased_inte_biased_clampedrrrr s   zMXScaleTensor._from_floatr{)NN)r|r}r~rr!r4r rrrrrs    r)__doc__r rrrrrrs ^