o پi`@s\ddlZdZejr dndZejgdejedZejgdejedZGdd d Z dS) Ng@cudacpu)rg?g?dtypedevice)g?g?g?g?g@g @c @sleZdZdZeejdejdeejejffddZ eejej fdejdejdej dejfd d Z d S) KVFP4QuantizeUtilzCUtility class for MXFP4 quantization and dequantization operations.tensorreturncCs|j\}}}||||dd}|jdddj}tttj|t dd}|d d tj }|t |}|dk tj d >} |} tj| dtkdd } | | tj } | |||} | d d d dfd>| d dd df}||fS)a Quantize tensor to KVFP4 format Args: tensor: Input tensor of shape [B, M, N] Returns: quant_tensor: Quantized tensor of shape [B, M, N/2] scale_factors: Scale factors of shape [B, M*N/16] T)dimkeepdimg|=)minrr)r.rNrr)shapeviewabsmaxvaluestorchceillog2clampE2M1_MAXsqueezetouint8exp2sum unsqueeze E2M1_BOUNDS)rbmnreshaped block_max scale_exp scale_factorsscaled sign_bitsabs_valsmagnitude_bitsfp4_vals fp4_reshapedpackedr5_/home/ubuntu/.local/lib/python3.10/site-packages/sglang/srt/layers/quantization/kvfp4_tensor.pybatched_quantizes (z"KVFP4QuantizeUtil.batched_quantize quant_tensorr-r cCs|j\}}}|d}tj|||tj|jd}|d@|ddddf<|d?d@|ddddf<|d @dk}|d @} t| } t|| | } | |||d d } | d } | t | d } | ||| |S)a$ Dequantize KVFP4 tensor Args: quant_tensor: Quantized tensor of shape [B, M, N/2] scale_factors: Scale factors of shape [B, M*N/16] dtype: Target dtype for output Returns: Dequantized tensor of shape [B, M, N] rr .rNrrrrr) rremptyr"r E2M1_VALUESlongwhererfloatr#r%r!)r8r-r r'r(n_halfr)r2 sign_mask magnitude_idx float_valsr*r,r.r5r5r6batched_dequantizeHs    z$KVFP4QuantizeUtil.batched_dequantizeN) __name__ __module__ __qualname____doc__ staticmethodrcompileTensortupler7bfloat16r rEr5r5r5r6r s"$'r ) rrr is_available_devicerfloat32r=r&r r5r5r5r6s