o پi4 @sddlmZmZmZddlZddlmZmZ  ddejdejdee dejdeejd eejd ejfd d Z ej fd ejdej d eejejffddZ dejdejdee d ejfddZdS))ListOptionalTupleN)per_token_group_quant_int8w8a8_block_int8_matmulinputweight block_size weight_scale input_scalebiasreturnc Cs|dusJ|d|jd}g|jdd|jd}t||d\}} t||| |||jd} |dur9| |} | j|jdj|S)Nr) output_dtype)dtype)viewshaperrrto) rrr r r r input_2d output_shapeq_inputx_scaleoutputr]/home/ubuntu/.local/lib/python3.10/site-packages/sglang/srt/layers/quantization/int8_utils.pyapply_w8a8_block_int8_linear s rxrc Cstt|}|\}}t||jdd}|j|j}}||}||j||d} | | | fS)zRThis function quantizes input values to int8 values with tensor-wise quantization.g-q=)min)rmax) torchiinfoaminmaxmaximumabsclamprrr contiguousfloat reciprocal) rrr!min_valmax_valamaxint8_minint8_maxscale x_scl_satrrr input_to_int8"s  r0 x_q_blockx_sc Cs|d|d}}|j\}}||d|}||d|}||jdks'J||jdks0J|tj} t|D]-} t|D]&} | | |t| d||| |t| d||f|| | 9<q@q:| S)zThis function conducts block-wise dequantization. The inputs are block-wise quantization tensor `x_q_block`, block-wise quantization scale and the block size. The outputs are dequantized tensor. rr)rrr float32ranger) r1r2r block_nblock_knkn_tilesk_tiles x_dq_blockijrrr block_dequant/s$     r>)NN)typingrrrr *sglang.srt.layers.quantization.int8_kernelrrTensorintrint8rr0r>rrrrsJ