o ÁÙ¾i9ã@søddlZdejdedededejf dd„Zdejd ejded ed ejf d d „Zdejd ejded ed ejf dd„Zdejdejdejdejdejded ededed ejfdd„Zdejdejdejdeded eded ejfdd„Zded efdd„Z dS)éNÚweightÚ quant_typeÚMÚNÚdtypecCs0|dkr|dks Jdƒ‚tjjj |||||¡S)Nrz6GGUF weight Input shape must be of positive dimensions)ÚtorchÚopsÚ sgl_kernelÚggml_dequantizeÚdefault)rrrrr©r úP/home/ubuntu/.local/lib/python3.10/site-packages/sgl_kernel/quantization/gguf.pyr sr ÚxÚrowÚreturncCótjjj ||||¡S©N)rrr Úggml_mul_mat_vec_a8r ©rrrrr r r r órcCrr)rrr Úggml_mul_mat_a8r rr r r rrrÚinputÚsorted_token_idsÚ expert_idsÚnum_token_post_paddedÚtypeÚtopkÚtokensc Cs tjjj |||||||||¡ Sr)rrr Ú ggml_moe_a8r ) rrrrrrrrrr r r rs ÷rÚtopk_idsÚtop_kc Cstjjj |||||||¡Sr)rrr Úggml_moe_a8_vecr )rrrr rrrr r r r!/s ÿr!cCstjjj |¡Sr)rrr Úggml_moe_get_block_sizer )rr r r r"=sr") rÚTensorÚintrr rrrr!r"r r r r Ús’ÿÿÿÿ ÿÿÿÿÿ þÿÿÿÿ þÿþýüûúùø ÷ öÿþýüûúù ø