o پi@stddlmZddlmZddlZddlmZmZer ddlm Z eddd Z edd d Z dddZ dddZ dS)) annotations) TYPE_CHECKINGN) cache_onceload_jit)ModulereturnrcCtddgdgdS)N concat_mla_kelementwise/concat_mla.cuh)r zConcatMlaKKernel::run cuda_files cuda_wrappersrrrP/home/ubuntu/.local/lib/python3.10/site-packages/sglang/jit_kernel/concat_mla.py_jit_concat_mla_k_module rcCr)Nconcat_mla_absorb_qr )rzConcatMlaAbsorbQKernel::runr rrrrr_jit_concat_mla_absorb_q_modulerrk torch.Tensork_nopek_ropeNonecCst}||||dS)a Concatenate k_nope and k_rope into k for MLA (Multi-head Latent Attention). This kernel efficiently broadcasts k_rope across all heads while copying k_nope values directly. Args: k: Output tensor of shape [num_tokens, num_heads=128, k_head_dim=192], dtype=bfloat16 k_nope: Input tensor of shape [num_tokens, num_heads=128, nope_head_dim=128], dtype=bfloat16 k_rope: Input tensor of shape [num_tokens, 1, rope_head_dim=64], dtype=bfloat16 N)rr )rrrmodulerrrr s r abcCsPtjg|jdd|jd|jdR|j|jd}t}|||||S)aM Concatenate tensors a and b for MLA absorbed Q computation. Args: a: Input tensor of shape [dim_0, dim_1, a_last_dim], dtype=bfloat16 b: Input tensor of shape [dim_0, dim_1, b_last_dim], dtype=bfloat16 Returns: Output tensor of shape [dim_0, dim_1, a_last_dim + b_last_dim], dtype=bfloat16 N)dtypedevice)torchemptyshaperrrr)rroutrrrrr/s &r)rr)rrrrrrrr)rrrrrr) __future__rtypingrr sglang.jit_kernel.utilsrrtvm_ffi.modulerrrr rrrrrs