o پi@sddlmZddlZddZ ddejdejdedeejd ejf d d Z ddejdejd ejd ejdedeejd ejfddZ ddejdejdejdedeejd ejf ddZdS))OptionalNcCs(|dkr tj||ddStj|||dS)NT)dimkeepdim)r)torchmaxtopk)valuesrrr D/home/ubuntu/.local/lib/python3.10/site-packages/sgl_kernel/top_k.py fast_topksr scorelengthsr row_startsreturncCsR|dksJd|dksJ|j|d|ftjd}tjj|||||S)a Get the topk indices of the score tensor. Args: score: The score tensor of shape (B, L). The score tensor is the logits between the query and the key whose layout is either ragged or paged. row_starts is only required when the key is ragged. lengths: The lengths tensor of shape (B) topk: The number of topk indices to get row_starts: The start index of each row in the score tensor of shape (B). For each row i, topk only applies to section [row_starts[i], row_starts[i] + lengths[i]] of the score tensor. Returns: The topk indices tensor of shape (B, topk) zGfast_topk_v2 is only optimized for deepseek v3.2 model, where topk=2048rdtype)r new_emptysizerint32ops sgl_kernelr )r rrr topk_indicesr r r fast_topk_v2s rpage_table_size_1 cu_seqlens_qcCsZ|dksJd|dksJ|}|j|jd|ftjd}tjj|||||||S)a Get the topk indices of the score tensor and then transform the topk indices to indices to the page table (page_size = 1) Args: score: The score tensor of shape (B, L). The score tensor is the logits between the query and the key whose layout is either ragged or paged. row_starts is only required when the key is ragged. lengths: The lengths tensor of shape (B) page_table_size_1: The page table tensor of shape (Batch, topk) cu_seqlens_q: The cumulative sequence lengths tensor of shape (Batch + 1) topk: The number of topk indices to get row_starts: The start index of each row in the score tensor of shape (B). For each row i, topk only applies to section [row_starts[i], row_starts[i] + lengths[i]] of the score tensor. It's only used for cases where the key is ragged, i.e. during extend and draft extend. Returns: The topk indices tensor of shape (B, topk) rzTfast_topk_transform_fused is only optimized for deepseek v3.2 model, where topk=2048rrr)rrshaperrrrfast_topk_transform_fused)r rrrrrsrc_page_tabledst_page_tabler r r r-s  rtopk_indices_offsetcCsT|dksJd|dksJ|j|jd|ftjd}tjj||||||S)a" Get the topk indices of the score tensor and then transform the topk indices to indices to ragged kv (non-paged). This function is only used for extend, not including draft extend. Args: score: The score tensor of shape (B, L). The score tensor is the logits between the query and the key which can be ragged or paged. row_starts is only required when the key is ragged. lengths: The lengths tensor of shape (B) topk_indices_offset: The offset of topk indices in ragged kv of shape (B) topk: The number of topk indices to get row_starts: The start index of each row in the score tensor of shape (B). For each row i, topk only applies to section [row_starts[i], row_starts[i] + lengths[i]] of the score tensor. It can be None if only the fast path is triggered, in the case of all values in lengths <= topk (not checked in the kernel, guaranteed by the caller). Returns: The topk indices tensor of shape (B, topk) rz[fast_topk_transform_ragged_fused is only optimized for deepseek v3.2 model, where topk=2048rrr)rrrrrrr fast_topk_transform_ragged_fused)r rr"rrtopk_indices_raggedr r r r#Ss  r#)N) typingrrr Tensorintrrr#r r r r sZ  # +