o ۾i<@sddlZddlmZmZejdejfddZdejdejdejd dfd d Zejdejd ejfd dZ dejdejdejdejdejd e d ejfddZ dS)N)tltriton BLOCK_SIZEc Cstd}t||}t||tj}|dks|dkr!dStd} | |td|} | |k} tj|||| | d} | tj} | |} tj|||| | | ddS)Nrg?)mask)r program_idloadtofloat32arangestore) logits_ptr logits_strideidx_mapping_ptrtemperature_ptr vocab_sizer batch_idx req_state_idx temperature block_idxblockrlogitsrT/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/worker/gpu/sample/gumbel.py_temperature_kernels    rr idx_mappingrreturncCs@|j\}}d}t||}t||f||d||||ddS)Ni r)r)shapercdivrstride)rrrnum_reqsrr num_blocksrrrapply_temperature"s    r#APPLY_TEMPERATUREc Csjtd} t|| }td}|| td| }|| k}tj|| |||tdd}|tj}t| |tj}|dkrt||}t|| }t||}t||tj }t t |d d }|tj}| r|||}t |||td}tj |dd}|| |}tj |dd}t|| |||t|| |||dS)Nrrz-inf)rotherrg#B ;)axis)rrr r floatr r randintrandfloat64logwhereargmaxmaxr )local_argmax_ptrlocal_argmax_stride local_max_ptrlocal_max_striderrr seeds_ptrpos_ptrtemp_ptrrrr$rrrrrrtempseedpos gumbel_seedr gumbel_noiseidxtoken_idvaluerrr_gumbel_sample_kernel4s6      r?r7r8cCs|j\}}d}t||} tj|| tj|jd} tj|| tj|jd} t|| f| | d| | d|| d|||||||d | j ddd} | j d| d d} | S) Ni)dtypedevicer)rr$T)dimkeepdim)rCindex) rrrtorchemptyint64rAr r?r r-gatherview)rrrr7r8r#r!rrr" local_argmax local_max max_block_idxsampledrrr gumbel_samplemsB   rO) rFvllm.triton_utilsrrjit constexprrTensorr#r?boolrOrrrrsH    8