o ٷi@slddlmZddlmZddlZddlZddlmZer"ddl m Z ddl m Z m Z mZGdddeZdS) )Callable) TYPE_CHECKINGN)Request) BlockHash)AdditionalInformationPayloadOmniEngineCoreRequestPromptEmbedsPayloadcseZdZdZ   ddeejBdBdedBdedBffdd Z e deejBdBdejdBfd d Z e d e d ed gedfdBdd fddZZS) OmniRequesta*Request class for omni models, extending the base Request. This class extends the base vLLM Request with support for prompt embeddings and additional information payloads, enabling direct transfer of pre-computed embeddings between stages. Args: prompt_embeds: Optional serialized prompt embeddings payload. Used for direct transfer of embeddings between stages. additional_information: Optional additional information payload containing tensors or lists to be passed along with the request. N prompt_embedsexternal_req_idadditional_informationcsD||}tj|d|i|t|tr|nd|_||_||_dS)Nr )_maybe_decode_prompt_embedssuper__init__ isinstancerprompt_embeds_payloadr r )selfr r r argskwargsprompt_embeds_tensor __class__E/home/ubuntu/.local/lib/python3.10/site-packages/vllm_omni/request.pyrs  zOmniRequest.__init__returncCs@t|trtt|j}tj|j|d}||j}t |S|S)N)dtype) rrgetattrnpr frombufferdatareshapeshapetorch from_numpy)r rarrrrrr 0s    z'OmniRequest._maybe_decode_prompt_embedsrequest block_hasherrrcCs|did|jd|jd|jd|jd|jd|jd|jd|jd |jd |j d |j d |j d |j d|j d|d|jd|jd|jS)aECreate an OmniRequest from an OmniEngineCoreRequest. Args: request: The OmniEngineCoreRequest to convert block_hasher: Optional function to compute block hashes for prefix caching Returns: OmniRequest instance created from the engine core request request_idr client_indexprompt_token_idsr mm_featuressampling_paramspooling_params eos_token_id arrival_time lora_request cache_saltpriority trace_headersr&r resumablereasoning_endedNr)r'r r(r)r r*r+r,r-r.r/r0r1r2r r3r4)clsr%r&rrrfrom_engine_core_request;sJ     z$OmniRequest.from_engine_core_request)NNN)__name__ __module__ __qualname____doc__rr"Tensorstrrr staticmethodr classmethodrrlistr6 __classcell__rrrrr s4    r )collections.abcrtypingrnumpyrr"vllm.v1.requestrvllm.v1.core.kv_cache_utilsrvllm_omni.enginerrrr rrrrs