o -i2 @sddlmZddlmZmZddlmZddlmZm Z ddl m Z ddl m Z GdddeZGd d d eZGd d d eZd S))Any) BaseModelField)ChatCompletionLogProbs)SamplingParams StreamOptions)Logprob random_uuidc@seZdZUeddddZeed<eeed< dZ edBed< e ed < dZ edBed <d Z e dBed <dZedBed <edddZedBed<edddZeed<edddZeeefdBed<dS)GenerateRequestcCtSNr rrc/home/ubuntu/veenaModal/venv/lib/python3.10/site-packages/vllm/entrypoints/serve/disagg/protocol.pyzGenerateRequest.The request_id related to this request. If the caller does not set it, a random_uuid will be generated. This id is used through out the inference process and return in response.default_factory description request_id token_idsNfeaturessampling_paramsmodelFstreamstream_optionsa/If specified, the prefix cache will be salted with the provided string to prevent an attacker to guess prompts in multi-user environments. The salt should be random, protected from access by 3rd parties, and long enough to be unpredictable (e.g., 43 characters base64-encoded, corresponding to 256 bit).defaultr cache_saltrzThe priority of the request (lower means earlier handling; default: 0). Any priority other than 0 will raise an error if the served model does not use priority scheduling.priority5KVTransfer parameters used for disaggregated serving.kv_transfer_params)__name__ __module__ __qualname__rrstr__annotations__listintrrrrboolrrrr r"dictrrrrrr s4   r c@sJeZdZUeed<dZedBed<dZedBed<dZ e edBed<dS)GenerateResponseChoiceindexNlogprobsstop finish_reasonr) r#r$r%r)r'r.rr0r&rr(rrrrr,Bs r,c@sreZdZUeddddZeed<eeed<dZ ee e e fdBdBed<edd d Z e eefdBed <dS) GenerateResponsecCr r r rrrrrLrzGenerateResponse.rrrchoicesNprompt_logprobsr!rr")r#r$r%rrr&r'r(r,r3r+r)rr"rrrrrr1Js   r1N)typingrpydanticrr0vllm.entrypoints.openai.chat_completion.protocolr'vllm.entrypoints.openai.engine.protocolrr vllm.logprobsr vllm.utilsr r r,r1rrrrs    1