o -i&@s`UddlZddlmZmZmZmZddlZddlm Z m Z m Z m Z ddl mZddlmZddlmZddlmZddlmZeeZGd d d e ZGd d d eZGd ddeZGdddeZGdddeZGdddeZGdddeZGdddeZ Gddde Z!GdddeZ"GdddeZ#Gdd d eZ$Gd!d"d"eZ%e$e%BZ&ee'd#<Gd$d%d%eZ(e(e%Be$BZ)ee'd&<Gd'd(d(eZ*Gd)d*d*eZ+Gd+d,d,e Z,e-e.e,BZ/d-e/dBd.e.dBd/e-edBfd0d1Z0Gd2d3d3eZ1Gd4d5d5eZ2Gd6d7d7e Z3Gd8d9d9eZ4Gd:d;d;e Z5Gdd?d?e Z7dS)@N)AnyClassVarLiteral TypeAlias) BaseModel ConfigDictFieldmodel_validator)make_tool_call_id) init_logger)SamplingParams random_uuid)resolve_obj_by_qualnamec@sFeZdZUeddZdZeeedBe d<e dde ddZ dS) OpenAIBaseModelallowextraN field_nameswrapmodecs||}t|ts |S|jdur4t|jD]\}}|t|dd}r0|q|_tfdd|DrIt d| |S)Naliasc3s|]}|vVqdSN).0krrd/home/ubuntu/veenaModal/venv/lib/python3.10/site-packages/vllm/entrypoints/openai/engine/protocol.py 4sz7OpenAIBaseModel.__log_extra_fields__..z@The following fields were present in the request but ignored: %s) isinstancedictrset model_fieldsitemsaddgetattranyloggerwarningkeys)clsdatahandlerresult field_namefieldrrrr__log_extra_fields__#s$    z$OpenAIBaseModel.__log_extra_fields__) __name__ __module__ __qualname__r model_configrrr"str__annotations__r classmethodr1rrrrrs rc@s6eZdZUeed<eed<dZedBed<eed<dS) ErrorInfomessagetypeNparamcode)r2r3r4r6r7r<intrrrrr9<s  r9c@seZdZUeed<dS) ErrorResponseerrorN)r2r3r4r9r7rrrrr?Cs r?c@seZdZUedddZeed<dZeed<edddZe ed<d Z e ed <d Z e ed <d Z e ed <d Ze ed<d Ze ed<d Ze ed<dZeed<dZedBed<d Ze ed<dS)ModelPermissioncCs dtS)Nz modelperm-r rrrrH zModelPermission.default_factoryidmodel_permissionobjectcC ttSrr>timerrrrrBJrCcreatedFallow_create_engineTallow_samplingallow_logprobsallow_search_indices allow_viewallow_fine_tuning* organizationNgroup is_blocking)r2r3r4rrFr6r7rHrLr>rMboolrNrOrPrQrRrTrUrVrrrrrAGs         rAc@seZdZUeed<dZeed<edddZeed<dZ eed <d Z ed Bed <d Z ed Bed <d Z ed Bed <ee dZe eed<d S) ModelCardrFmodelrHcCrIrrJrrrrrBYrCzModelCard.rDrLvllmowned_byNrootparent max_model_len permission)r2r3r4r6r7rHrrLr>r[r\r]r^listr_rArrrrrXVs   rXc@s0eZdZUdZeed<eedZee ed<dS) ModelListr`rHrDr,N) r2r3r4rHr6r7rr`r,rXrrrrraas rac@seZdZUdZedBed<dS)PromptTokenUsageInfoN cached_tokens)r2r3r4rcr>r7rrrrrbfs rbc@sFeZdZUdZeed<dZeed<dZedBed<dZe dBed<dS) UsageInfor prompt_tokens total_tokensNcompletion_tokensprompt_tokens_details) r2r3r4rer>r7rfrgrhrbrrrrrdjs  rdc@s&eZdZUeed<dZedBed<dS)RequestResponseMetadata request_idNfinal_usage_info)r2r3r4r6r7rkrdrrrrriqs ric@sVeZdZUeed<dZedBed<edddZeee fdBed<dZ e dBed<dS)JsonSchemaResponseFormatnameN descriptionschemadefaultr json_schemastrict) r2r3r4r6r7rnrrrr!rrsrWrrrrrlvs  rlc@s>eZdZUeed<edddZeeefdBed<eed<dS)LegacyStructuralTagbeginNrorpstructural_tag_schemaend) r2r3r4r6r7rrvr!rrrrrrts   rtc@s2eZdZUeded<eeed<eeed<dS)!LegacyStructuralTagResponseFormatstructural_tagr; structurestriggersN)r2r3r4rr7r`rtr6rrrrrxs  rxc@s"eZdZUeded<eed<dS)StructuralTagResponseFormatryr;formatN)r2r3r4rr7rrrrrr|s  r|AnyStructuralTagResponseFormatc@s*eZdZUeded<dZedBed<dS)ResponseFormat)text json_objectrrr;Nrr)r2r3r4rr7rrrlrrrrrs rAnyResponseFormatc@s.eZdZUdZedBed<dZedBed<dS) StreamOptionsTN include_usageFcontinuous_usage_stats)r2r3r4rrWr7rrrrrr rc@s>eZdZUeed<dZedBed<dZeeefdBed<dS)FunctionDefinitionrmNrn parameters) r2r3r4r6r7rnrr!rrrrrrs rc@sLeZdZUeed<dZeedBed<dZe eefdBed<e ddZ dS)LogitsProcessorConstructorqualnameNargskwargsforbidr) r2r3r4r6r7rr`rrr!rr5rrrrrs r processorspatternreturnc Cs|rZ|rZg}|D]O}t|tr|n|j}t||s"td|dzt|}Wnty?}z td|d||d}~wwt|trR||j pJgi|j pOi}| |q|S|r`tddS)NzLogits processor 'ze' is not allowed by this server. See --logits-processor-pattern engine argument for more information.z' could not be resolved: zThe `logits_processors` argument is not supported by this server. See --logits-processor-pattern engine argument for more information.) r r6rrematch ValueErrorr Exceptionrrrappend)rrlogits_processors processorrlogits_processorerrrget_logits_processorss>     rc@seZdZUeed<eed<dS) FunctionCallrm argumentsN)r2r3r4r6r7rrrrrs  rc@s8eZdZUeedZeed<dZe ded<e ed<dS)ToolCallrDrFfunctionr;N) r2r3r4rr rFr6r7r;rrrrrrrs  rc@s.eZdZUdZedBed<dZedBed<dS)DeltaFunctionCallNrmr)r2r3r4rmr6r7rrrrrrrrc@sJeZdZUdZedBed<dZeddBed<eed<dZ e dBed<dS) DeltaToolCallNrFrr;index) r2r3r4rFr6r7r;rr>rrrrrrrs rc@s2eZdZUeed<eeed<dZedBed<dS)ExtractedToolCallInformation tools_called tool_callsNcontent) r2r3r4rWr7r`rrr6rrrrrs  rc@sxeZdZUdZedBed<dZedBed<dZedBed<dZedBed< e e dZ e e ed<e dd d d ZdS) DeltaMessageNroler reasoningreasoning_contentrDrafterrcCs |j|_|S)z?Copy reasoning to reasoning_content for backward compatibility.)rr)selfrrr#handle_deprecated_reasoning_contentsz0DeltaMessage.handle_deprecated_reasoning_content)r2r3r4rr6r7rrrrr`rrr rrrrrrs rc@seZdZUeeddZeed<ee ed< dZ edBed< e ed< dZ edBed<d Z edBed <dZedBed <edd d ZedBed<eddd Ze ed<eddd ZeeefdBed<dS)GenerateRequestzThe request_id related to this request. If the caller does not set it, a random_uuid will be generated. This id is used through out the inference process and return in response.)rErnrj token_idsNfeaturessampling_paramsrYFstreamstream_optionsa/If specified, the prefix cache will be salted with the provided string to prevent an attacker to guess prompts in multi-user environments. The salt should be random, protected from access by 3rd parties, and long enough to be unpredictable (e.g., 43 characters base64-encoded, corresponding to 256 bit).)rqrn cache_saltrzThe priority of the request (lower means earlier handling; default: 0). Any priority other than 0 will raise an error if the served model does not use priority scheduling.priorityz5KVTransfer parameters used for disaggregated serving.kv_transfer_params)r2r3r4rrrjr6r7r`r>rr rYrrWrrrrrr!rrrrrrs4   r)8rKtypingrrrrregexrpydanticrrrr vllm.entrypoints.chat_utilsr vllm.loggerr vllm.sampling_paramsr vllm.utilsrvllm.utils.import_utilsrr2r(rr9r?rArXrarbrdrirlrtrxr|r~r7rrrrrr`r6LogitsProcessorsrrrrrrrrrrrrs\               "