o iq&@s`UddlZddlmZmZmZmZddlZddlm Z m Z m Z m Z ddl mZddlmZddlmZddlmZddlmZeeZGd d d e ZGd d d eZGd ddeZGdddeZGdddeZGdddeZGdddeZGdddeZ Gddde Z!GdddeZ"GdddeZ#Gdd d eZ$Gd!d"d"eZ%e$e%BZ&ee'd#<Gd$d%d%eZ(e(e%Be$BZ)ee'd&<Gd'd(d(eZ*Gd)d*d*eZ+Gd+d,d,e Z,e-e.e,BZ/d-e/dBd.e.dBd/e-edBfd0d1Z0Gd2d3d3eZ1Gd4d5d5eZ2Gd6d7d7e Z3Gd8d9d9eZ4Gd:d;d;e Z5Gdd?d?e Z7dS)@N)AnyClassVarLiteral TypeAlias) BaseModel ConfigDictFieldmodel_validator)make_tool_call_id) init_logger)SamplingParams random_uuid)resolve_obj_by_qualnamec@sFeZdZUeddZdZeeedBe d<e dde ddZ dS) OpenAIBaseModelallowextraN field_nameswrap)modecs||}t|ts |S|jdur4t|jD]\}}|t|dd}r0|q|_tfdd|DrIt d| |S)Naliasc3s|]}|vVqdSN).0krr]/home/ubuntu/vllm_env/lib/python3.10/site-packages/vllm/entrypoints/openai/engine/protocol.py 2sz7OpenAIBaseModel.__log_extra_fields__..z@The following fields were present in the request but ignored: %s) isinstancedictrset model_fieldsitemsaddgetattranyloggerwarningkeys)clsdatahandlerresult field_namefieldrrrr__log_extra_fields__!s$    z$OpenAIBaseModel.__log_extra_fields__) __name__ __module__ __qualname__r model_configrrr!str__annotations__r classmethodr0rrrrrs rc@s6eZdZUeed<eed<dZedBed<eed<dS) ErrorInfomessagetypeNparamcode)r1r2r3r5r6r;intrrrrr8:s  r8c@seZdZUeed<dS) ErrorResponseerrorN)r1r2r3r8r6rrrrr>As r>c@seZdZUedddZeed<dZeed<edddZe ed<d Z e ed <d Z e ed <d Z e ed <d Ze ed<d Ze ed<d Ze ed<dZeed<dZedBed<d Ze ed<dS)ModelPermissioncCs dtS)Nz modelperm-r rrrrF zModelPermission.default_factoryidmodel_permissionobjectcC ttSrr=timerrrrrAHrBcreatedFallow_create_engineTallow_samplingallow_logprobsallow_search_indices allow_viewallow_fine_tuning* organizationNgroup is_blocking)r1r2r3rrEr5r6rGrKr=rLboolrMrNrOrPrQrSrTrUrrrrr@Es         r@c@seZdZUeed<dZeed<edddZeed<dZ eed <d Z ed Bed <d Z ed Bed <d Z ed Bed <ee dZe eed<d S) ModelCardrEmodelrGcCrHrrIrrrrrAWrBzModelCard.rCrKvllmowned_byNrootparent max_model_len permission)r1r2r3r5r6rGrrKr=rZr[r\r]listr^r@rrrrrWTs   rWc@s0eZdZUdZeed<eedZee ed<dS) ModelListr_rGrCr+N) r1r2r3rGr5r6rr_r+rWrrrrr`_s r`c@seZdZUdZedBed<dS)PromptTokenUsageInfoN cached_tokens)r1r2r3rbr=r6rrrrrads rac@sFeZdZUdZeed<dZeed<dZedBed<dZe dBed<dS) UsageInfor prompt_tokens total_tokensNcompletion_tokensprompt_tokens_details) r1r2r3rdr=r6rerfrgrarrrrrchs  rcc@s&eZdZUeed<dZedBed<dS)RequestResponseMetadata request_idNfinal_usage_info)r1r2r3r5r6rjrcrrrrrhos rhc@sVeZdZUeed<dZedBed<edddZeee fdBed<dZ e dBed<dS)JsonSchemaResponseFormatnameN descriptionschemadefaultr json_schemastrict) r1r2r3r5r6rmrrqr rrrrVrrrrrkts  rkc@s>eZdZUeed<edddZeeefdBed<eed<dS)LegacyStructuralTagbeginNrnrostructural_tag_schemaend) r1r2r3r5r6rrur rrrrrrs}s   rsc@s2eZdZUeded<eeed<eeed<dS)!LegacyStructuralTagResponseFormatstructural_tagr: structurestriggersN)r1r2r3rr6r_rsr5rrrrrws  rwc@s"eZdZUeded<eed<dS)StructuralTagResponseFormatrxr:formatN)r1r2r3rr6rrrrrr{s  r{AnyStructuralTagResponseFormatc@s*eZdZUeded<dZedBed<dS)ResponseFormat)text json_objectrqr:Nrq)r1r2r3rr6rqrkrrrrr~s r~AnyResponseFormatc@s.eZdZUdZedBed<dZedBed<dS) StreamOptionsTN include_usageFcontinuous_usage_stats)r1r2r3rrVr6rrrrrr rc@s>eZdZUeed<dZedBed<dZeeefdBed<dS)FunctionDefinitionrlNrm parameters) r1r2r3r5r6rmrr rrrrrrs rc@sLeZdZUeed<dZeedBed<dZe eefdBed<e ddZ dS)LogitsProcessorConstructorqualnameNargskwargsforbidr) r1r2r3r5r6rr_rrr rr4rrrrrs r processorspatternreturnc Cs|rZ|rZg}|D]O}t|tr|n|j}t||s"td|dzt|}Wnty?}z td|d||d}~wwt|trR||j pJgi|j pOi}| |q|S|r`tddS)NzLogits processor 'ze' is not allowed by this server. See --logits-processor-pattern engine argument for more information.z' could not be resolved: zThe `logits_processors` argument is not supported by this server. See --logits-processor-pattern engine argument for more information.) rr5rrematch ValueErrorr Exceptionrrrappend)rrlogits_processors processorrlogits_processorerrrget_logits_processorss>     rc@s6eZdZUedddZedBed<eed<eed<dS) FunctionCallNT)rpexcluderErl arguments)r1r2r3rrEr5r6rrrrrs  rc@s8eZdZUeedZeed<dZe ded<e ed<dS)ToolCallrCrEfunctionr:N) r1r2r3rr rEr5r6r:rrrrrrrs  rc@s.eZdZUdZedBed<dZedBed<dS)DeltaFunctionCallNrlr)r1r2r3rlr5r6rrrrrrrrc@sJeZdZUdZedBed<dZeddBed<eed<dZ e dBed<dS) DeltaToolCallNrErr:index) r1r2r3rEr5r6r:rr=rrrrrrrs rc@s2eZdZUeed<eeed<dZedBed<dS)ExtractedToolCallInformation tools_called tool_callsNcontent) r1r2r3rVr6r_rrr5rrrrrs  rc@sTeZdZUdZedBed<dZedBed<dZedBed<ee dZ e e ed<dS) DeltaMessageNroler reasoningrCr) r1r2r3rr5r6rrrr_rrrrrrrs rc@seZdZUeeddZeed<ee ed< dZ edBed< e ed< dZ edBed<d Z edBed <dZedBed <edd d ZedBed<eddd Ze ed<eddd ZeeefdBed<dS)GenerateRequestzThe request_id related to this request. If the caller does not set it, a random_uuid will be generated. This id is used through out the inference process and return in response.)rDrmri token_idsNfeaturessampling_paramsrXFstreamstream_optionsa/If specified, the prefix cache will be salted with the provided string to prevent an attacker to guess prompts in multi-user environments. The salt should be random, protected from access by 3rd parties, and long enough to be unpredictable (e.g., 43 characters base64-encoded, corresponding to 256 bit).)rprm cache_saltrzThe priority of the request (lower means earlier handling; default: 0). Any priority other than 0 will raise an error if the served model does not use priority scheduling.priorityz5KVTransfer parameters used for disaggregated serving.kv_transfer_params)r1r2r3rrrir5r6r_r=rr rXrrVrrrrrr rrrrrr s4   r)8rJtypingrrrrregexrpydanticrrrr vllm.entrypoints.chat_utilsr vllm.loggerr vllm.sampling_paramsr vllm.utilsrvllm.utils.import_utilsrr1r'rr8r>r@rWr`rarcrhrkrsrwr{r}r6r~rrrrr_r5LogitsProcessorsrrrrrrrrrrrrs\               "