o iB@sddlmZmZddlmZmZddlmZmZddl m Z ddl m Z m Z ddlmZddlmZmZmZGdd d e ZGd d d e ZGd d d e ZGddde ZGdddeZGddde ZdS)) AnnotatedAny)Fieldmodel_validator)ChatCompletionMessageParamChatTemplateContentFormatOption)OpenAIBaseModel) ChatParams merge_kwargs) random_uuid) EmbedDTypeEncodingFormat Endiannessc@seZdZUdZedBed<dZedBed<dZee e ddfdBed<e e ddZ eed <e d d d Z e ed <e ddd ZeeefdBed<e ddd ZedBed<dS)PoolingBasicRequestMixinNmodeluser)getruncate_prompt_tokenszThe request_id related to this request. If the caller does not set it, a random_uuid will be generated. This id is used through out the inference process and return in response.)default_factory description request_idrzThe priority of the request (lower means earlier handling; default: 0). Any priority other than 0 will raise an error if the served model does not use priority scheduling.defaultrpriorityz.Additional kwargs to pass to the HF processor.mm_processor_kwargsa/If specified, the prefix cache will be salted with the provided string to prevent an attacker to guess prompts in multi-user environments. The salt should be random, protected from access by 3rd parties, and long enough to be unpredictable (e.g., 43 characters base64-encoded, corresponding to 256 bit). cache_salt)__name__ __module__ __qualname__rstr__annotations__rrrintrr rrrdictrrr$r$\/home/ubuntu/vllm_env/lib/python3.10/site-packages/vllm/entrypoints/pooling/base/protocol.pyrs( rc@sFeZdZUeeeeeBeBeeBed<edddZe ed<dS)CompletionRequestMixininputTzMIf true (the default), special tokens (e.g. BOS) will be added to the prompt.radd_special_tokensN) rrrlistr"r r!rr(boolr$r$r$r%r&=s $r&c@seZdZUeeed<edddZeed<edddZ eed<edddZ eed <ed d dZ e d Bed <ed d dZ ee efd Bed<eddeddZde d BdedefddZd S)ChatRequestMixinmessagesFzIf true, the generation prompt will be added to the chat template. This is a parameter used by chat template in tokenizer config of the model.radd_generation_prompta:If this is set, the chat will be formatted so that the final message in the chat is open-ended, without any EOS tokens. The model will continue this message rather than starting a new one. This allows you to "prefill" part of the model's response for it. Cannot be used at the same time as `add_generation_prompt`.continue_final_messagezIf true, special tokens (e.g. BOS) will be added to the prompt on top of what is added by the chat template. For most models, the chat template takes care of adding the special tokens so this should be set to false (as is the default).r(NzA Jinja template to use for this conversion. As of transformers v4.44, default chat template is no longer allowed, so you must provide a chat template if the tokenizer does not define one. chat_templatezbAdditional keyword args to pass to the template renderer. Will be accessible by the chat template.chat_template_kwargsbefore)modecCs |dr|drtd|S)Nr.r-zMCannot set both `continue_final_message` and `add_generation_prompt` to True.)get ValueError)clsdatar$r$r%check_generation_prompts z(ChatRequestMixin.check_generation_promptdefault_templatedefault_template_content_formatreturnc Cs(t|jp||t|jt|j|jddS)N)r-r.)r/chat_template_content_formatr0)r r/r r0r#r-r.)selfr8r9r$r$r%build_chat_paramssz"ChatRequestMixin.build_chat_params)rrrr)rr!rr-r*r.r(r/r r0r#rr classmethodr7rr r=r$r$r$r%r+Ms@      r+c@sBeZdZUdZeed<edddZeed<edddZ e ed <d S) EncodingRequestMixinfloatencoding_formatfloat32zWhat dtype to use for encoding. Default to using float32 for base64 encoding to match the OpenAI python client behavior. This parameter will affect base64 and binary_response.r embed_dtypenativezWhat endianness to use for encoding. Default to using native for base64 encoding to match the OpenAI python client behavior.This parameter will affect base64 and binary_response. endiannessN) rrrrAr r!rrCr rErr$r$r$r%r?s r?c@sNeZdZUdZedBed<edddZedBed<edddZ edBed<dS)EmbedRequestMixinN dimensionsrWhether to use activation for the pooler outputs. `None` uses the pooler's default, which is `True` in most cases.ruse_activationz0Deprecated; please pass `use_activation` instead normalize) rrrrGr"r!rrIr*rJr$r$r$r%rFs rFc@s&eZdZUedddZedBed<dS)ClassifyRequestMixinNrHrrI)rrrrrIr*r!r$r$r$r%rKs rKN)typingrrpydanticrrvllm.entrypoints.chat_utilsrr'vllm.entrypoints.openai.engine.protocolrvllm.renderersr r vllm.utilsr vllm.utils.serial_utilsr r rrr&r+r?rFrKr$r$r$r%s  *P