o c۷i` @sZddlZddlZddlmZmZmZddlmZddlm Z ddl m Z m Z m Z ddlmZmZmZmZmZmZmZmZmZmZmZddlmZmZmZddlmZm Z m!Z!dd l"m#Z#m$Z$dd l%m&Z&d Z'Gd d d eeeeee#fZ(Gddde(Z)Gddde)Z*Gddde*Z+ dde$de&dBde(fddZ, dde$de&dBde(fddZ-dS)N)GenericSequenceoverload) assert_never)InvalidRequestException) TextChunk ThinkChunkUserContentChunk) UATSAssistantMessageAssistantMessageTypeFinetuningAssistantMessageRoles SystemMessageSystemMessageType ToolMessageToolMessageType UserMessageUserMessageType)ChatCompletionRequestInstructRequest ModelSettings) FunctionCallToolToolCall)InstructRequestTypeTokenizerVersion)ModelSettingsBuilderz c @s*eZdZUdZdZeed<dZeed<dee dee dee dee d ee d ed Bf d d Zed2d ed BddfddZdedefddZded BdefddZedeeeBeBdeeeeBBfddZededefddZedeedefddZdeeeeBeBBeeBdeeeeBBfddZdeeded BfddZdeedeedee fd d!Zd"edefd#d$Z deedee fd%d&Z!deede fd'd(Z"deede fd)d*Z#deed+e$d Bdeede%efd,d-Z&deedeefd.d/Z'deede fd0d1Z(d S)3InstructRequestNormalizeraTakes a [ChatCompletionRequest][mistral_common.protocol.instruct.request.ChatCompletionRequest] and normalizes it into an [InstructRequest][mistral_common.protocol.instruct.request.InstructRequest]. The normalization process does several things such as: - Aggregate consecutive messages of the same role - Aggregate system prompts - Normalize json content - Normalize tool calls Examples: >>> normalizer = InstructRequestNormalizer.normalizer() F_system_prompt_in_begin_allow_tool_call_and_contentuser_message_classassistant_message_classtool_message_classsystem_message_classinstruct_request_classmodel_settings_builderNcCs(||_||_||_||_||_||_dS)aInitializes the normalizer with the appropriate message classes. Args: user_message_class: The class for user messages. assistant_message_class: The class for assistant messages. tool_message_class: The class for tool messages. system_message_class: The class for system messages. instruct_request_class: The class for instruct requests. model_settings_builder: The builder for model settings, or None if unsupported. N)_user_message_class_assistant_message_class_tool_message_class_instruct_request_class_system_message_class_model_settings_builder)selfr!r"r#r$r%r&r.`/home/ubuntu/vllm_env/lib/python3.10/site-packages/mistral_common/protocol/instruct/normalize.py__init__5s  z"InstructRequestNormalizer.__init__returncC0|dur td|ttttttttfdS)aReturns a normalizer for the default instruct request. Args: model_settings_builder: Must be None for this normalizer version. Returns: A normalizer for the default instruct request. Raises: ValueError: If model_settings_builder is not None. Examples: >>> normalizer = InstructRequestNormalizer.normalizer() NzGmodel_settings_builder must be None for InstructRequestNormalizer, got ) ValueErrorrrr rrrr rr&r.r.r/ normalizerPz$InstructRequestNormalizer.normalizerrequestcCs.|jdurtdt|jd|jtS)zBuild model settings from a chat completion request. For pre-v15 normalizers, model settings are all `None`. Args: request: The chat completion request. Returns: Returns `ModelSettings.none()`. Nz*model_settings_builder should be None for , got )r,rtype__name__rnoner-r7r.r.r/build_settingshs z(InstructRequestNormalizer.build_settingscontentcCsR|dus t|dkr dSzt|}tj|dd}W|Stjy(|}Y|Sw)Nrz{}F) ensure_ascii)lenjsonloadsdumpsJSONDecodeError)r-r> parsed_jsonnormalized_contentr.r.r/_normalize_json_contentys z1InstructRequestNormalizer._normalize_json_contentcCdSNr.r-r>r.r.r/_aggregate_content_chunkssz3InstructRequestNormalizer._aggregate_content_chunkscCrHrIr.rJr.r.r/rKcCrHrIr.rJr.r.r/rKrLcCst|tr|St|tsJdt|g}|D]@}t|tr%t|d}t|trF|r@t|dtr@|djt|j7_q||qt|trQ||qt dt|t |dkrlt|dtrl|djS|S)NzExpected list, got textzUnsupported chunk type r) isinstancestrlistr9rrNCHUNK_JOIN_STRappendrr3r@)r-r>aggregated_contentchunkr.r.r/rKs"        messagescCsLg}|D]}|jtjkr|jr||j}||qt|r$t|SdSrI) rolersystemr>rKrUr@rTjoin)r-rX system_promptmessagerVr.r.r/_aggregate_system_promptss  z3InstructRequestNormalizer._aggregate_system_promptslatest_call_idscCsng}|D]0}t||jsJd|j}t|ts"tdd|D}||}||j||j|j dq|S)zm We currently do not do any aggregation for tool messages, but we normalize the json content zExpected tool messagecSsg|]}|jqSr.rM).0rWr.r.r/ szFInstructRequestNormalizer._aggregate_tool_messages..)r> tool_call_idname) rQr)r>rRrTr[rGrUrbrc)r-rXr_ tool_messagesr]r>rFr.r.r/_aggregate_tool_messagess   z2InstructRequestNormalizer._aggregate_tool_messages tool_callcCs(||jj}tt|jj|d|jdS)N)rc arguments)functionid)rGrhrgrrrcri)r-rfnormalized_function_arumentsr.r.r/_normalize_tool_calls z.InstructRequestNormalizer._normalize_tool_callcCsgSrIr.r-rXr.r.r/_aggregate_system_messagesrLz4InstructRequestNormalizer._aggregate_system_messagesc Csg}g}d}d}|D]]}t||jsJd|js&|jr&|jr&td||jr9|jD] }||}||q,|j} durM|t| t rJ| gn| ||j O}t|t rg|durd||j ksdJd|j }q |rp| |} nd} |j| |pxd|d} |durt| dr|| _ | S)NFzExpected assistant messagezDTool calls and content cannot be used together in the same message. zEExpected weights of aggregated FinetuningAssistantMessage to be equal)r> tool_callsprefixweight)rQr(r rnr>r3rkrUextendrRror rprKhasattr) r-rXmessages_contentsrnrorpr]rfnormalized_tool_callr>rVaggregated_messager.r.r/_aggregate_assistant_messagessB       z7InstructRequestNormalizer._aggregate_assistant_messagescCsg}g}|D]D}t||jsJdt|t|jtr$||jq|jD]"}t|tr5||jq'|rD|tt |dg}||q'q|rRt |nd}|s\|j|dS|rf|t|d|j|dS)z: Just coalesce neighboring blocks of text zExpected user message got rMr>) rQr'r9r>rRrUrrNrTr[)r-rX all_content text_chunksr]rW text_contentr.r.r/_aggregate_user_messagess(       z2InstructRequestNormalizer._aggregate_user_messagesrYcCsL|tjkr |||S|tjkr||gS|tjkr!||gS||SrI)rtoolre assistantrvuserr|rmr-rXrYr_r.r.r/_aggregate_roles       z)InstructRequestNormalizer._aggregate_rolec Csg}g}d}d}g}|D]V}t|dd}||jks||krX||||||tjkrK|d} t|dts9J| jdurJ| jD]} | | j qAn |tj krT| | |}|j}| |q |||||t |dks~|js|djtjkr|d|jdd|S)NrprOrrwrx)getattrrYrqrrr~rQr rnrUrir}clearr@rrinsertr') r-rXaggregated_messagesmessages_to_aggregate current_rolecurrent_weightr_r] new_weightassistant_messagerfr.r.r/_aggregate_messages&s:       z-InstructRequestNormalizer._aggregate_messagescCs`||j}||j}||}|tkr$tdt|jd||j |||j |j |dS)aConverts a chat completion request to an instruct request. Args: request: The chat completion request to convert. Returns: The converted instruct request. Examples: >>> from mistral_common.protocol.instruct.messages import UserMessage, AssistantMessage >>> request = ChatCompletionRequest( ... messages=[ ... UserMessage(content="Hello"), ... AssistantMessage(content="Hi"), ... ], ... ) >>> normalizer = InstructRequestNormalizer.normalizer() >>> instruct_request = normalizer.from_chat_completion_request(request) %Model settings are not supported for r8)rXr\available_toolscontinue_final_messagesettings) r^rXrr=rr;rr9r:r*toolsr)r-r7r\rXrr.r.r/from_chat_completion_requestOs    z6InstructRequestNormalizer.from_chat_completion_requestrI))r: __module__ __qualname____doc__rbool__annotations__r r9rr rrrrr0 staticmethodr5rrr=rRrGrrSrrrKr r^rerrkrmrvr|rrrrrr.r.r.r/r"sZ      " -*  )rc @seZdZUdZdZeed<dZeed<edde dBddfdd Z d e e de e fd d Zd e e d edBde edee fddZd e e dedBfddZdee defddZdS)InstructRequestNormalizerV7zvNormalizer for the v7 tokenizer. Examples: >>> normalizer = InstructRequestNormalizerV7.normalizer() Trr Nr&r1cCr2)aReturns a normalizer for the default instruct request. Args: model_settings_builder: Must be None for this normalizer version. Returns: A normalizer for the V7 instruct request. Raises: ValueError: If model_settings_builder is not None. Examples: >>> normalizer = InstructRequestNormalizerV7.normalizer() NzImodel_settings_builder must be None for InstructRequestNormalizerV7, got ) r3rrr rrrr rr4r.r.r/r5}r6z&InstructRequestNormalizerV7.normalizerrXcsfdd|DS)Ncs,g|]}t|jrj|jdqS)rx)rQr+rKr>)r`r]r-r.r/ras  zJInstructRequestNormalizerV7._aggregate_system_messages..r.rlr.rr/rms z6InstructRequestNormalizerV7._aggregate_system_messagesrYr_cCsr|tjkr |||S|tjkr||gS|tjkr!||gS|tjkr+||S|dur5t |dks7JgS)Nr) rr}rer~rvrr|rZrmr@rr.r.r/rs        z+InstructRequestNormalizerV7._aggregate_rolecCstd)Nz&We should not aggregate system prompts)NotImplementedErrorrlr.r.r/r^sz5InstructRequestNormalizerV7._aggregate_system_promptsr7cCsP||j}||}|tkrtdt|jd||j|d|j |dS)aConverts a chat completion request to an instruct request. Args: request: The chat completion request to convert. Returns: The converted instruct request. Examples: >>> from mistral_common.protocol.instruct.messages import UserMessage, AssistantMessage >>> request = ChatCompletionRequest( ... messages=[ ... UserMessage(content="Hello"), ... AssistantMessage(content="Hi"), ... ], ... ) >>> normalizer = InstructRequestNormalizerV7.normalizer() >>> instruct_request = normalizer.from_chat_completion_request(request) rr8NrXr\rr) rrXr=rr;rr9r:r*rr-r7rXrr.r.r/rs    z8InstructRequestNormalizerV7.from_chat_completion_requestrI)r:rrrrrrr rrr5rSr rrmrrRrrr^rrrr.r.r.r/rss   * rcsVeZdZdZed dedBddfddZdeedee dee ffd d Z Z S) InstructRequestNormalizerV13zNormalizer for the v13 tokenizer. It reorders tool messages based on the tool call order. Examples: >>> normalizer = InstructRequestNormalizerV13.normalizer() Nr&r1cCr2)a5Returns a normalizer for the default instruct request. Args: model_settings_builder: Must be None for this normalizer version. Returns: A normalizer for the V13 instruct request. Raises: ValueError: If model_settings_builder is not None. NzJmodel_settings_builder must be None for InstructRequestNormalizerV13, got ) r3rrr rrrr rr4r.r.r/r5s z'InstructRequestNormalizerV13.normalizerrXr_csLt||}ddt|Dddt|D|jfddd|S)NcSsi|]\}}||qSr.r.)r`idxcall_idr.r.r/ szIInstructRequestNormalizerV13._aggregate_tool_messages..cSsi|]\}}|j|qSr.)rb)r`rr]r.r.r/rscs |jpdtd|jfS)Nnullinf)getrbfloat)msgid_to_tool_call_idxid_to_tool_result_idxr.r/szGInstructRequestNormalizerV13._aggregate_tool_messages..)key)superre enumeratesort)r-rXr_rd __class__rr/res z5InstructRequestNormalizerV13._aggregate_tool_messagesrI) r:rrrrrr5rSr rRrre __classcell__r.r.rr/rs .rc@sTeZdZdZed dedBddfddZdedefdd Z dee de fd d Z dS) InstructRequestNormalizerV15zNormalizer for the v15 tokenizer. It reorders tool messages based on the tool call order and builds model settings. Examples: >>> normalizer = InstructRequestNormalizerV15.normalizer() Nr&r1cCstttttt|S)zReturns a normalizer for the V15 instruct request. Args: model_settings_builder: The builder for model settings. Returns: A normalizer for the V15 instruct request. )rrr rrrr4r.r.r/r5s z'InstructRequestNormalizerV15.normalizerr7cCs*|jdurtdt|j|j|S)a%Build model settings using the configured model settings builder. Args: request: The chat completion request. Returns: The built model settings. Raises: InvalidRequestException: If no model settings builder is configured. Nz,model_settings_builder must not be None for )r,rr9r:r=r<r.r.r/r=s  z+InstructRequestNormalizerV15.build_settingscCs*||j}||}|j|d|j|dS)zConverts a chat completion request to an instruct request. Args: request: The chat completion request to convert. Returns: The converted instruct request. Nr)rrXr=r*rrr.r.r/r!s  z9InstructRequestNormalizerV15.from_chat_completion_requestrI) r:rrrrrr5rrr=r rrr.r.r.r/rs rversionr&r1cCstdtt||dS)zCDeprecated in favor to `get_normalizer`, will be removed in 1.12.0.zu`normalizer_for_tokenizer_version` is deprecated and will be removed in 1.12.0. Please call `get_normalizer` instead.rr&)warningswarn FutureWarningget_normalizerrr.r.r/ normalizer_for_tokenizer_version1s  rcCs|tjkrntjkrntjkrnnt}n)tjkr$t}n tjkr+ntjkr1nnt }n tj kr?t }n t ||j |dS)aGets the appropriate normalizer for the given tokenizer version. Args: version: The tokenizer version to get the normalizer for. model_settings_builder: The builder for model settings, or None if unsupported. Returns: The appropriate normalizer for the given tokenizer version. Examples: >>> normalizer = get_normalizer(TokenizerVersion.v1) r4)rv1v2v3rv7rv11v13rv15rrr5)rr&normalizer_clsr.r.r/r=s. " rrI).rArtypingrrrtyping_extensionsrmistral_common.exceptionsr&mistral_common.protocol.instruct.chunkrrr )mistral_common.protocol.instruct.messagesr r r r rrrrrrr(mistral_common.protocol.instruct.requestrrr+mistral_common.protocol.instruct.tool_callsrrr%mistral_common.tokens.tokenizers.baserr7mistral_common.tokens.tokenizers.model_settings_builderrrTrrrrrrr.r.r.r/sH  4   SV,=