o ۾i}h@sddlmZddlZddlmZddlmZmZm Z m Z m Z m Z ddl mZddlmZddlmZeeZGdd d eZdS) )SequenceN)ChatCompletionRequest)DeltaFunctionCall DeltaMessage DeltaToolCallExtractedToolCallInformation FunctionCallToolCall) init_logger) TokenizerLike) ToolParsercseZdZdeffdd ZdedeeeeffddZdd d Z dd d Z d ede de fddZ dedededeedeedeede dedBfddZZS)KimiK2ToolParser tokenizercs:t|d_g_d_g_d_d_d_d_ d_ d_ d_ d_ dd g_dd g_d _d _td tj_td_td_jsUtdjj _jj _fddjD_fddjD_jj_ jj_!jdusjdurt"ddS)NFiri z<|tool_calls_section_begin|>z<|tool_calls_section_end|>z<|tool_call_section_begin|>z<|tool_call_section_end|>z<|tool_call_begin|>z<|tool_call_end|>z<\|tool_call_begin\|>\s*(?P[^<]+:\d+)\s*<\|tool_call_argument_begin\|>\s*(?P(?:(?!<\|tool_call_begin\|>).)*?)\s*<\|tool_call_end\|>zV(?P.+:\d+)\s*<\|tool_call_argument_begin\|>\s*(?P.*)z(?P.+:\d+)\s*zUThe model tokenizer must be passed to the ToolParser constructor during construction.c$g|]}j|durqSNvocabget.0variantselftidY/home/ubuntu/.local/lib/python3.10/site-packages/vllm/tool_parsers/kimi_k2_tool_parser.py T z-KimiK2ToolParser.__init__..crrrrrrrrYrzQKimi-K2 Tool parser could not locate tool call start/end tokens in the tokenizer!)#super__init__current_tool_name_sentprev_tool_call_arrcurrent_tool_idstreamed_args_for_toolin_tool_section token_bufferbuffer_max_sizesection_char_countmax_section_chars_buffer_overflow_loggedtool_calls_start_tokentool_calls_end_tokentool_calls_start_token_variantstool_calls_end_token_variantstool_call_start_tokentool_call_end_tokenrecompileDOTALLtool_call_regexstream_tool_call_portion_regexstream_tool_call_name_regexmodel_tokenizer ValueErrorrrtool_calls_start_token_idtool_calls_end_token_idtool_calls_start_token_idstool_calls_end_token_idstool_call_start_token_idtool_call_end_token_id RuntimeError)rr __class__rrr!sd      zKimiK2ToolParser.__init__textreturncCs^d}d}|}|jD]}||vr||d}d}q |jD]}||vr)||d}d}q|||fS)z Check for section begin/end markers in text and strip them. Returns: (cleaned_text, found_section_begin, found_section_end) FrT)r.replacer/)rrC found_begin found_endcleanedrrrr_check_and_strip_markersks     z)KimiK2ToolParser._check_and_strip_markersNcCsd|_d|_d|_dS)z&Reset state when exiting tool section.FrrN)r&r'r)rrrr_reset_section_states z%KimiK2ToolParser._reset_section_statecCs.|d|_g|_d|_g|_tddS)z Reset all streaming state. Call this between requests to prevent state leakage when parser instance is reused. FrzStreaming state resetN)rKr"r#r$r%loggerdebugrJrrrreset_streaming_states z&KimiK2ToolParser.reset_streaming_state model_outputrequestc Cs|j|vr tdg|dSzJ|j|}td|g}|D] }|\}}|dddd}|t|dt ||d d q|d| |j} td || rR| dWSddWSt ykt d tdg|dYSw) NF) tools_called tool_callscontentzfunction_call_tuples: %s:r.rfunction)name arguments)idtyperVTz,Error in extracting tool call from response.) r,rr5findallrLrMsplitappendr rfind Exception exception) rrOrPfunction_call_tuplesrRmatch function_id function_args function_namerSrrrextract_tool_callssD       z#KimiK2ToolParser.extract_tool_calls previous_text current_text delta_textprevious_token_idscurrent_token_idsdelta_token_idsc(sltd|td|d}|j|7_t|j|jkr6|js*td|jd|_|j|j dd|_||j\} } } | rR|jsRtdd|_| |_d|_ | r|jrtd |j |v} | rnd}td | |_n8td | d } |j D]}||vr| |d }t|d kr|d } nq|| rt| dStd dS| |_tfdd|jD}|s|jstdt|dS||\}}}|jr|j t|7_ |j |jkrtd|j| t|r|dSd dSz1||j}||j }|j}|j }d}d}||krA||krA|j|vrA|jr6|dkr6td|td dWStdt|dWS|j|vrttd||}| |jd |jd}| |jd}| |jd}||kr||krt|d kr| |jd}nd}d}d}|jd 7_d|_|jd td|jn||kr||kr| |jd}d}n||kr[||kr[|jdust|jdkrtd|r|jr| WdS|j|jd}|rZ|tur| d!dn|}d|vr|r|jr| WdS|"d}|d|d}td||j|j|7<|rG|jrGtd| tt#|jt$|dj%dd d!gd"WSn8|jrqtd#|rk| td dWS|&|jd }|&|jd }tg|d$}|r|jr| |WSt'}|r|j()|}|r|*\}} | d%d d&d}!||d'<|!|d(<| |d<n1|j+)|}"|"r|"*\}#|# d%d d&d}!|#|d'<|!|d(<d |d<ntd)WdS|js(|durWdS|d(}$|d'}|$r%d|_tt#|jd*|t$|$d+j%dd d,gd"WSWdS|durF|jr4WdS|durAt|d}|WSd}|WStd-|jt|j|jkr\|ji|j|jd}%|d}&td.|%td/|&|&s|%std0|d}nv|&s|%rt,d1d}nh|&r|%stt#|jt$|&dj%dd d!gd"}|&|j|j<nI|&r|%rt-|tr|&|%krt|&t|%kr|&.|%r|&t|%d}'td2|tt#|jt$|'dj%dd d!gd"}|&|j|j<nd}|jt|jd kr ||j|j<n|j||r#|jr#td| |WSt/y5t0d3YdSw)4Nzdelta_text: %szdelta_token_ids: %sFzxToken buffer exceeded max size (%d bytes), flushing excess. This may indicate very long markers or unusual tokenization.TzEntering tool sectionrzDetected section end markerz3Deferring section exit: tool_call_end in same chunkzExiting tool sectionr)rSc3s|]}|vVqdSrr)rrrkrr s z@KimiK2ToolParser.extract_tool_calls_streaming..zNo tool call tokens found!zdTool section exceeded max length (%d chars), forcing exit. This may indicate malformed model output.z2In tool section before first tool, suppressing: %sz/Generating text content! skipping tool parsing.z!tool_call_end_token in delta_textrzStarting on a new tool %sz/attempting to close tool call, but no tool callrXzutf-8unicode_escapez"}z@Finishing tool and found diff that had not been streamed yet: %sz Completing deferred section exit)rX) exclude_none)indexrV)rRz,In tool section, suppressing text generation)rRrSrTrUrYrWzNot enough tokenrV)rW)rsrZrYrVz,Trying to parse current tool call with ID %szdiffing old arguments: %szagainst new ones: %szSkipping text %s - no argumentszSshould be impossible to have arguments reset mid-call. skipping streaming anything.z got diff %sz+Error trying to handle streaming tool call.)1rLrMr'lenr(r+warningrIr&r)r?rKr/r\stripranyr<r*countr>r1r0rstriplstripr$r"r%r]r#rstrencodedecoderindexrr model_dumprEdictr6rbgroupsr7error isinstance startswithr_r`)(rrgrhrirjrkrlrPdeferred_section_exit buffered_textfound_section_beginfound_section_end has_tool_endpost_section_contentrpartshas_section_token_prev_tool_start_countprev_tool_end_countcur_tool_start_countcur_tool_end_counttool_call_portion text_portion full_textdeltadiffend_locrCcurrent_tool_callcurrent_tool_call_matchestool_id tool_args tool_namecurrent_tool_call_name_matches tool_id_strreprev_arguments cur_argumentsdelta_argumentsrrorextract_tool_calls_streamings                                            %                                z-KimiK2ToolParser.extract_tool_calls_streaming)rDN)__name__ __module__ __qualname__r r!r{tupleboolrIrKrNrrrfrintrr __classcell__rrrArr s:M   1 r )collections.abcrregexr20vllm.entrypoints.openai.chat_completion.protocolr'vllm.entrypoints.openai.engine.protocolrrrrrr vllm.loggerr vllm.tokenizersr &vllm.tool_parsers.abstract_tool_parserr rrLr rrrrs