o
i @ s d dl Z d dlmZ d dlZd dlmZ d dlmZm Z m
Z
m Z m Z m
Z
d dlmZ d dlmZ d dlmZ eeZG dd d eZdS )
N)Sequence)ChatCompletionRequest)DeltaFunctionCall DeltaMessage
DeltaToolCall ExtractedToolCallInformation FunctionCallToolCall) init_logger)
TokenizerLike)
ToolParserc sr e Zd Zdef fdd ZdededefddZd ed
ed ed e e
d
e e
de e
dede dB fddZ Z
S )Ernie45ToolParser tokenizerc s t | d| _g | _d| _g | _d| _d| _d| _d| _ d| _
| j | _ d| _ t
d t
j| _| js7td
| j| j| _| j| j| _| j| j| _| j| j | _| j| j
| _| j| j | _| j| j| jg| _d | _ d S )
zc
Ernie thinking model format:
abc
def
Fzz
z z z z<0x0A>z.\s*(?P\{.*?\})\s*zUThe model tokenizer must be passed to the ToolParser constructor during construction. N) super__init__current_tool_name_sentprev_tool_call_arrcurrent_tool_idstreamed_args_for_toolthink_end_tokenresponse_start_tokenresponse_end_tokentool_call_start_tokentool_call_end_tokentool_calls_start_token
newline_tokenrecompileDOTALLtool_call_regexmodel_tokenizer
ValueErrorvocabgetthink_end_token_idresponse_start_token_idresponse_end_token_idtool_call_start_token_idtool_call_end_token_idnewline_token_idparser_token_ids_buffer)selfr __class__ [/home/ubuntu/vllm_env/lib/python3.10/site-packages/vllm/tool_parsers/ernie45_tool_parser.pyr s<
zErnie45ToolParser.__init__ model_outputrequestreturnc
C s | j |vr tdg |dS zJ| j|}g }|D ]#}t|}tj|di dd}|t dt
|dd|dd q|d | | j d
}td ||rR|dW S d dW S t
yk td tdg |d Y S w )
NF) tools_called
tool_callscontent arguments) ensure_asciifunctionnamer r< r9 )typer;
Tz,Error in extracting tool call from response.)r r r! findalljsonloadsdumpsr% appendr r findrstrip Exceptionlogger exception) r. r3 r4 tool_call_json_listr7 tool_call_jsontool_call_dictargs_strr8 r1 r1 r2 extract_tool_callsH sN
z$Ernie45ToolParser.extract_tool_calls
previous_text current_text
delta_textprevious_token_idscurrent_token_idsdelta_token_idsNc
s j |7 _ j }| j} | dkrd _ jdkr d} jdkr3t fdd|D r3|d}|}
j|v ra|
d}
|
j} |
| t j d }
|
j } | dkr`|
d | }
n j |v rr|
j } |
d | }
t |dkr|d j
v rt |dkr|d jkr|
d}
t|
r|
dS d dS td| | j}
|
dkrP jdkrd _g _g _t j jkrш ji t j jkst j jkr jd t j jksو |d |
t j |}t |jdkrtd d S |jd }|jjt |jj d
j j< |jj j j< t|j t j|j |j!t"|jj|jj d
d gd } jd
7 _||
t j d _ |S || d _ |d | #d}
t|
rh|
dS d dS )Nr r r c 3 s | ]}| j kV qd S )N)r+ ).0token_idr. r1 r2 s
zAErnie45ToolParser.extract_tool_calls_streaming..r? )r8 z
cur_text = %sz!Failed to extract any tool calls.r= )indexidr> r; )r8 r7 )$r- rE r r allstripr' lstripr lenrfindr r( r, r+ r rH debugr r r rD rN r7 warningr; r< rA rB r9 r8 r rZ r> r rF )r. rO rP rQ rR rS rT r4 cur_text start_idxr8 response_start_idxresponse_end_idxend_idxextracted_tool_calls tool_calldeltar1 rW r2 extract_tool_calls_streamingv s
z.Ernie45ToolParser.extract_tool_calls_streaming)__name__
__module__ __qualname__r r strr r rN r intr rk
__classcell__r1 r1 r/ r2 r
s4 *
. r
)rA collections.abcr regexr 0vllm.entrypoints.openai.chat_completion.protocolr 'vllm.entrypoints.openai.engine.protocolr r r r r r vllm.loggerr
vllm.tokenizersr &vllm.tool_parsers.abstract_tool_parserr rl rH r
r1 r1 r1 r2 s