o
iJ @ s& d dl Z d dlmZ d dlmZ d dlmZ d dlmZm Z m
Z
m Z m Z m
Z
d dlmZ d dlmZ d dlmZ d d lmZ d d
lmZmZ d d lmZm Z m Z m Z d d l m Z d d
l!m"Z" d dl#m$Z$ d dl%m&Z& d dl'm(Z( d dl)m*Z* e"e+Z,G dd dZ-G dd de-Z.G dd de.Z/dS ) N)abstractmethod)Sequence)cached_property)ResponseFunctionToolCallResponseOutputItemResponseOutputMessageResponseOutputTextResponseReasoningItemToolChoiceFunction)Logprob)Content) TypeAdapter)make_tool_call_id)"ChatCompletionNamedToolChoiceParamChatCompletionRequest) DeltaMessage ExtractedToolCallInformation FunctionCallFunctionDefinition)ResponsesRequest) init_logger)ReasoningParser)
TokenizerLike)
ToolParser) random_uuidc @ s( e Zd ZU dZdZee dB ed< dZee dB ed< de
fddZ e de
eef fd d
ZededB fd d Zejd
edB ddfdd Zede dB fddZejd
e dB ddfddZedee defddZdee dee defddZedee dee fddZe d3d ed ed ed ed ee dB dee f d!d"Z ed ed e eB de edB edB f fd#d$Z!ed%ed&ed'ed(e"e d)e"e d*e"e de#dB fd+d,Z$d e de fd-d.Z%ed ed e de&fd/d0Z'ed%ed&ed'ed(e"e d)e"e d*e"e d e de#dB fd1d2Z(dS )4ParseraW
Abstract Parser class that unifies ReasoningParser and ToolParser into
a single interface for parsing model output.
This class provides a unified way to handle both reasoning extraction
(e.g., chain-of-thought content in tags) and tool call extraction
(e.g., function calls in XML/JSON format) from model outputs.
Subclasses can either:
1. Override the abstract methods directly for custom parsing logic
2. Set `reasoning_parser` and `tool_parser` properties to delegate to
existing parser implementations
Class Attributes:
reasoning_parser_cls: The ReasoningParser class to use (for compatibility
with code that needs the class, not instance).
tool_parser_cls: The ToolParser class to use (for compatibility with
code that needs the class, not instance).
Nreasoning_parser_clstool_parser_cls tokenizerc O s || _ d| _d| _dS )z
Initialize the Parser.
Args:
tokenizer: The tokenizer used by the model. This is required for
token-based parsing operations.
N)model_tokenizer_reasoning_parser _tool_parser)selfr argskwargs r% Q/home/ubuntu/vllm_env/lib/python3.10/site-packages/vllm/parser/abstract_parser.py__init__H s
zParser.__init__returnc C s
| j S )z.Get the vocabulary mapping from tokens to IDs.)r get_vocabr" r% r% r& vocabT s
z Parser.vocabc C | j S )z(The underlying reasoning parser, if any.r r* r% r% r& reasoning_parserY zParser.reasoning_parserparserc C
|| _ d S Nr- r" r0 r% r% r& r. ^
c C r, )z#The underlying tool parser, if any.r! r* r% r% r& tool_parserb r/ zParser.tool_parserc C r1 r2 r5 r3 r% r% r& r6 g r4 input_idsc C dS )a[
Check if the reasoning content ends in the input_ids.
Used by structured engines like `xgrammar` to check if the
reasoning content ends in the model output.
Args:
input_ids: The token IDs of the model output.
Returns:
True if the reasoning content ends in the input_ids.
Nr% r" r7 r% r% r& is_reasoning_endm zParser.is_reasoning_end delta_idsc C s
| |S )a7
Check if the reasoning content ends during a decode step.
Args:
input_ids: The entire model output token IDs.
delta_ids: The last few computed tokens at the current decode step.
Returns:
True if the reasoning content ends in the delta_ids.
)r: )r" r7 r< r% r% r& is_reasoning_end_streaming| s
z!Parser.is_reasoning_end_streamingc C r8 )a+
Extract content token IDs from the input_ids.
This extracts the non-reasoning content (e.g., everything after
the tag).
Args:
input_ids: The token IDs of the model output.
Returns:
The extracted content token IDs.
Nr% r9 r% r% r& extract_content_ids r; zParser.extract_content_idsFrandom model_outputrequestenable_auto_toolstool_call_id_typelogprobsc C r8 )a
Extract reasoning, content, and tool calls from a complete
model-generated string and return as ResponseOutputItem objects.
Used for non-streaming responses where we have the entire model
response available before sending to the client.
Args:
model_output: The complete model-generated string.
request: The request object used to generate the output.
enable_auto_tools: Whether to enable automatic tool call parsing.
tool_call_id_type: Type of tool call ID generation ("random", etc).
logprobs: Pre-computed logprobs for the output text, if any.
Returns:
A list of ResponseOutputItem objects.
Nr% )r" r@ rA rB rC rD r% r% r& extract_response_outputs r; z Parser.extract_response_outputsc C r8 )a
Extract reasoning content from a complete model-generated string.
Used for non-streaming responses where we have the entire model
response available before sending to the client.
Args:
model_output: The complete model-generated string.
request: The request object used to generate the output.
Returns:
A tuple of (reasoning_content, response_content).
Nr% r" r@ rA r% r% r& extract_reasoning r; zParser.extract_reasoning
previous_text current_text
delta_textprevious_token_idscurrent_token_idsdelta_token_idsc C r8 )a"
Extract reasoning content from a streaming delta message.
Args:
previous_text: Text from all previous tokens.
current_text: Text including the current delta.
delta_text: The new text in this delta.
previous_token_ids: Token IDs from previous generation.
current_token_ids: All token IDs including current.
delta_token_ids: The new token IDs in this delta.
Returns:
A DeltaMessage with reasoning and/or content fields, or None.
Nr% r" rH rI rJ rK rL rM r% r% r& extract_reasoning_streaming r; z"Parser.extract_reasoning_streamingc C s |S )a9
Adjust the request parameters for tool calling.
Can be overridden by subclasses to modify request parameters
(e.g., setting structured output schemas for tool calling).
Args:
request: The original request.
Returns:
The adjusted request.
r% )r" rA r% r% r& adjust_request s
zParser.adjust_requestc C r8 )a`
Extract tool calls from a complete model-generated string.
Used for non-streaming responses.
Args:
model_output: The complete model-generated string.
request: The request object used to generate the output.
Returns:
ExtractedToolCallInformation containing the tool calls.
Nr% rF r% r% r& extract_tool_calls r; zParser.extract_tool_callsc C r8 )a5
Extract tool calls from a streaming delta message.
Args:
previous_text: Text from all previous tokens.
current_text: Text including the current delta.
delta_text: The new text in this delta.
previous_token_ids: Token IDs from previous generation.
current_token_ids: All token IDs including current.
delta_token_ids: The new token IDs in this delta.
request: The request object.
Returns:
A DeltaMessage with tool_calls field, or None.
Nr% r" rH rI rJ rK rL rM rA r% r% r& extract_tool_calls_streaming r; z#Parser.extract_tool_calls_streamingFr? N))__name__
__module__ __qualname____doc__r typer __annotations__r r r r' r dictstrintr+ propertyr. setterr6 r listboolr: r= r> r r r rE r tuplerG r r rO rP r rQ rS r% r% r% r& r . s
r c @ s$ e Zd ZdZdedeeB deedB edB f fddZ d deded
e d ed e
e dB de
e f d
dZ
dededB d
e dee
e edB f fddZdedededee dee dee dedB fddZdededefddZdedededee dee dee dededB fd d ZdS ) DelegatingParsera
A Parser implementation that delegates to separate ReasoningParser and
ToolParser instances.
This is the recommended base class for creating model-specific parsers
that combine existing reasoning and tool parser implementations.
Subclasses should set `self._reasoning_parser` and `self._tool_parser`
in their `__init__` method.
If either parser is None, the corresponding methods will return default
values (no reasoning extraction, no tool calls).
r@ rA r( Nc C s | j d u r d |fS | j ||S r2 )r rG rF r% r% r& rG 0 s
z"DelegatingParser.extract_reasoningFr? rB rC rD c C s | ||\}}| j|||d\}}g } |r-tdt g dt|ddgd d}
| |
|rJt|g d|d} td t | gd
d d d
} | | |rvt|D ]%\}
}t dt |j
r`|j
nt ||j |
ddd |j |j
d}| | qP| S )N)rA contentrB rs_ reasoningreasoning_text)textrY )idsummaryrY rd status output_text)rh annotationsrY rD msg_ assistant completedmessage)ri rd rolerk rY fc_)id_type func_nameidx
function_call)ri call_idrY rk name arguments)rG _parse_tool_callsr r ResponseReasoningTextContentappendr r enumerater ri r ry rz )r" r@ rA rB rC rD rf rd
tool_callsoutputsreasoning_item
res_text_part message_itemhistory_tool_call_cnt tool_calltool_call_itemr% r% r& rE 9 sd
z)DelegatingParser.extract_response_outputsrd c C sN g }|j r t|j tr |dusJ |t|j j|d |dfS |j r?t|j tr?|dus/J |t|j jj|d |dfS |j dkra|dusJJ tt t
|}| dd |D |dfS | j
dur|r|j dksr|j du r| j
j|dur{|nd|d}|dur|jr| d d |jD |j}|r| dkrd}||fS g |fS )
a4
TODO(qandrew): merge _parse_tool_calls_from_content
for ChatCompletions into this function
Parse tool calls from content based on request tool_choice settings.
Returns:
A tuple of (function_calls, remaining_content) if tool calls
were parsed
Nry rz requiredc s s* | ]}t |jtj|jd ddV qdS )F) ensure_asciir N)r ry jsondumps
parameters.0r r% r% r& s
z5DelegatingParser._parse_tool_calls..auto )rA c s s( | ]}t |j|jj|jjd V qdS ))ri ry rz N)r ri functionry rz r r% r% r& r s
) tool_choice
isinstancer
r} r ry r r r
r` r
validate_jsonextendr! rQ tools_calledr rd strip)r" rA rd rB function_callsr tool_call_inforemaining_contentr% r% r& r{ sN
z"DelegatingParser._parse_tool_callsrH rI rJ rK rL rM c C s* | j d u r
t|dS | j ||||||S )N)rd )r r rO rN r% r% r& rO s
z,DelegatingParser.extract_reasoning_streamingc C s& | j d u r tdg |dS | j ||S )NF)r r rd )r! r rQ rF r% r% r& rQ s
z#DelegatingParser.extract_tool_callsc C s& | j d u rd S | j |||||||S r2 )r! rS rR r% r% r& rS s
z-DelegatingParser.extract_tool_calls_streamingrT )rU rV rW rX r\ r r rb rG ra r` r r rE r r{ r r] r rO r rQ rS r% r% r% r& rc " s
K
K
rc c sP e Zd ZU dZdZee dB ed< dZee dB ed< de
f fdd Z Z S )_WrappedParsera
A DelegatingParser subclass that instantiates parsers from class attributes.
This class is used to dynamically create a parser that wraps individual
ReasoningParser and ToolParser classes. The class attributes
`reasoning_parser_cls` and `tool_parser_cls` should be set before
instantiation.
Usage:
_WrappedParser.reasoning_parser_cls = MyReasoningParser
_WrappedParser.tool_parser_cls = MyToolParser
parser = _WrappedParser(tokenizer)
Nr r r c sH t | | jjd ur| j|| _| jjd ur"| j|| _d S d S r2 )superr' __class__r r r r! )r" r r r% r& r' s z_WrappedParser.__init__)
rU rV rW rX r rY r rZ r r r r'
__classcell__r% r% r r& r s
r )0r abcr collections.abcr functoolsr openai.types.responsesr r r r r r
+openai.types.responses.response_output_textr .openai.types.responses.response_reasoning_itemr r| pydanticr
vllm.entrypoints.chat_utilsr 0vllm.entrypoints.openai.chat_completion.protocolr r 'vllm.entrypoints.openai.engine.protocolr r r r *vllm.entrypoints.openai.responses.protocolr vllm.loggerr $vllm.reasoning.abs_reasoning_parsersr vllm.tokenizersr &vllm.tool_parsers.abstract_tool_parserr
vllm.utilsr rU loggerr rc r r% r% r% r& s. u d