o -iJ&@sddlZddlZddlmZmZddlmZddlmZddl m Z ddl m Z m Z ddlmZmZddlmZdd lmZdd lmZdd lmZdd lmZdd lmZeeZGdddZGdddZ dS)N)CallableSequence)cached_property)"ResponseFormatTextJSONSchemaConfig)ChatCompletionRequest) DeltaMessageExtractedToolCallInformation)ResponsesRequestResponseTextConfig) init_logger)StructuredOutputsParams) TokenizerLike)get_json_schema_from_tools) is_list_of)import_from_pathc@seZdZdZdefddZedeee ffddZ de de fd d Z d ede de fd d Zdedededee dee dee de dedBfddZdS) ToolParserz Abstract ToolParser class that should not be used directly. Provided properties and methods should be used in derived classes. tokenizercCs"g|_d|_d|_g|_||_dS)NF)prev_tool_call_arrcurrent_tool_idcurrent_tool_name_sentstreamed_args_for_toolmodel_tokenizer)selfrrc/home/ubuntu/veenaModal/venv/lib/python3.10/site-packages/vllm/tool_parsers/abstract_tool_parser.py__init__)s  zToolParser.__init__returncCs |jSN)r get_vocab)rrrrvocab2s zToolParser.vocabrequestcCsj|js|St|j|jd}|dur3t|trt|d|_d|_t|tr3t |_ t d|dddd|j _ |S) zK Static method that used to adjust the request parameters. ) tool_choicetoolsN)jsontool_calling_response json_schemaz Response format for tool callingT)nameschematype descriptionstrict) r#rr" isinstancerr structured_outputsresponse_formatr r textrformat)rr!json_schema_from_toolrrradjust_request8s*   zToolParser.adjust_request model_outputcCtd)a, Static method that should be implemented for extracting tool calls from a complete model-generated string. Used for non-streaming responses where we have the entire model response available before sending to the client. Static because it's stateless. z?AbstractToolParser.extract_tool_calls has not been implemented!NotImplementedError)rr3r!rrrextract_tool_callsVs zToolParser.extract_tool_calls previous_text current_text delta_textprevious_token_idscurrent_token_idsdelta_token_idsNcCr4)av Instance method that should be implemented for extracting tool calls from an incomplete response; for use when handling tool calls and streaming. Has to be an instance method because it requires state - the current tokens/diffs, but also the information about what has previously been parsed and extracted (see constructor) zIAbstractToolParser.extract_tool_calls_streaming has not been implemented!r5)rr8r9r:r;r<r=r!rrrextract_tool_calls_streamingdsz'ToolParser.extract_tool_calls_streaming)__name__ __module__ __qualname____doc__r rrdictstrintr rr2rr7rrr>rrrrr"s<   rc@sLeZdZUdZiZeeeefe d<iZ eee eeffe d<e dedeefddZ e dedeefdd Ze dd eed eeeBd Bdedd fddZe dedededd fddZe ddeeeBd Bded eed BdeeeeegeefBfddZe deefddZe dedd fddZd S)ToolParserManagerz Central registry for ToolParser implementations. Supports two modes: - Eager (immediate) registration via `register_module` - Lazy registration via `register_lazy_module` tool_parsers lazy_parsersr'rcCs8||jvr |j|S||jvr||Std|d)z Retrieve a registered or lazily registered ToolParser class. If the parser is lazily registered, it will be imported and cached on first access. Raises KeyError if not found. z Tool parser 'z ' not found.)rGrH_load_lazy_parserKeyErrorclsr'rrrget_tool_parsers   z!ToolParserManager.get_tool_parserc Cs||j|\}}z!t|}t||}t|ts!t|d|d||j|<|WSty=}z t d|||d}~ww)z+Import and register a lazily loaded parser.z in z is not a ToolParser subclass.z2Failed to import lazy tool parser '%s' from %s: %sN) rH importlib import_modulegetattr issubclassr TypeErrorrG Exceptionlogger exception)rLr' module_path class_namemod parser_clserrrrIs(    z#ToolParserManager._load_lazy_parserNTmodule module_nameforcecCst|tstdt||dur|j}t|tr|g}n t|tr&|}ntd|D]}|sD||jvrD|j|}t |d|j ||j|<q,dS)z(Register a ToolParser class immediately.z/module must be subclass of ToolParser, but got Nz,module_name must be str, list[str], or None.z is already registered at ) rQrrRr)r?r,rDrrGrJr@)rLr[r\r] module_namesr'existedrrr_register_modules"      z"ToolParserManager._register_modulerVrWcCs||f|j|<dS)a Register a lazy module mapping. Example: ToolParserManager.register_lazy_module( name="kimi_k2", module_path="vllm.tool_parsers.kimi_k2_parser", class_name="KimiK2ToolParser", ) N)rH)rLr'rVrWrrrregister_lazy_modules z&ToolParserManager.register_lazy_modulecs\t|tstdt||durj||d|Sdttdttffdd }|S)a) Register module immediately or lazily (as a decorator). Usage: @ToolParserManager.register_module("kimi_k2") class KimiK2ToolParser(ToolParser): ... Or: ToolParserManager.register_module(module=SomeToolParser) z!force must be a boolean, but got N)r[r\r]objrcsX|j}|j}ttrg}ndurttr}n|g}|D] }||fj|<q |Sr)r@r?r,rDrrH)rbrVrWnamesnrKrr _decorators z5ToolParserManager.register_module.._decorator)r,boolrRr)r`r)rLr'r]r[rerrKrregister_modules  z!ToolParserManager.register_modulecCs tt|jt|jBS)z?Return names of all eagerly and lazily registered tool parsers.)sortedsetrGkeysrH)rLrrrlist_registereds z!ToolParserManager.list_registered plugin_pathcCsLtjtj|d}zt||WdSty%td||YdSw)z6Import a user-defined parser file from arbitrary path.rz#Failed to load module '%s' from %s.N)ospathsplitextbasenamerrSrTrU)rLrlr\rrrimport_tool_parser s  z$ToolParserManager.import_tool_parser)NT)NTN)r?r@rArBrGrCrDr)r__annotations__rHtuple classmethodrMrIlistrfr`rarrgrkrqrrrrrFzsN     -rF)!rNrmcollections.abcrr functoolsr>openai.types.responses.response_format_text_json_schema_configr0vllm.entrypoints.openai.chat_completion.protocolr'vllm.entrypoints.openai.engine.protocolrr*vllm.entrypoints.openai.responses.protocolr r vllm.loggerr vllm.sampling_paramsr vllm.tokenizersr vllm.tool_parsers.utilsrvllm.utils.collection_utilsrvllm.utils.import_utilsrr?rTrrFrrrrs"         X