o -i> @sfUddlZddlZddlmZddlZdZdZee d<dZ ee d<dZ ee d <d Z ee d <d Z ee d <dZee d<dZee d<dZee d<dZdZee d<dZee d<dZdZee d<dedefddZd d!Zd"d#Zd$d%Zd&eeefdefd'd(Zd)ed*eeeeeffdeeeffd+d,Zd-eeeeeeefBfdefd.d/Zd0eeeefde fd1d2Z!d3e d0eeeefd4edefd5d6Z" dHd0eeeefd7e dBdeeeeffd8d9Z#  : :dId0eeeefd4ed;eeeefdBdd?Z%d3e d@edAeedee edeBffdBdCZ&d3e d@efdDdEZ'd@ed4efdFdGZ(dS)JN)Anya## Tools You have access to a set of tools you can use to answer the user's question. You can invoke functions by writing a "<{dsml_token}function_calls>" block like the following as part of your reply to the user: <{dsml_token}function_calls> <{dsml_token}invoke name="$FUNCTION_NAME"> <{dsml_token}parameter name="$PARAMETER_NAME" string="true|false">$PARAMETER_VALUE ... <{dsml_token}invoke name="$FUNCTION_NAME2"> ... String and scalar parameters should be specified as is without any escaping or quotes, while lists and objects should use JSON format. The "string" attribute should be set to "true" for string type parameters and "false" for other types (numbers, booleans, arrays, objects). If the thinking_mode is enabled, then after function results you should strongly consider outputting a thinking block. Here is an example: <{dsml_token}function_calls> ... ... {thinking_start_token}...thinking about results{thinking_end_token} Here are the functions available in JSONSchema format: {tool_schemas} u<｜begin▁of▁sentence｜> bos_tokenu<｜end▁of▁sentence｜> eos_tokenzthinking_start_tokenzthinking_end_tokenu ｜DSML｜ dsml_tokenz {content}system_msg_templateu&<｜User｜>{content}<｜Assistant｜>user_msg_templateu;{reasoning}{content}{tool_calls}<｜end▁of▁sentence｜>assistant_msg_templatez{reasoning_content}zX## Response Format: You MUST strictly adhere to the following schema to reply: {schema}response_format_templatezD<{dsml_token}invoke name="{name}"> {arguments} tool_call_templatezG<{dsml_token}function_calls> {tool_calls} z {content}tool_output_templatevaluereturncCs2ztj|ddWStytj|ddYSw)NF) ensure_asciiT)jsondumps Exception)rrb/home/ubuntu/veenaModal/venv/lib/python3.10/site-packages/vllm/tokenizers/deepseek_v32_encoding.pyto_json=s  rcCdd|DS)NcSsg|]}|dqS)functionr).0toolrrr Ez,tools_from_openai_format..r)toolsrrrtools_from_openai_formatDsrcCr)NcSs&g|]}|dd|dddqS)rname argumentsrr rr tool_callrrrrIs  z1tool_calls_from_openai_format..r tool_callsrrrtool_calls_from_openai_formatHsr&cCr)NcSs$g|]}d|d|dddqS)rrr r!)typerrr"rrrrSsz/tool_calls_to_openai_format..rr$rrrtool_calls_to_openai_formatRsr(r#cCsd}g}t|dtrt|d}n|d}|D]"\}}|jt|t|tr*dndt|tr2|nt|d}||qd |S)NzU<{dsml_token}parameter name="{key}" string="{is_str}">{value}r truefalse)rkeyis_strr ) isinstancestrrloadsitemsformatrrappendjoin)r#p_dsml_template P_dsml_strsr kv p_dsml_strrrrencode_arguments_to_dsml_s  r: tool_name tool_argscsFdtdtdtfddddfdd |Dd }t||d S) Nr+rstringcSs"|dkrt|}t|d|S)Nr)z: r)r+rr=rrr _decode_valuewsz/decode_dsml_to_arguments.._decode_value{z, cs"g|] \}\}}|||dqS))r=r)rr7r8r,r?rrrs"z,decode_dsml_to_arguments..}r!)r/r4r1dict)r;r<tool_args_jsonrrArdecode_dsml_to_argumentsts rErcCs&dd|D}tjd|tttdS)NcSsg|]}t|qSrr>)rtrrrrrz render_tools..r-) tool_schemasrrr)TOOLS_SYSTEM_TEMPLATEr2r4rrr)r tools_jsonrrr render_toolssrJmessagescCs>d}tt|dddD]}||ddvr|}|Sq |S)Nrole)user developer)rangelenget)rKlast_user_indexidxrrrfind_last_user_indexsrVindex thinking_modecCsd|kr t|ksntd|dt||dvr%td|dd}||}t|}|d}|d }|d }|d } |d } |d pQ|d} |dd} |r^t|}| rdt| } |dkr|tj|pndd7}|r||dt|7}| r|dt jt | d7}|S|dkr|std|d|d} |r| dt|7} | r| dt jt | d7} | d|7} |t j| d7}||kr|dkr|t 7}|S|t 7}|S|dkr|t j|d7}||kr|dkr|t 7}|S|t 7}|S|dkr|d}||}|dkr(|ddkr(|d8}||}|dkr(|ddks|dksD|dkr:|ddksDtd|d|||}|d }|rWt||ks[td |dkrd|d!7}|tj|d7}|t|kr|d"7}||kr|dkr|dt 7}|S|dt 7}|S|dkr|}d}d}| rd#d$| D} |dtjtd%| d&7}|pd}|dkr||kr| s| std'|d(|d)tj| pdd*t }| s| r||7}|S|tj|||d+7}|Std,|)-NrzIndex z* out of range for messages list of length )chatthinkingzInvalid thinking_mode ``rNcontentrresponse_formatr% reasoningreasoning_contentprefixFsystem)r]z )schemarPzInvalid message for role `z`: z # The user's message is: {}rZrOrrM assistantzInvalid messages at z: z#No tool calls but found tool outputz z cSs&g|]}tjt|dt|dqS)r)rrr )r r2rrSr:r"rrrrsz"render_message..r-)rr%zThinkingMode: z8, invalid message without reasoning_content/tool_calls `z` after last user message)r`)r_r]r%zUnknown role: )rR ValueErrorrVrSrr&rr2rJr rr rrr tool_calls_templaterr4thinking_templater NotImplementedError)rWrKrXpromptmsg last_user_idxrNr]rr^r%r` is_prefixcontent_developerprev_assistant_idx assistant_msgtool_call_orderassistant_tool_calls thinking_parttool_calls_contentsummary_contentrrrrender_messages       i USMK      - +      rurkcCsg}|dur t|n|}t|D]1\}}|d}|dvs!||kr'||q|dkrAt|}|dd|dd||q|S)NrN)rOrbrrdr`r_)rV enumeraterSr3copypop)rKrkmessages_wo_thinkingrUrjrNmsg_wo_thinkingrrrdrop_thinking_messages's      r{Tcontext drop_thinkingadd_default_bos_tokencCsp|r|ng}||}|rt|dkrtnd}|dkr |r t|}tt|D]}|t|t|||d7}q&|S)Nrr\rZ)rX)rRrr{rQru)rKrXr|r}r~ full_messagesrirUrrrencode_messages=s   rtextstopcCsvt|}d}|D]}|||}|dkr||kr|}|}q|r.|||}|t|||fS||d}t||dfS)NrL)rRfind)rWrrmin_pos matched_stopsposr]rrr_read_until_stopTs   rcCsg}d}dtd}|t|krt||dtd|g\}}}|dkr'td||kr,n|dur4tdt||dtddtdg\}}}tjd |tjd }t|d krZtd |d }i} |dtdkrt||dtdg\}} }tjd| tjd } t| d krtd| d \} } }| | vrtd|| f| | <t||dtddtdg\}}}|dkrtd|dtdksht|| d}|||t|ks|||fS)Nz<invokez> zTool call format errorzMissing special token parameterz^\s*name="(.*?)">\n$)flagsrMzTool name format errorr/z,^ name="(.*?)" string="(true|false)">(.*?)<$zParameter format errorzDuplicate parameter name)r;r<) rrRr RuntimeErrorrefindallDOTALLrEr3)rWrr% stop_tokentool_calls_end_token_tool_name_content p_tool_namer;r< param_contentparam_kv param_namer= param_valuer]r#rrrparse_tool_callshs`            4rc Cs,ddg}}}d\}}dtd}|dkd}} |r0t||t|g\}} }| }|tkr0tdt||t|g\}} }| }||krDd} n|tkrLtd | rft||\}}}t||tg\}} }| rftd t||krr|tdfvsvtd tttttfD]} | |vs| |vrtd q}d |||t |dS)Nr\)rNz <function_callsrZFzInvalid thinking formatTzInvalid summary formatz#Unexpected content after tool callszUnexpected content at endz#Unexpected special token in contentrd)rNr]r`r_r%) rrrrrrrRrrr() rrXrtr`r%rWrtool_calls_start_token is_thinkingis_tool_calling content_deltatool_ends_textsp_tokenrrr"parse_message_from_completion_textsR     r)N)NTT))rwrtypingrregexrrHrr/__annotations__rrrrrr r rgr r rfr rrr&r(rCr:tuplerElistrJintrVrur{boolrrrrrrrrs                 *      ?