o پi$@sdZddlZddlZddlmmZddlmZe e Z dej j dedefddZdej j ded edefd d Z ddej j ded edefd d ZdefddZdedefddZ ddedededededededefddZdS)zTemplate utilities for Jinja template processing. This module provides utilities for analyzing and processing Jinja chat templates, including content format detection and message processing. N) ImageDatanodevarnamereturncCs&t|tjjr|jdko|j|kSdS)z5Check if node is a variable access like {{ varname }}loadF) isinstancejinja2nodesNamectxname)rrr Z/home/ubuntu/.local/lib/python3.10/site-packages/sglang/srt/parser/jinja_template_utils.py_is_var_accesssrkeycCs^t|tjjrt|j|ot|jtjjo|jj|kSt|tjj r-t|j|o,|j |kSdS)zSCheck if node is an attribute access like {{ varname['key'] }} or {{ varname.key }}F) rrr GetitemrrargConstvalueGetattrattrrrrr r r_is_attr_access#s  rcCst|tjjr|jduot|j||St|tjjr!t|j||St|tjjr7t|jtjj r7t|j||S|r?t |||St ||S)zACheck if node accesses varname or varname[key] with filters/testsN) rrr Filterr_is_var_or_elems_accessTestrrSlicerrrr r rr2s  r chat_templatec CsPz t|}|j|WSty'}ztd|WYd}~dSd}~ww)z+Try to parse the Jinja template into an ASTz%Error when compiling Jinja template: N) hf_chat_utils_compile_jinja_template environmentparse Exceptionloggerdebug)rjinja_compileder r r_try_extract_astGs r'c stfdddDr dSt}|durdSz*|tjjD]}|j}t|ddr.WdSt|d ds:t|d dr>WdSqWdSty\}zt d |WYd}~dSd}~ww) ut Detect whether a chat template expects 'string' or 'openai' content format. - 'string': content is a simple string (like DeepSeek templates) - 'openai': content is a list of structured dicts (like Llama4 templates) Detection logic: - If template has loops like {%- for content in message['content'] -%} → 'openai' - Otherwise → 'string' c3s|]}|vVqdSNr ).0keywordrr r ]s z7detect_jinja_template_content_format..)imageaudiovideovisionopenaiNstringmessagecontentmsgmz*Error when parsing AST of Jinja template: ) anyr'find_allrr Foriterrr"r#r$)r jinja_astloop_ast loop_iterr&r r+r$detect_jinja_template_content_formatQs2   r>Fmsg_dictcontent_format image_data video_data audio_data modalitiesuse_dpsk_v32_encodingc CsVt|dtsdd|DS|dks|rg}g}|dD]} t| tr| d} | dkra| dp5i} | dd} |t| d | d d | d | d rY|| d |ddiq| dkr| dpki} | dd} | dur|| dd n || d | d| d r|| d |ddiq| dkr|| dd |ddiq| dkr|r|| dq|| qdd|D}|r|rd|nd|d<|S||d<|S|dkr$g}|dD]} t| tr| ddkr|| dq|}|rd|nd|d<dd|D}|St d|)aN Process message content based on detected template format. Args: msg_dict: Message dictionary with content content_format: 'string' or 'openai' (detected via AST analysis) image_data: List to append extracted image URLs video_data: List to append extracted video URLs audio_data: List to append extracted audio URLs modalities: List to append modalities use_dpsk_v32_encoding: If True, extract multimodal data and convert content to string (for DeepSeek-V3.2 encoding) Returns: Processed message dictionary r4cSi|] \}}|dur||qSr(r r)kvr r r z7process_content_for_template_format..r1type image_urlmax_dynamic_patchNurldetailauto)rOrPrNrDr- video_url)rOrNr/ audio_urlr.textcSs&i|]\}}|dur|dkr||qS)Nr4r rGr r rrJs  r2cSrFr(r rGr r rrJrKzInvalid content format: ) rgetlistitemsdictappendrjoincopy ValueError)r?r@rArBrCrDrEprocessed_content_parts text_partschunk chunk_type image_objmdp video_objnew_msgr r r#process_content_for_template_format{sz            rgr()F)__doc__loggingr&transformers.utils.chat_template_utilsutilschat_template_utilsrsglang.srt.utilsr getLogger__name__r#r Nodestrboolrrrr'r>rZrXrgr r r rsL    1