o .iR/@sXddlmZddlZddlZddlZddlZddlmZmZddl m Z ddl Z ddl m ZddlmZddlmZddlmZmZmZdd lmZmZmZe r[ddlZddlmZned ed Zed ed Zej dkrze!dZ"e"j#Z$e"j%Z&nddl&Z&ddl$Z$eGdddeZ'eGdddeZ(d#ddZ)d$ddZ*d%ddZ+d&d!d"Z,dS)') annotationsN) dataclassfield) TYPE_CHECKING)escape)SamplingParams) LazyLoader)StructuredOutputBackendStructuredOutputGrammarStructuredOutputOptions)OutlinesVocabularyget_outlines_cacheget_outlines_vocabularyoc outlines_core json_schemazoutlines_core.json_schema) rec@s:eZdZddZdd d ZdddZdddZddZdS)OutlinesBackendcCst|j|_t|_dSN)r tokenizer vocabularyr cacheselfrg/home/ubuntu/veenaModal/venv/lib/python3.10/site-packages/vllm/v1/structured_output/backend_outlines.py __post_init__4s  zOutlinesBackend.__post_init__ regex_stringstrrr returnoc.IndexcCs@|jd|}||jvr|j|St||j}||j|<|S)N_)_hashrrIndexinner)rrr cache_keyindexrrr_compile_index8s    zOutlinesBackend._compile_index request_typer grammar_specr cCs|tjkr t|}n+|tjkr|}n#|tjkr.t|}dd|D}dd|d}nt d|d| ||j }|j j durH|j j jnd}t|jtj||dd S) NcSsg|]}t|qSr) regex_escape).0crrr Msz3OutlinesBackend.compile_grammar..(|)z+Invalid request type for Outlines backend (r) max_rollback) vocab_sizeguide)r JSONrbuild_regex_from_schemaREGEXCHOICEast literal_evaljoin ValueErrorr)r vllm_configspeculative_confignum_speculative_tokensOutlinesGrammarr4rGuide)rr*r+regexchoicesr(max_rollback_tokensrrrcompile_grammarDs(         zOutlinesBackend.compile_grammar max_num_seqsint torch.TensorcCs(tj||jddfdtjtjdS)N )dtype pin_memory)torchfullr4int32cuda is_available)rrGrrrallocate_token_bitmask^s z&OutlinesBackend.allocate_token_bitmaskcCsdSrrrrrrdestroyfszOutlinesBackend.destroyN)rr rr r!r")r*r r+r r!r )rGrHr!rI)__name__ __module__ __qualname__rr)rFrTrUrrrrr2s    rc@seZdZUded<eddZded<edddddd Zded <eddddd Zd ed <d%ddZd&ddZ d'ddZ d(ddZ d)d d!Z d"d#Z d$S)*rArHr4F)hashzoc.Guider5cCsdS)NrrrrrroszOutlinesGrammar.)default_factoryreprrYinitnum_processed_tokens)defaultr]r\rYbool_prev_finished request_idr tokens list[int]r!cCs8|j|r|D]}|j||jd7_qdSdS)zAccepts a list of tokens and advances the FSM. Returns True if the FSM was advanced successfully. Returns False if the FSM failed to advance. TF)r5accepts_tokensadvancer^)rrbrctrrr accept_tokensvs  zOutlinesGrammar.accept_tokens num_tokensNonecCs|j||j|8_dSr)r5rollback_stater^)rrjrrrrollbacks zOutlinesGrammar.rollbackcCs6g}|D]}|||j|s||Sq|Sr)appendr5rfpop)rrcacceptedtokrrrvalidate_tokenss  zOutlinesGrammar.validate_tokensbitmaskrIidxcCs(||}|j|||dSr)r5write_mask_intodata_ptrnumel element_size)rrsrtmaskrrr fill_bitmasks zOutlinesGrammar.fill_bitmaskcCs|j}|j}||_|Sr)r5 is_finishedra)rcurrprevrrr is_terminateds zOutlinesGrammar.is_terminatedcCsd|_d|_|jdS)NrF)r^rar5resetrrrrrszOutlinesGrammar.resetN)rbr rcrdr!r`)rjrHr!rk)rcrdr!rd)rsrIrtrHr!rkr!r`)rVrWrX__annotations__rr5r^rarirmrrrzr~rrrrrrAjs       rAparamsrc Cs |jdurdS|j}|jrt|jdS|jrct|jtr;z t|j|j}Wn/tjy:}ztd|d}~wwzt |j}Wnt yW}ztd||d}~wwt |}t|dS|j r}dd|j D}dd|d}t|dS|jrtddS) Nz#Invalid JSON grammar specification.z1Error serializing structured outputs jsonschema: cSsg|]}tt|qSr)r,r )r-choicerrrr/sz?validate_structured_output_request_outlines..r0r1r2zKOutlines structured outputs backend does not support grammar specifications)structured_outputsrCvalidate_regex_is_buildablejson isinstancer loadsJSONDecodeErrorr=dumps Exceptionrr7rr<grammar)r so_paramsschemaepatternrDrCrrr+validate_structured_output_request_outlinessF        rr!r`csd fdd t|dr|jn|}|D]r\}}|tjks$|tjtjfvr'dS|tjkr?t|dr5dS|dr>dSq|tj krad}|d D]}t|rTdS|rZd}qJ|r`dSq|tj tj tj fvrndS|tj krt|d r|dS|d rdSqdS) z@Return True if there's a look-around/anchor before any consumer.r!r`cst|dr|jn|}|D]M\}}|tjtjtjfvrdS|tjkr3|\}}}|dkr2|r2dSq |tjkrK|\}}tfdd|DrJdSq |tj krY|drYdSq dS)z=Return True if subpattern can consume at least one character.dataTrc3s|]}|VqdSrr)r-brsubpattern_consumesrr szE_prefix_needs_context..subpattern_consumes..rF) hasattrr sre_parseLITERALINANY MAX_REPEATBRANCHany SUBPATTERN)parsedrcttypetvalr#mxsubbranchesrrrrs$    z2_prefix_needs_context..subpattern_consumesrTrFreNr)rrrAT sre_constantsASSERT ASSERT_NOTr_prefix_needs_contextrrrrr)rrcrr saw_consumerrrrrrsD          rrkcCst|dr|jn|}|D]J\}}|tjtjfvrtd|tjtjfvr(td|tj kr:|tj tj fvr9tdq |tj krK|dD]}t |qCq |tjkrVt |dq dS)z6Check for regex features unsupported by regex-automatarzBackreferences are unsupported.z&Look-Around assertion are unsupported.z(Unicode word boundaries are unsupported.rerN)rrrGROUPREFGROUPREF_EXISTSr=rrrr AT_BOUNDARYAT_NON_BOUNDARYr_check_unsupportedr)rrcrrbranchrrrrs$       rrr c Cszt|}Wntjy}ztd||d}~wwzt|Wnty8}z td|d|d}~wwt|rDtd|dS)a[ Validates that the input regex is not using unsupported features of the `regex-automata` crate (outlines_core regex engine) and has a universal start state. definition of universal start state used can be found at: https://docs.rs/regex-automata/latest/regex_automata/dfa/trait.Automaton.html#method.universal_start_state zError parsing regex: Nz7Regex uses unsupported feature for structured outputs: um. Only basic matching constructs are supported—lookarounds, backreferences, and unicode boundaries are not.a<Regex does not have a anchored universal start stateThis means that the Regex uses anchors (^) or look-arounds in a way which requires context before any token is matched.structured outputs needs regexes that can match without needing that context. Try rewriting the pattern without using these constructs. Pattern: )rparsererrorr=rr)rrrrrrr*s.  r)rrr)r!rk)rr r!rk)- __future__rr: importlibrsys dataclassesrrtypingrrOrCrr,vllm.sampling_paramsrvllm.utils.import_utilsr'vllm.v1.structured_output.backend_typesr r r vllm.v1.structured_output.utilsr r rrroutlines_core.json_schemarglobals version_info import_module_re_parserr _constantsrrrArrrrrrrrs@       7 ; $ E