o پi0 @sdZddlZddlmZmZmZmZmZmZm Z ddl Z ddl m Z ddl mZddlmZmZmZddlmZdd lmZerFdd lmZGd d d eZGd ddeZGdddeZGdddeZdS)a3 _______________________________ / Don't want to self-host? \ Try .json at http://dottxt.co / ------------------------------- \ ^__^ \ (oo)\_______ (__)\ )\/ ||----w | || || Copyright 2024- the Outlines developers Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. N) TYPE_CHECKINGAnyDictListOptionalTypeUnion)build_regex_from_schema) BaseModel)CFGGuideGuide RegexGuide)convert_json_schema_to_str)OutlinesLogitsProcessor) Tokenizerc@sveZdZUdZded<eed<eeefed<e eed<dddefddZ d e j d e j d e jfd d ZdddZdS)GuideLogitsProcessorzBias generation using a finite Attributes ---------- tokenizer The tokenizer used to convert tokens to ids. guide The `outlines.fsm.Guide` which is used to bias the logits. r tokenizerguide _guide_states_seq_start_idxcCs,||_||_ttg|jji|_d|_dS)zA Guide-based logits processor. Parameters ---------- tokenizer The tokenizer used to convert tokens to ids. guide The `outlines.fsm.Guide. which is used to bias the logits. N)rrhashtuple initial_staterr)selfrrrR/home/ubuntu/.local/lib/python3.10/site-packages/outlines/processors/structured.py__init__:s  zGuideLogitsProcessor.__init__ input_idslogitsreturncCs0|jdur t|d|_g}|D]=}||jd}tt|}||jvrE|jtt|dd}|j||d}||j|<| |j|qt j |t j d} g} g} t |D]\} } |j| jj| jdd}| || t || q^t | }t | }d| ||f<|| td|S) aUse the Guide to bias the logits before sampling the next token. Parameters ---------- input_ids The input token ids. logits The logits. Returns ------- torch.Tensor The biased logits. Nr)dtypeT) non_blockingFz-inf)rlenrrtolistrrget_next_stateitemappendtorch ones_likebool enumerateget_next_instructiontokenstodevice full_likecat masked_fill_float)rrrsequence_statesseq_idsgen_idscurr_state_key prev_state curr_statemaskallowed_tokens_batch batch_indicesi guide_stateallowed_tokensallowed_tokens_concatbatch_indices_concatrrrprocess_logitsIs6        z#GuideLogitsProcessor.process_logitscCst|j|jdS)z&Return a copy of the logits processor.rr)rrrcopy)rrrrrEszGuideLogitsProcessor.copyN)r r)__name__ __module__ __qualname____doc____annotations__r rintrrrr) LongTensor FloatTensorTensorrCrErrrrr*s    6rcs*eZdZdZdeddffdd ZZS)RegexLogitsProcessorzBias generation based on a regular expression. Attributes ---------- tokenizer The tokenizer used to convert tokens to ids. guide The `outlines.fsm.RegexGuide. which is used to bias the logits. regex_stringrrcs t||}tj||ddS)zCompile the RegexGuide that drives the regex-guided generation. Parameters ---------- regex_string A string that represents a regular expression tokenizer An Outlines tokenizer rDN)r from_regexsuperr)rrPrr __class__rrr zRegexLogitsProcessor.__init__)rFrGrHrIstrr __classcell__rrrSrrOs rOcsDeZdZdZ d deeeeefddde effdd Z Z S) JSONLogitsProcessorzBias generation based on a JSON schema. Attributes ---------- tokenizer The tokenizer used to convert tokens to ids. guide The `outlines.fsm.RegexGuide. which is used to bias the logits. Nschemarrwhitespace_patterncs(t|d}t||}tj||ddS)aCompile the Guide that drives the JSON-guided generation. Parameters ---------- schema A JSON schema that encodes the structure we want the model to generate. tokenizer The tokenizer used to convert tokens to ids. whitespace_pattern Pattern to use for JSON syntactic whitespace (doesn't impact string literals). For example, to allow only a single space or newline with `whitespace_pattern=r"[ ]?"` ) json_schema)rPrN)rr rRr)rrYrrZ schema_strrPrSrrrs  zJSONLogitsProcessor.__init__)N) rFrGrHrIrdictrr rVrrrWrrrSrrXsrXcsPeZdZUdZeed<deddffdd Zdej d ej d ej fd d Z Z S) CFGLogitsProcessorzBias generation based on a context-free grammar. Attributes ---------- tokenizer The tokenizer used to convert tokens to ids. guide The `outlines.fsm.CFGGuide. which is used to bias the logits. rcfg_strrrcs t||d}tj||ddS)zCompile the CFGGuide that drives the CFG-guided generation. Parameters ---------- cfg_str A string that represents a grammar tokenizer The tokenizer used to convert tokens to ids. ) cfg_stringrrDN)r rRr)rr_r cfg_guiderSrrrrUzCFGLogitsProcessor.__init__rrr c Cs|jdur t|d|_g}|D]=}||jd}tt|}||jvrE|jtt|dd}|j||d}||j|<| |j|qt |t j } t|D] \} } t|j| t j|| dd} || | gf| | | gf<qZ| S)zBSame behavior as GuideLogitsProcessor, but uses rejection samplingNrr!T) descending)rr$rrr%rrr&r'r(r)r1mathinfr,nextiter_valid_token_idsargsort) rrrr5r6r7r8r9r:r;r>r?first_legal_tokenrrrrCs(   z!CFGLogitsProcessor.process_logits) rFrGrHrIr rJrVrr)rLrNrCrWrrrSrr^s   r^)rIrctypingrrrrrrrr)outlines_core.fsm.json_schemar pydanticr outlines.fsm.guider r r outlines.fsm.json_schemarbase_logits_processorroutlines.models.tokenizerrrrOrXr^rrrrs$     Z#