o i @sddlZddlmZmZmZmZmZmZzddlm Z ddl m Z m Z ddl mZWn ey5edwzddlZWn eyGedwdd lmZdd lmZmZdd lmZGd d d e ZGdddZdededeeeeeffddZdedeedefddZ  d'dededeedefddZGdddZ deeefdede fd d!Z!d"e deeefded#e"deee"ff d$d%Z#gd&Z$dS)(N)AnyCallableListOptionalTupleUnion)AutoModelForCausalLM)LogitsProcessor PrefixConstrainedLogitsProcessor)PreTrainedTokenizerBasezWtransformers is not installed. Please install it with "pip install transformers[torch]"zfpytorch is not installed. See https://pytorch.org/get-started/locally/ for installation instructions.")CharacterLevelParser) TokenEnforcerTokenEnforcerTokenizerData)FormatEnforcerAnalyzerc@s:eZdZdeddfddZdejdejdejfdd ZdS) LogitsSaverWarperanalyzerreturnNcC ||_dSN)r)selfrrg/home/ubuntu/veenaModal/venv/lib/python3.10/site-packages/lmformatenforcer/integrations/transformers.py__init__ zLogitsSaverWarper.__init__ input_idsscorescCs6|}|}t||D] \}}|j||q |Sr)tolistziprreport_raw_logits)rrr cpu_inputs cpu_scoressingle_batch_inputssingle_batch_scoresrrr__call__s zLogitsSaverWarper.__call__) __name__ __module__ __qualname__rrtorch LongTensor FloatTensorr$rrrrrs rc@s:eZdZUeed<dedefddZd ddZd d Z dS) LogitsSaverManagerwarpermodelrcCs||_d|_d|_||_dSr)r-r, old_warperr)rr-rrrrr!s zLogitsSaverManager.__init__Ncs$jj_fdd}|j_dS)NcsF|}tj_|djdur!td}|d||S)Nr)r.rrr,insertr )generation_configwarpers processor filter_funcrrrnew_logits_warper*s    zCLogitsSaverManager.replace_logits_warper..new_logits_warper)r-_get_logits_warperr.)rr5r6rr4rreplace_logits_warper's  z(LogitsSaverManager.replace_logits_warpercCs|j|j_dSr)r.r-r7)rrrrunreplace_logits_warper4sz*LogitsSaverManager.unreplace_logits_warperr) r%r&r'r__annotations__rrrr8r9rrrrr+s   r+ tokenizer vocab_sizercCsr|dd}g}t|D])}||jvrq |||gdd}||g}t|t|k}||||fq |S)N0r/)encoderangeall_special_idsdecodelenappend)r;r<token_0regular_tokens token_idxdecoded_after_0decoded_regularis_word_start_tokenrrr_build_regular_tokens_list8s   rKtokenscCs||}|d}|S)Nu�)rBrstrip)r;rLdecodedcleanedrrr_decode_functionFs  rPF use_bitmaskcCs4|pt|}t||}tt|}t|||j||Sr)rCrK functoolspartialrPr eos_token_id)r;rQr<rF decode_fnrrr#build_token_enforcer_tokenizer_dataLs   rVc@s6eZdZdefddZdedejdeefddZ d S) !TransformersPrefixAllowedTokensFntoken_enforcercCrr)rX)rrXrrrrWrz*TransformersPrefixAllowedTokensFn.__init__batch_idsentrcCs|}|j|jSr)rrXget_allowed_tokensallowed_tokens)rrYrZtoken_sequencerrrr$Zsz*TransformersPrefixAllowedTokensFn.__call__N) r%r&r'rrintr(Tensorrr$rrrrrWVs rWtokenizer_datacharacter_level_parsercCs$t|tr t|}t||}t|S)aBuild the prefix allowed tokens function that transformers will use to filter the tokens generated by the model. The result can be passed to the prefix_allowed_tokens_fn parameter of the generate() method of transformers models or pipeline configurations.) isinstancer rVrrW)r`rarXrrr+build_transformers_prefix_allowed_tokens_fn_s  rcr-kwargscKst||}|djddk}|dddk}|p| }|dd}|dd} |o,| o,|} | rdt|j} t|| } | ||} z|jd i| }W| n| w| |d d }||_ |S|jd i|d |i}|S) aGenerate text from a model while enforcing a given format, generating enforcing diagnostic information. This can be used instead of calling model.generate(). If return_dict_in_generate and output_scores parameters are True, diagnostic information will be returned in the result. If you don't need this, consider using prefix_allowed_tokens_fn + build_transformers_prefix_allowed_tokens_fn() insteadrrr/ num_beamsreturn_dict_in_generateF output_scoresN sequencesprefix_allowed_tokens_fnr) rcshapegetrrXr+r8generater9generate_report_dictrenforced_scores)r-r;rard"transformers_filter_allowed_tokensis_multi_inputsis_multi_beamssupport_diagnosticsrfrgshould_run_in_advanced_moder logits_savergenerate_kwargsoutputdf_dictrrrgenerate_enforcedis(        rx)rcrxrV)FN)%rRtypingrrrrrr transformersr&transformers.generation.logits_processr r transformers.tokenization_utilsr ImportErrorr(characterlevelparserr tokenenforcerrrrrrr+r^strboolrKrPrVrWrcdictrx__all__rrrrs\        $       '