o i @sUddlmZddlmZddlmZiddddddd dd dd dd dd dddddddddddddddddddZded<iddddddddddd dd!dd"dd#dd$dd%dd&dd'dd(dd)dd*d+d,d+id-d.d/d.d0d.d1d.d2d.d3d.d4d.d5d.d6d+d7d+d8d+d9d+d:d+d;d+dd=d?d.d.d.d.d.d.d.d.d.d.d@d@dA ZdedB<dLdFdGZ dMdIdJZ dKS)N) annotations)Encoding) get_encodingzo1- o200k_basezo3-zo4-mini-zgpt-5-zgpt-4.5-zgpt-4.1-z chatgpt-4o-zgpt-4o-zgpt-4- cl100k_basezgpt-3.5-turbo-z gpt-35-turbo-zgpt-oss- o200k_harmonyz ft:gpt-4ozft:gpt-4zft:gpt-3.5-turbozft:davinci-002zft:babbage-002zdict[str, str]MODEL_PREFIX_TO_ENCODINGo1o3zo4-minizgpt-5zgpt-4.1zgpt-4ozgpt-4z gpt-3.5-turbozgpt-3.5z gpt-35-turboz davinci-002z babbage-002ztext-embedding-ada-002ztext-embedding-3-smallztext-embedding-3-largeztext-davinci-003 p50k_baseztext-davinci-002ztext-davinci-001 r50k_baseztext-curie-001ztext-babbage-001z text-ada-001davincicuriebabbageadazcode-davinci-002zcode-davinci-001zcode-cushman-002zcode-cushman-001z davinci-codexz cushman-codexztext-davinci-edit-001 p50k_editzcode-davinci-edit-001ztext-similarity-davinci-001gpt2) ztext-similarity-curie-001ztext-similarity-babbage-001ztext-similarity-ada-001ztext-search-davinci-doc-001ztext-search-curie-doc-001ztext-search-babbage-doc-001ztext-search-ada-doc-001zcode-search-babbage-code-001zcode-search-ada-code-001rzgpt-2MODEL_TO_ENCODING model_namestrreturncCsXd}|tvr t|}ntD] \}}||r|Sq|dur*td|dd|S)zrReturns the name of the encoding used by a model. Raises a KeyError if the model name is not recognised. NzCould not automatically map z_ to a tokeniser. Please use `tiktoken.get_encoding` to explicitly get the tokeniser you expect.)rr items startswithKeyError)r encoding_name model_prefixmodel_encoding_namerK/home/ubuntu/veenaModal/venv/lib/python3.10/site-packages/tiktoken/model.pyencoding_name_for_modelXs   r rcCs tt|S)zfReturns the encoding used by a model. Raises a KeyError if the model name is not recognised. )rr )rrrrencoding_for_modelqs r!N)rrrr)rrrr) __future__rcorerregistryrr __annotations__rr r!rrrrs            !"#$%&()+, ;