o Y۷i<@sNdZddlZddlZddlmZddlmZmZm Z ddl m Z m Z m Z mZmZddlmZddlmZedZejd ks@Je jrNejejd d d d Zn ejejd d d d Zed ZejZejejejdeZ ej!ej"ej#deZ$ddZ%e ddZ&e ddZ'e ddZ(eddddZ)eddZ*eddddZ+edd Z,ed!d"Z-d#d$Z.d%d&Z/d'd(Z0d)d*Z1ed+d,Z2eejej3d-d.Z4d/d0Z5eej3ejd1d2Z6eej7ejd3d4Z8e d5d6Z9eej7ej:d7d8Z;e ej<d9d:Z=e e>d;d<Z?e ej@e ejAd=d>ZBe ejCd?d@ZDe ejEdAdBZFe ejGdCdDZHe ejIdEdFZJe ejKdGdHZLe ejMdIdJZNe ejOdKdLZPe ejQdMdNZRe ejSdOdPZTe ej:dQe ej3dQe ejdQdRdSZUe ej:dTe ej3dTdUdVZVe ej7dWdXdYZWe ej3dZe ejdZd[d\ZXe ej:d]d^d_ZYe eZd`daZ[e ej:dbdcddZ\e ej3dbdedfZ]e ej:dgdhdiZ^e ej3dgdjdkZ_e ej:dldmdnZ`e ej3dldodpZae ej:dqe ej3dqe ejdqdrdsZbe ej:dte ej3dte ejdtdudvZce ej:dwe ej3dwe ejdwdxdyZde ej:dze ej3dze ejdzd{d|Zeed}d~Zfe ej:de ej3de ejddddZge ej:de ej3de ejddddZhe ej:de ej3de ejddddZie ej:de ej3de ejddddZje ej:de ej3de ejdddZke ej:de ej3de ejddddZle ej:de ej3de ejddddZme ej:de ej3de ejddddZne ej:de ej3de ejdddZodS)z=Implements operations on bytes and str (unicode) array items.N)ir)typescgutilsconfig)overload intrinsicoverload_method lower_castregister_jitable) is_nonelike)unicodeS1CT)readonlyU1)rcsfdd}|S)Nc sR|\}}tj||d}||t}||||g}||tdS)Nvalue ) ralloca_once_valuebitcastrIntType as_pointerloadgepzext) contextbuilder signatureargsdataidxrawptrptrchbitsizeK/home/ubuntu/vllm_env/lib/python3.10/site-packages/numba/cpython/charseq.pycodegen(s z#make_deref_codegen..codegenr))r(r+r)r'r*make_deref_codegen's r,cCt|tj}|tdfS)Nruint32intpr, typingctxr"offsetsigr)r)r* deref_uint81 r6cCr-)Nr/r2r)r)r* deref_uint167r7r9cCr-)Nrr/r2r)r)r* deref_uint32=r7r:F)_nrtcCs t||S)z6Access i-th item of CharSeq object via code value )r6air)r)r*charseq_get_codeCs r?cCst||}|dkr td|S)zcAccess i-th item of CharSeq object via code value. null code is interpreted as IndexError rindex out of range)r? IndexErrorr=r>coder)r)r*charseq_get_valueJs rDcCs>tdkr t||Stdkrt||Stdkrt||Std)z=Access i-th item of UnicodeCharSeq object via code value rrrz=unicode_charseq_get_code: unicode_byte_width not in [1, 2, 4])unicode_byte_widthr:r9r6NotImplementedErrorr<r)r)r*unicode_charseq_get_codeVs   rGcCs t||S)z,Access i-th item of UnicodeType object. )r _get_code_pointr<r)r)r*unicode_get_codees rIcCs||S)z*Access i-th item of Bytes object. r)r<r)r)r*bytes_get_codelsrJcCsDt|tjrtSt|tjrtSt|tjrtSt|tjr t SdSN) isinstancerCharSeqr?BytesrJUnicodeCharSeqrG UnicodeTyperIr=r)r)r*_get_code_implss    rRcCs<tjtjftjtjffD]}t||rt||rdSq dS)NTF)rrMrNrOrPrL)r=btr)r)r* _same_kind~s  rUcCst|tjtjfSrK)rLrrMrNrQr)r)r* _is_bytesrVcCs||kp t|tjSrK)rLrOmitted)xdefaultr)r)r* is_defaultr[cCs0t||}|dkr tdt|ttdS)zlAccess i-th item of UnicodeCharSeq object via unicode value null code is interpreted as IndexError rr@r))rGrAnparray unicode_uintviewu1_dtyperBr)r)r*unicode_charseq_get_values rbc CsDt||||d}||jtd}|j}||}td} t ||} || | } t |j |j } | d|| } || || }|| t|| t |j |j dWdn1sewYt||%}|||jg}|||| }|||| |jgWdn1swY|| S)Nrr..codegen)r unicode_typer)r3rr5r+r)rr*_unicode_to_bytess rc Csrt||||d}||jtd}||jtd}||jtd}|d|jt |jj d} |d|jt |jj d} |d|jt |jj d} |j } | |} tdt }t|| }|||}t | j |j}|d | |}||| |}||t||t | j |jt d Wdn1swY|| ;t||%}|||jg}||||}|||||jgWdn1swYWdn1swY|| Pt dkr8t||%}|||jg}||||}|||||jgWdn 1s2wYn|j|td t d|fWdn 1sQwY|| Pt dkrt||%}|||jg}||||}|||||jgWdn 1swYn|j|td t d|fWdn 1swY||S) Nrr.r8rz==rrrrcrz,cannot cast 16-bit unicode_type to %s-bit %sz,cannot cast 32-bit unicode_type to %s-bit %s)rrdrr"rrrrkrrhrirrfrErgrjrlrmrnrorrprrrqrrr)rrrrrsrtrrsrc2src4kind1kind2kind4rwrxryrzr{r|r}rjr~rrr)r)r*unicode_to_unicode_charseqs             rcsbdt|tjrt|tjrtt|tjrtdur/|jddfdd}|SdS)Nzindex out of range [0, %s]rcs"|kr |dkr ||Str)rArr> get_valuemax_imsgr)r* getitem_implXs z%charseq_getitem..getitem_impl)rLrIntegerrMrDrOrbrj)rr>rr)rr*charseq_getitemLs    rcsJt|tjtjfr#t||jdkrdd}|Sfdd}|SdS)NrcSsdSrr)rr)r)r*len_impleszcharseq_len..len_implcsD}d}|dkr|d}|dkr |dS||}|dks|dS)Nrrr))rr>rCget_codenr)r*ris )rLrrMrOrRrj)rrr)rr* charseq_len_s rcCst||sdSt|tjrt|tjrdd}|St|tjr+t|tjr+dd}|St|tjr=t|tjr=dd}|St|tjtjfrUt|tjtjfrWdd}|SdSdS)NcSs t||SrKstrr=rSr)r)r*impl~ zcharseq_concat..implcSs |t|SrKrrr)r)r*rrcSst|t|SrKrrr)r)r*rcSs||SrK_to_str _to_bytesrr)r)r*rr\)rUrLrrOrPrMrNr=rSrr)r)r*charseq_concatws.       rcCspt|tjr dd}|St|tjrdd}|St|tjtjfr'dd}|St|tjtjfr6dd}|SdS)NcSs t||SrKrrr)r)r*wraprzcharseq_repeat..wrapcSs |t|SrKrrr)r)r*rrcSs||SrKrrr)r)r*rrcSs||SrKrrr)r)r*rrrLrrOrMrN)r=rSrr)r)r*charseq_repeats  rcCs&t|tjtjtjfrdd}|SdS)NcSs t|dkSr)lenrQr)r)r*rrzcharseq_not..implr)r=rr)r)r* charseq_notsrcHt||sdSt|t|dur dur"fdd}|SdSdS)NcsDt|}|t|kr dSt|D]}||||krdSqdS)NFT)rrange)r=rSrr> left_code right_coder)r*eq_impls  zcharseq_eq..eq_implrUrR)r=rSrr)rr* charseq_eqs rcCBt||sdSt|}t|}|dur|durdd}|SdSdS)NcSs ||k SrKr)rr)r)r*ne_impl zcharseq_ne..ne_implr)r=rSrrrr)r)r* charseq_ne rcr)NcsZt|}t|}t||}t|D]}||||}}||kr(||kSq||kSrK)rminr)r=rSnanbrr>cacbrr)r*lt_impls   zcharseq_lt..lt_implr)r=rSrr)rr* charseq_lts  rcCr)NcSs||kSrKr)rr)r)r*gt_implzcharseq_gt..gt_implr)r=rSrrrr)r)r* charseq_gtrrcCr)NcSs ||k SrKr)rr)r)r*le_implrzcharseq_le..le_implr)r=rSrrrr)r)r* charseq_lerrcCr)NcSs ||k SrKr)rr)r)r*ge_implrzcharseq_ge..ge_implr)r=rSrrrr)r)r* charseq_gerrcCsVt||sdSt|}t|}|dur'|dur)t|r!dd}|Sdd}|SdSdS)NcSs||vSrKrrr)r)r* contains_impl sz'charseq_contains..contains_implcSst|t|vSrKrrr)r)r*rr)rUrRrV)r=rSrrrr)r)r*charseq_containss  risasciict|fdd}|S)Ncs*tt|D] }||dkrdSqdS)NFT)rrrrr)r*rs zcharseq_isascii..implrRrrr)rr*charseq_isasciis r _get_kindcr)NcsNd}tt|D] }||}||kr|}q|dkrtjS|dkr$tjStjS)Nri)rrr PY_UNICODE_4BYTE_KINDPY_UNICODE_2BYTE_KINDPY_UNICODE_1BYTE_KIND)rmax_coder>rCrr)r*r*s zcharseq_get_kind..implrrr)rr*charseq_get_kind%s  rrcC dd}|S)zConvert unicode_type object to Bytes object. Note: The usage of _to_bytes method can be eliminated once all Python bytes operations are implemented for numba Bytes objects. cSst|SrK)rrr)r)r*r@rz"unicode_to_bytes_mth..implr)rr)r)r*unicode_to_bytes_mth8srrcr)zConvert bytes array item or bytes instance to UTF-8 str. Note: The usage of _to_str method can be eliminated once all Python bytes operations are implemented for numba Bytes objects. csJt|}|}ttj||}t|D]}||}t|||q|SrK)rrr _empty_stringrr_set_code_point)rris_asciiresultr>rCrr)r* tostr_implOs  z&charseq_to_str_mth..tostr_implr)rrr)rr*charseq_to_str_mthEs  r__str__cr)NcsXt|}|}|dko|}t|||}t|D]}||}t|||q|Sr)rrrr rrr)rrrrrr>rCrr)r*str_impl_s  zcharseq_str..str_implr)rrr)rr* charseq_str[s  rcCst|tjr ddSdS)NcSs|SrKr)rr)r)r*oszcharseq_bytes..)rLrrMrr)r)r* charseq_bytesls r__hash__cCr)NcSs tt|SrK)hashrrr)r)r*rtrz"unicode_charseq_hash..implr)rr)r)r*unicode_charseq_hashrrcCr)NcSs t|SrK)rrrr)r)r*r{s zcharseq_hash..implr)rr)r)r* charseq_hashysrisuppercCr)NcSst| SrK)rrrr)r)r*rz%unicode_charseq_isupper..implr)rr)r)r*unicode_charseq_isupperrcCr)NcSs| SrK)rrrr)r)r*rrzcharseq_isupper..implr)rr)r)r*charseq_isupperrruppercCr)NcS t|SrK)rr rr)r)r*rrz#unicode_charseq_upper..implr)rr)r)r*unicode_charseq_upperrr cCr)NcS|SrK)rr rrr)r)r*rrzcharseq_upper..implr)rr)r)r* charseq_upperrr findcCt|tjrt|tjrdd}|St|tjrdd}|St|tjr3t|tjtjfr3dd}|St|tjrEt|tjrEdd}|St|tjrWt|tjrYdd}|SdSdS)NcSt|t|SrKrrrr)r)r*rrWz"unicode_charseq_find..implcSt||SrKrrr)r)r*rcS||SrKrrrr)r)r*rrWcS|t|SrK)rrrr)r)r*rrcSrrKrrr)r)r*rrWrLrrOrPrMrNrr)r)r*unicode_charseq_find*        rrfindcCr)NcSrrKrrrr)r)r*rrWz#unicode_charseq_rfind..implcSrrKrrr)r)r*rrcSrrKrrrr)r)r*rrWcSrrK)rrrr)r)r*rrcSrrKrrr)r)r*rrWrrr)r)r*unicode_charseq_rfindrr startswithcCtt|tjrt|tjrdd}|St|tjrdd}|St|tjtjfr6t|tjtjfr8dd}|SdSdS)NcSrrKrrrr)r)r*rrWz(unicode_charseq_startswith..implcSrrKr rr)r)r*rrcSrrK)rrrr)r)r*rrWrrr)r)r*unicode_charseq_startswith   r!endswithcCr)NcSrrKrr#rr)r)r*rrWz&unicode_charseq_endswith..implcSrrKr$rr)r)r*rrcSrrK)rr#rr)r)r*rrWrrr)r)r*unicode_charseq_endswithr"r%cCsdd|DS)NcSg|]}|qSr))r).0rr)r)r* z_map_bytes..r))seqr)r)r* _map_bytessr+splitcCs|dkst|tjtjtjfsdSt|tjrHt|tjr$d dd}|St|tjr1d dd}|St|rHt|drAd dd}|Sd dd}|St|tj tj frxt|tj tj frad dd}|St|rzt|drqd dd}|Sd d d}|SdSdS) Nr-cSst|jt||dSN)sepmaxsplitrr,r=r/r0r)r)r*rz#unicode_charseq_split..implcSst|j||dSr.r1r2r)r)r*rrWcSr rKr1r2r)r)r*r rcSst|j|dSN)r0r1r2r)r)r*r#rcSst|j||dSr4r+rr,r2r)r)r*r(scSst|SrKr5r2r)r)r*r.rcSst|j|dSr4r5r2r)r)r*r1r\Nr-) rLrrXrIntegerLiteralrOrPr r[rMrN)r=r/r0rr)r)r*unicode_charseq_split s>             r8ljust cCt|tjr,t|drd dd}|St|tjrd dd}|St|tjr,d dd}|St|tjtjfrVt|ds?t|drFd dd}|St|tjtjfrXd dd}|SdSdS) Nr:cSrrKrr9r=widthfillcharr)r)r*r>rz#unicode_charseq_ljust..implcSt||t|SrKr<r=r)r)r*rBr\cSt|||SrKr<r=r)r)r*rFr cS||SrKrr9rr=r)r)r*rKrWcS|||SrKrDr=r)r)r*rOr:rLrrOr[rPrMrNr=r>r?rr)r)r*unicode_charseq_ljust8&         rJrjustcCr;) Nr:cSrrKrrLr=r)r)r*rZrz#unicode_charseq_rjust..implcSr@rKrMr=r)r)r*r^r\cSrArKrMr=r)r)r*rbrrBcSrCrKrrLrr=r)r)r*rgrWcSrErKrNr=r)r)r*rkrFrGrHrIr)r)r*unicode_charseq_rjustTrKrOcentercCr;) Nr:cSrrKrrPr=r)r)r*rvrz$unicode_charseq_center..implcSr@rKrQr=r)r)r*rzr\cSrArKrQr=r)r)r*r~rrBcSrCrKrrPrr=r)r)r*rrWcSrErKrRr=r)r)r*rrFrGrHrIr)r)r*unicode_charseq_centerprKrSzfillcC:t|tjr dd}|St|tjtjfrdd}|SdS)NcSrrK)rrTr=r>r)r)r*rrz#unicode_charseq_zfill..implcSrCrK)rrTrrVr)r)r*rrWr)r=r>rr)r)r*unicode_charseq_zfills rWlstripcCt|tjr+t|rddd}|St|tjrddd}|St|tjr+ddd}|St|tjtjfrOt|r?ddd}|St|tjtjfrQddd}|SdSdS)NcSr rKrrXr=charsr)r)r*rrz$unicode_charseq_lstrip..implcSrrKrZr[r)r)r*rrWcSrrKrZr[r)r)r*rrcSr rKrrXrr[r)r)r*rrcS||SrKr]r[r)r)r*rr3rKrLrrOr rPrMrNr=r\rr)r)r*unicode_charseq_lstrip&        rarstripcCrY)NcSr rKrrcr[r)r)r*rrz$unicode_charseq_rstrip..implcSrrKrdr[r)r)r*rrWcSrrKrdr[r)r)r*rrcSr rKrrcrr[r)r)r*rrcSr^rKrer[r)r)r*rr3rKr_r`r)r)r*unicode_charseq_rstriprbrfstripcCrY)NcSr rKrrgr[r)r)r*rrz#unicode_charseq_strip..implcSrrKrhr[r)r)r*rrWcSrrKrhr[r)r)r*rrcSr rKrrgrr[r)r)r*rrcSr^rKrir[r)r)r*rr3rKr_r`r)r)r*unicode_charseq_striprbrjjoincCrU)NcSsdd|D}t||S)NcSsg|]}t|qSr)rr'pr)r)r*r(r)6unicode_charseq_join..impl..)rrkr=parts_partsr)r)r*rsz"unicode_charseq_join..implcSs dd|D}||S)NcSr&r)rrlr)r)r*r(r)rn)rrkrror)r)r*rsr)r=rprr)r)r*unicode_charseq_joins rrr6rGrK)p__doc__operatornumpyr]llvmliter numba.corerrrnumba.core.extendingrrrr r numba.core.cgutilsr numba.cpythonr dtypes1_dtyperUSE_LEGACY_TYPE_SYSTEMrNuint8rc_uint8rarEuint16r0r_rrr unicode_kindr,r6r9r:r?rDrGrIrJrRrUrVr[rbrMrrrrPrrrOrgetitemrrraddiaddrmulrnot_reqrnerltrgtrlergercontainsrrrrrrbytesrrrrrr r rrr!r%r+r8rJrOrSrWrarfrjrrr)r)r)r*sF                       L                                                    (