o 'NiB" @sddlZddlZddlZddlmZddlmZddlm Z  da da da da da dagdZddgddgdd gd d gd d gd d gd d gd dgddgddgddgddgd ZdZdZdZdZdZdZeZedZdZdZdZddZddZddZ dd Z!d!d"Z"d#d$Z#d%d&Z$d'd(Z%d)d*Z&d+d,Z'd-d.Z(d/d0Z)d1d2Z*d3d4Z+d5d6Z,d7d8Z-d9d:Z.d;d<Z/d=d>Z0d?d@Z1dAdBZ2dCdDZ3dEdFZ4dGdHZ5dIdJZ6dKdLZ7dMdNZ8dS)ON)common)IndicNlpException)langinfo&) basic_type vowel_lengthvowel_strength vowel_statusconsonant_typearticulation_place aspirationvoicing nasalizationvowel_horizontalvowel_verticalvowel_roundness  $ cCsztjtjtddddatjtjtddddatj ddt dfj a tj ddt dfj a t jdadS)zI To be called by library loader, do not call it in your program scriptzall_script_phonetic_data.csvzutf-8)encodingztamil_script_phonetic_data.csvNr)pdread_csvospathjoinrget_resources_pathALL_PHONETIC_DATATAMIL_PHONETIC_DATAilocPHONETIC_VECTOR_START_OFFSETvaluesALL_PHONETIC_VECTORSTAMIL_PHONETIC_VECTORSshapePHONETIC_VECTOR_LENGTHr5r5Q/home/ubuntu/.local/lib/python3.10/site-packages/indicnlp/script/indic_scripts.pyinit`s r7cCs|ttjvS)N)listli SCRIPT_RANGESkeys)langr5r5r6is_supported_languageosr=cCs,t|s td|t|tj|dS)NLanguage {} not supportedr)r=rformatordr9r:cr<r5r5r6 get_offsetrsrCcCs,t|s td|t|tj|dS)z5 Applicable to Brahmi derived Indic scripts r>r)r=rr?chrr9r:)offr<r5r5r6offset_to_charwsrFcCsLt|s td|t||}|tkr|tkp%t|tjkp%t|tj kS)z Applicable to Brahmi derived Indic scripts Note that DANDA and DOUBLE_DANDA have the same Unicode codepoint for all Indic scripts r>) r=rr?rCSCRIPT_OFFSET_STARTSCRIPT_OFFSET_RANGEr@r9DANDA DOUBLE_DANDA)rBr<or5r5r6is_indiclang_chars   rLcCs|tjko |tjkS)z6 Applicable to Brahmi derived Indic scripts )r9!COORDINATED_RANGE_START_INCLUSIVECOORDINATED_RANGE_END_INCLUSIVE)c_offsetr5r5r6in_coordinated_range_offsetsrPcCs$t|s td|tt||SNr>)r=rr?rPrCrAr5r5r6in_coordinated_rangesrRcCsBt|s td||tjkrtnt}|tjkrtnt}||fSrQ) r=rr?r9LC_TAr,r-r1r2)r< phonetic_dataphonetic_vectorsr5r5r6get_phonetic_infos rVcCstdgtSNr)nparrayr4r5r5r5r6invalid_vectorsrZcCsDt||}t|s tSt|\}}|j|ddkrtS||SNzValid Vector Representationr)rCrPrZrVr.)rBr<offsetrTrUr5r5r6get_phonetic_feature_vectors  r]cCs:t|stSt|\}}|j|ddkrtS||Sr[)rPrZrVr.)r\r<rTrUr5r5r6"get_phonetic_feature_vector_offsets  r^cCst|dkSrW)rXsumvr5r5r6is_validsrbcC |tdkSNr)PVIDX_BT_VOWELr`r5r5r6is_vowel rfcCrcrd)PVIDX_BT_CONSONANTr`r5r5r6 is_consonantrgricCrcrd)PVIDX_BT_HALANTr`r5r5r6 is_halantrgrkcCrcrd)PVIDX_BT_NUKTAr`r5r5r6is_nuktargrmcCrcrd)PVIDX_BT_ANUSVAARr`r5r5r6 is_anusvaarrgrocCrcrd) PVIDX_BT_MISCr`r5r5r6is_miscrgrqcCst|o |tdkSrd)rfPVIDX_VSTAT_DEPr`r5r5r6is_dependent_vowelsrscCst|o t|dddkS)Nr rr)riget_property_vectorr`r5r5r6 is_plosiverucCtddt||DS)NcSs$g|]\}}||dkrdndqSrrr5.0b1b2r5r5r6 s$zor_vectors..rXrYzipv1v2r5r5r6 or_vectorsrvrcCrw)NcSs g|] \}}||kr dndqSrxr5ryr5r5r6r}s zxor_vectors..r~rr5r5r6 xor_vectorsrvrcCs|t|dt|dS)Nrr)PV_PROP_RANGES)ra prop_namer5r5r6rtsrtcCsFt||}d}d}|dddD] }|||7}|d}qt|S)Nrrg@)rttolistint)rar factor_bitsrBbr5r5r6get_property_values  rc Cs,tt|dt|df}t|dD]g\}}t|dD]]\}}t||} t||} t| rHt| rH| | krH||d|dfd|||f<qt| set| se||kre||d|dfd|||f<qt|||df||d|f|||f<qq|dttt|t|tt|tt|fS)a2 compute the Longest Common Subsequence Ratio (LCSR) between two strings at the character level. This works for Indic scripts by mapping both languages to a common script srcw: source language string tgtw: source language string slang: source language tlang: target language r?rr)rXzeroslen enumeraterCrPmaxfloat) srcwtgtwslangtlang score_matsisctitcsotor5r5r6 lcsr_indics   ""4 rcCstt|dt|df}t|dD]8\}}t|dD].\}}||kr6||d|dfd|||f<qt|||df||d|f|||f<qq|dttt|t|tt|tt|fS)zA LCSR computation if both languages have the same script rrr)rXrrrrr)rrrrrrrr5r5r6lcsr_any s"4 rcCs0||ks t|r t|st||||St||S)z compute the Longest Common Subsequence Ratio (LCSR) between two strings at the character level. srcw: source language string tgtw: source language string slang: source language tlang: target language )r=rr)rrrrr5r5r6lcsrs  r)9pandasr&numpyrXr(indicnlprindicnlp.commonrrr9r,r-r1r2r4r/PV_PROPrrerhrlrjrnrp PVIDX_BT_S PVIDX_BT_ErrrGrHr7r=rCrFrLrPrRrVrZr]r^rbrfrirkrmrorqrsrurrrtrrrrr5r5r5r6s