o 'N ib ã@sŠdZddlZddlZddlZddlmZdZe ded¡ZdZ e de d ¡Z d Z e de d ¡Z e d ¡Z d d „Zddd„ZdS)z$ De-tokenizer for Indian languages. éN)ÚIndicNlpExceptionz!%)\]},.:;>?\u0964\u0965z[ ]([z])z#$(\[{<@z([z])[ ]z-/\\z([0-9]+ [,.:/] )+[0-9]+c Cs$|}d}d}t |¡D]$}| ¡}| ¡}||kr/||||…}||||… dd¡}|}q |||d…}|}t d|¡}t d|¡}t d|¡}d}|D]?}d} g} |D]!} | |krt| ddkrj|   d¡n|   d ¡| d 7} qX|   | ¡qXd  | ¡ d |¡ d |¡ d|¡ d |¡}qP|S) a‡detokenize string for Indian language scripts using Brahmi-derived scripts A trivial detokenizer which: - decides whether punctuation attaches to left/right or both - handles number sequences - handles quotes smartly (deciding left or right attachment) Args: text (str): tokenized text to process Returns: str: detokenized string Úrú Nz\1z'"`éz@RAz@LAéz@RA z @LA) Ú pat_num_seqÚfinditerÚstartÚendÚreplaceÚpat_lraÚsubÚpat_laÚpat_raÚappendÚjoin) ÚtextÚsÚnew_sÚprevÚmr r Ú alt_attachÚpuncÚcntÚout_strÚc©rúV/home/ubuntu/.local/lib/python3.10/site-packages/indicnlp/tokenize/indic_detokenize.pyÚtrivial_detokenize_indic's>€        ÿrÚhicCst|ƒS)aÅdetokenize string for languages of the Indian subcontinent A trivial detokenizer which: - decides whether punctuation attaches to left/right or both - handles number sequences - handles quotes smartly (deciding left or right attachment) Args: text (str): tokenized text to process Returns: str: detokenized string Raises: IndicNlpException: If language is not supported )r)rÚlangrrrÚtrivial_detokenizeesr!)r)Ú__doc__ÚstringÚreÚsysÚindicnlp.commonrÚ left_attachÚcompilerÚ right_attachrÚ lr_attachr rrr!rrrrÚs   >