o qi @sHdZddlZddlmZmZddlmZddlmZGdddeZ dS) z Document data structure N)ListTuple) Conllable)Sentencec@seZdZdZddeeeefdefddZ e defdd Z e defd d Z e de fd d Zejdd Ze de fddZejddZddZddZddZdefddZdefddZddZdS)DocumentzW A document class that stores attributes of a document and carries a list of sentences.N sentencestextcCs&g|_||_d|_d|_||dS)a Construct a document given a list of sentences in the form of lists of CoNLL-U dicts. Args: sentences (list): List of sentences, which being a tuple of dict, list of token entry (CoNLL-U dict). text (str): Urdu text of the document. rN) _sentences_text _num_tokens _num_words_process)selfrr rO/home/ubuntu/.local/lib/python3.10/site-packages/urduhack/core/unit/document.py__init__s zDocument.__init__returncC|jS)zj Access the Urdu text for this document. Returns: str: Document text )r rrrrr sz Document.textcCr)z1 Access the list of sentences for this document. )r rrrrr(zDocument.sentencescCr)z0 Access the number of tokens for this document. r rrrr num_tokens-rzDocument.num_tokenscC ||_dS)z- Set the number of tokens for this document. Nrrvaluerrrr2 cCr)z/ Access the number of words for this document. r rrrr num_words7rzDocument.num_wordscCr)z, Set the number of words for this document. Nrrrrrr<rcCs|D]8}|jt||d|jdjdj|jdjdj}}t|jdu|du|dugr:|j|||jd_qtdd|jD|_ tdd|jD|_ dS)z(process sentences in to words and tokens)docrNcSg|]}t|jqSr)lentokens.0sentencerrr Iz%Document._process..cSr!r)r"wordsr$rrrr'Jr() rappendrr# start_charend_charallr sumrr)rrr& begin_idxend_idxrrrrAs&zDocument._processcc|jD]}|jEdHqdS)z= An iterator that returns all of the words in this Document. N)rr)rr&rrr iter_wordsL zDocument.iter_wordsccr1)z> An iterator that returns all of the tokens in this Document. N)rr#r2rrr iter_tokensQr4zDocument.iter_tokenscCsdd|jDS)zp Dumps the whole document into a list of list of dictionary for each token in each sentence in the doc. cSsg|]}|qSr)to_dictr$rrrr'Ysz$Document.to_dict..)rrrrrr6VszDocument.to_dictcCs(ttdd|j}|dd|S)z Output the Conll object to a CoNLL-U formatted string. Returns: str: The CoNLL-U object as a string. This string will end in a newline. cSs|SN)conll)sentrrrdsz Document.conll..z )listmaprr*join)r componentsrrrr8[s  zDocument.conllcCstj|dddS)NF)indent ensure_ascii)jsondumpsr6rrrr__repr__iszDocument.__repr__r7)__name__ __module__ __qualname____doc__rrdictr<strrpropertyr rintrsetterrrr3r5r6r8rErrrrrs*       r) rIrCtypingrrurduhack.conll.conllablerr&rrrrrrs