o V۷i}@s~dZddlmZmZddlmZmZddlmZddl m Z ddl m Z Gdd d eZ Gd d d e eZGd d d eZdS)z@Provides a post-lexer for implementing Python-style indentation.)ABCabstractmethod)ListIterator) LarkError)PostLex)Tokenc@s eZdZdS) DedentErrorN)__name__ __module__ __qualname__rrC/home/ubuntu/vllm_env/lib/python3.10/site-packages/lark/indenter.pyr sr c@seZdZUdZeed<eeed<dddZdede efd d Z d d Z d dZ e ddZe edefddZe edeefddZe edeefddZe edefddZe edefddZe edefddZdS)IndenteraThis is a postlexer that "injects" indent/dedent tokens based on indentation. It keeps track of the current indentation, as well as the current level of parentheses. Inside parentheses, the indentation is ignored, and no indent/dedent tokens get generated. Note: This is an abstract class. To use it, inherit and implement all its abstract methods: - tab_len - NL_type - OPEN_PAREN_types, CLOSE_PAREN_types - INDENT_type, DEDENT_type See also: the ``postlex`` option in `Lark`. paren_level indent_levelreturnNcCs d|_dg|_|jdksJdSNr)rrtab_lenselfrrr__init__ szIndenter.__init__tokenccs|jdkrdS|V|ddd}|d|d|j}||jdkr8|j|t|j||VdS||jdkrT|j t|j ||V||jdks?||jdkrft d||jdfdS)Nr r  z5Unexpected dedent to column %s. Expected dedent to %s) rrsplitcountrrappendr new_borrow_pos INDENT_typepop DEDENT_typer )rr indent_strindentrrr handle_NL%s    zIndenter.handle_NLccs|D]6}|j|jkr||EdHn|V|j|jvr%|jd7_q|j|jvr9|jd8_|jdks9Jqt|jdkrT|jt |j dVt|jdksA|jdgks_J|jdS)Nrr) typeNL_typer'OPEN_PAREN_typesrCLOSE_PAREN_typeslenrr#r r$)rstreamrrrr_process9s     zIndenter._processcCsd|_dg|_||Sr)rrr/)rr.rrrprocessLs zIndenter.processcCs|jfS)N)r*rrrr always_acceptRszIndenter.always_acceptcCt)zThe name of the newline tokenNotImplementedErrorrrrrr*VzIndenter.NL_typecCr2)z/The names of the tokens that open a parenthesisr3rrrrr+\r5zIndenter.OPEN_PAREN_typescCr2)z9The names of the tokens that close a parenthesis r3rrrrr,bszIndenter.CLOSE_PAREN_typescCr2)zeThe name of the token that starts an indentation in the grammar. See also: %declare r3rrrrr"izIndenter.INDENT_typecCr2)zbThe name of the token that end an indentation in the grammar. See also: %declare r3rrrrr$rr6zIndenter.DEDENT_typecCr2)z How many spaces does a tab equalr3rrrrr{r5zIndenter.tab_len)rN)r r r __doc__int__annotations__rrr rr'r/r0propertyr1rstrr*r+r,r"r$rrrrrrs8    rc@s0eZdZdZdZgdZgdZdZdZdZ dS) PythonIndenterzA postlexer that "injects" _INDENT/_DEDENT tokens based on indentation, according to the Python syntax. See also: the ``postlex`` option in `Lark`. _NEWLINE)LPARLSQBLBRACE)RPARRSQBRBRACE_INDENT_DEDENTN) r r r r7r*r+r,r"r$rrrrrr<sr<N)r7abcrrtypingrr exceptionsrlarkrlexerr r rr<rrrrs   s