o wiL! @sddlmZddlmZmZmZddlZddlmZdde dee de d e d d f d d Z ddee d e d d fddZ ddee d e d d fddZddee d e d d fddZdee d d fddZGdddeZdS)) lru_cache)ListOptionalUnionN)k2Tnametokens blank_numwith_self_loopsreturnk2.FsacCs|dkr t||}n$|dkrt||}n|dkrt||}n|dkr't|}ntd||dkrN|j}|dk}||dk||k@d8<|||<||_t|}|S) aHelper function to build a topology. It allows to build topologies with a non-zero blank ID. Args: name: The topology name. Choices: default, compact, shared_blank, minimal tokens: A list of tokens, e.g., phones, characters, etc. blank_num: Blank number. Must be in tokens with_self_loops: Whether to add token-to-epsilon self-loops to a topology Returns: Returns a topology FST. defaultcompact shared_blankminimalzUnknown topo name: r)build_default_topobuild_compact_topobuild_shared_blank_topobuild_minimal_topo ValueErrorlabelsrarc_sort)rrr r ansr blank_maskre/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/nemo/collections/asr/parts/k2/topologies.py build_topos"     rc Csd|vsJdd|vsJdt|}|}|rdnd}t|D]=}t|D],}||kr>|r=||d|d||d7}q&||d|d||d||d 7}q&||d|d 7}q ||7}tjj|d d }t|}|S) zZBuild the default CTC topology. Zero is assumed to be the ID of the blank symbol. r*We assume -1 is ID of the final transitionr)We assume 0 is the ID of the blank symbolz 0 0 0 0 0.0   0 0.0  0.0 -1 -1 0.0 rnum_aux_labelslenrangerFsafrom_strr)rr num_states final_statearcsijrrrrr:s"   *  rc Csd|vsJdd|vsJd|dd}t|}t||}|}d}t||D]}|d|d|||d|||d 7}q)|d|d 7}td|D]}||d |d 7}|rn||d|d|||d 7}qO||7}tjj|dd }t|}|S)zBuild the compact CTC topology. Zero is assumed to be the ID of the blank symbol. See https://arxiv.org/abs/2110.03098 rrrr rr!0 r"r$r%z 0 r#r&)intr)r*rr+r,r) rr eps_numselfloops_shiftr-r.r/r0rrrrrRs&  ."  rc Cs@d|vsJdd|vsJd|}|dt|}d}|d}g}|||dddg|||dddg||gt|D]2\}}|d7}|||||dg|||||dg||||ddg|ru||||ddgqCt|ddd}d d |D}d d |D}d |}tjj |dd }t |}|S)zBuild the shared blank CTC topology. Zero is assumed to be the ID of the blank symbol. See https://github.com/k2-fsa/k2/issues/746#issuecomment-856421616 rrrr rcSs|dS)Nrr)arcrrrsz)build_shared_blank_topo..)keycSsg|] }dd|DqS)cSsg|]}t|qSr)str.0r0rrr sz6build_shared_blank_topo...rr;r6rrrr<sz+build_shared_blank_topo..cSsg|]}d|qS)r")joinr=rrrr< r&) copyremover)append enumeratesortedr>rr+r,r) rr num_tokensstartfinalr/r0prrrrrls4    rcCsd|vsJdd|vsJdt|}d}d}t|D]}|d||d||d 7}q|d |d 7}||7}tjj|dd }t|}|S) zBuild the minimal topology. Zero is assumed to be the ID of the blank symbol. See https://arxiv.org/abs/2110.03098 rrrr rr!z0 0 r"r$r2r%r&r()rrFr.r/r0rrrrrs    rc@sleZdZdZddeededeefddZdee j eefd d fd d Z e d dded d fddZ dS)RnntEmissionAdapterBuilderaBuilder class for RNNT Emission Adapters. An Emission Adapter is an FSA used to emulate desired temporal Emissions FSA properties of a trivial Emissions FSA. Temporal properties are emulated by -arcs with zero log-weight. These additional arcs do not contribute to the lattice scores and can be easily removed from the best path. k2 does not have Emissions FSAs. Instead, it has DenseFsaVec, which is not a real FSA. Thus, Emission Adapters should be composed with Supervision FSAs. IMPOTRANT: -outputs are expected to be present in the DenseFsaVec. These RNNT adapters do only the re-routing (emulate hopping over U dimension). Redundant non- are not removed by these adapters. At initialization, the builder expects a list of tokens, number and number. When called, the builder returns adapters according to the provided text lengths. Nrr r4cCsjd|vsJd||vsJd|dus||vsJd||_||_|dur0|jdd|_dS||_dS)NrrzThe blank ID must be in tokensz#The epsion ID must not be in tokensr)rr r4)selfrr r4rrr__init__s &z#RnntEmissionAdapterBuilder.__init__adapter_lengthsr r cstfdd|DS)Ncsg|]}|qSr)_build_single_adapterr:rKrrr<r?z7RnntEmissionAdapterBuilder.__call__..)rcreate_fsa_vectolist)rKrMrrOr__call__sz#RnntEmissionAdapterBuilder.__call__i)maxsizeadapter_lengthc Cs$|dksJd|d}|dd}d}t|D]1}tt|jD]}||jkr:||d|dd|j|d7}q!||d|d|jd7}q||d|d|jd7}t||D]$}||d||dkrm|dndd|jd7}||d|d7}q]||7}ttjj|d d S) Nrz(`adapter_length` cannot be less than oner!r"r$rz -1 0.0 T)acceptor) r*r)rr r4rrr+r,)rKrTfirst_eps_stater.r/r0r1rrrrNs    $0 z0RnntEmissionAdapterBuilder._build_single_adapter)N)__name__ __module__ __qualname____doc__rr3rrLrtorchTensorrRrrNrrrrrJs   rJ)T) functoolsrtypingrrrr\nemo.core.utils.k2_guardrr9r3boolrrrrrobjectrJrrrrs  $#!