o }oi6@s@ddlmZmZmZmZmZmZddlZGdddejj Z dS))AnyDictListOptionalTupleUnionNcsFeZdZdZfddZ d deejdeeejfddZ Z S) StatelessNeta Helper class used in transducer models with stateless decoders. This stateless simply outputs embedding or concatenated embeddings for the input label[s], depending on the configured context size. Args: context_size: history context size for the stateless decoder network. Could be any positive integer. We recommend setting this as 2. vocab_size: total vocabulary size. emb_dim: total embedding size of the stateless net output. blank_idx: index for the blank symbol for the transducer model. normalization_mode: normalization run on the output embeddings. Could be either 'layer' or None. We recommend using 'layer' to stabilize training. dropout: dropout rate on the embedding outputs. c st|dks J||_||_||_tj||_tj |_ |dkr.tjj |dd|_ g}t |jD],}|dkrC|d|j} n||d|j|jd} tjj |d| |d} || q5tj||_||_dS)NrlayerF)elementwise_affine) padding_idx)super__init__ context_size vocab_sizeemb_dimtorchnnDropoutdropoutIdentitynorm LayerNormrange Embeddingappend ModuleListembeds blank_idx) selfrrrrnormalization_moderri embed_sizeembed __class__g/home/ubuntu/.local/lib/python3.10/site-packages/nemo/collections/asr/parts/submodules/stateless_net.pyr#s$      zStatelessNet.__init__Nystatec Csg}|j\}}|}|dkrytj|d|gdd}|jd}||jkr@tj||jgtj|jd|j}||dd|j|df<n||jdkrR|ddddf}n|}t|jD]} |j | |dd|jd| |j| f} | | qYnDt|jD]>} |j | |} | dkr| ddd| ddf | dd| dddf<| ddd| ddfd9<| | q~| tj|dd} | | } d}|dur|dd|jd|jddfg}| |fS)aA Although this is a *stateless* net, we use the "state" parameter to pass in the previous labels, unlike LSTMs where state would represent hidden activations of the network. Args: y: a Integer tensor of shape B x U. state: a list of 1 tensor in order to be consistent with the stateful decoder interface, and the element is a tensor of shape [B x context-length]. Returns: The return dimension of this function's output is B x U x D, with D being the total embedding dim. Nrr )axis)dtypedeviceg)shaperconcatroneslongr-rrrrclonerr) r r)r*outsBU appended_yr padded_stater"outr'r'r(forwardCs@    . "  &zStatelessNet.forward)NN) __name__ __module__ __qualname____doc__rrrTensorrr: __classcell__r'r'r%r(rs ! r) typingrrrrrrrrModulerr'r'r'r(s