o wiҾ@sddlZddlmZmZmZmZmZddlZddlm Z ddl m m Z ddlm Z ddlmZddlmZddlmZddlmZddlmZdd lmZzdd lmZdd lm Zdd lmZdd lmZdZ Wn e!yvdZ YnweZ"d6ddZ#d6ddZ$d7dee%fddZ&ddZ'de(fddZ)ddZ*ej+j,d ej d!e(d"ej fd#d$Z-ej+j,d ej d!e(d"ej fd%d&Z.ej+j,d'ej d(e(d)ej fd*d+Z/Gd,d-d-e j0Z1Gd.d/d/e j0Z2Gd0d1d1e j0Z3Gd2d3d3e j0eeZ4Gd4d5d5e j0Z5dS)8N)CallableIterableListOptionalTuple)Tensor)_calculate_correct_fan)_single)activation_registry) AccessMixin)AdapterModuleMixin)logging)calib)nn) quant_modules)QuantDescriptorTFfan_incCsZt||}d}|t|}|}t|| |WdS1s&wYdS)av Uniform Initialization from the paper [Sequence-to-Sequence Speech Recognition with Time-Depth Separable Convolutions](https://www.isca-speech.org/archive/Interspeech_2019/pdfs/2460.pdf) Normalized to - .. math:: \text{bound} = \text{2} \times \sqrt{\frac{1}{\text{fan\_mode}}} Args: tensor: an n-dimensional `torch.Tensor` mode: either ``'fan_in'`` (default) or ``'fan_out'``. Choosing ``'fan_in'`` preserves the magnitude of the variance of the weights in the forward pass. Choosing ``'fan_out'`` preserves the magnitudes in the backwards pass. @N)rmathsqrttorchno_graduniform_tensormodefangainstdboundr i/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/nemo/collections/asr/parts/submodules/jasper.py tds_uniform_+s   $r"cCsXt||}d}|t|}|}t|d|WdS1s%wYdS)au Normal Initialization from the paper [Sequence-to-Sequence Speech Recognition with Time-Depth Separable Convolutions](https://www.isca-speech.org/archive/Interspeech_2019/pdfs/2460.pdf) Normalized to - .. math:: \text{bound} = \text{2} \times \sqrt{\frac{1}{\text{fan\_mode}}} Args: tensor: an n-dimensional `torch.Tensor` mode: either ``'fan_in'`` (default) or ``'fan_out'``. Choosing ``'fan_in'`` preserves the magnitude of the variance of the weights in the forward pass. Choosing ``'fan_out'`` preserves the magnitudes in the backwards pass. rN)rrrrrnormal_rr r r! tds_normal_Bs   $r%xavier_uniformrcCsFt|tr t|j|t|tjtjfrs|durq|dkr'tjj|j dddS|dkr6tjj |j dddS|dkrEtjj |j dddS|dkrTtjj |j dddS|d kr_t |j dS|d krjt|j dStd |dSt|tjr|jr|j|jd |j|jrtj|j tj|jdSdSdS) Nr&?)r xavier_normalkaiming_uniformrelu) nonlinearitykaiming_normal tds_uniform tds_normalz Unknown Initialization mode: {0}) isinstance MaskedConv1d init_weightsconvrConv1dLinearinitxavier_uniform_weightxavier_normal_kaiming_uniform_kaiming_normal_r"r% ValueErrorformat BatchNorm1dtrack_running_stats running_meanzero_ running_varfill_num_batches_trackedaffineones_zeros_bias)mrr r r!r2Ys8      r2cCs*tt||d}|ddkr|d7}|S)Nr/r)maxint) kernel_size kernel_widthnew_kernel_sizer r r!compute_new_kernel_sizevs rPreturncCs(|dkr |dkr td||ddS)Nr/-Only stride OR dilation may be greater than 1rJ)r<)rMstridedilationr r r!get_same_padding~srUc Cs|dkr |dkr td|d|}|}t|||}||kr2td|d|d|d|d |S||krDtd|d |d||dkrZ||d||}||}||fS||fS) Nr/rRzEFuture context window is larger than the kernel size! Left context = z | Right context = greater than z | Kernel size = z@ Switching to symmetric padding (left context = right context = )zFuture context window is larger than half the kernel size! Conv layer therefore uses more future information than past to compute its output! Left context = z | Right context = )r<rUr warning)rMrSrTfuture_context left_context right_contextsymmetric_paddingr r r!get_asymtric_paddingsB  r\xcontext_windowmaskcCsf|jd}||krtj|ddd|jddd|j}|Stj|ddd|jddd|j}|S)a Calculates the masked average over padded limited context segment during inference mode. Args: x: Input tensor. Shape = [B, C, T] context_window: Integer context window, must be 0 or greater. mask: Mask tensor, 1 represents value index, 0 represents padded index. Shape = [B, 1, T]. Returns: A tensor reduced via masked average pool over some limited context. Shape = [B, C, 1] Tdimkeepdim)shapersumtodtype)r]r^r_ timestepsyr r r!_se_pool_step_script_infers & &rjcCs|jd}||krtj|ddd|jddd|j}|Stjd||dgtjdd}|dddd|||f}|dddd|||f}|jddd|j}|jddd}||d}|S) aO Calculates the masked average over padded limited context segment during training mode. Randomly slices a segment of length `context_window` from signal+padded input tensor across all channels and uses it for computing masked limited context. Args: x: Input tensor. Shape = [B, C, T] context_window: Integer context window, must be 0 or greater. mask: Mask tensor, 1 represents value index, 0 represents padded index. Shape = [B, 1, T]. Returns: A tensor reduced via masked average pool over some limited context. Shape = [B, C, 1] r`Trarr/)sizergNg:0yE>)rdrrerfrgrandintint32)r]r^r_rhri start_idxr r r!_se_pool_step_script_trains &  rolenscurrent_maxlenoriginal_maxlencCs4||krt|}t|}||fS|}|}||fSN)raranger)rprqrrnew_lens new_max_lensr r r!_masked_conv_init_lenss  rwcsXeZdZgdZ        dfdd Zd d Zd d ZdddZddZZ S)r1) use_conv_maskreal_out_channelsheadsr/rr`FTc s~tt||dks||kstd||_|dkr |}|}|}||_t|ttfvr6t j |dd|_ d}nd|_ t rL| rLt j|||||||| d|_nt sT| rTtdt j|||||||| d|_| |_||_|jjddkod |jjd|jjd|jjddk|_|j durd |_n|jjddkot|j|jjd|jjddk|_|jrtd|_td|_dSdS) Nr`z)Only use heads for depthwise convolutionsr#)valuer)rSpaddingrTgroupsrH~pytorch-quantization is not installed. Install from https://github.com/NVIDIA/TensorRT/tree/master/tools/pytorch-quantization.r/rJF)superr1__init__r<ry_paddingtypetuplelistr ConstantPad1d pad_layerPYTORCH_QUANTIZATION_AVAILABLEquant_nn QuantConv1dr3 ImportErrorr4use_maskrzrSr|rTrM same_paddingsame_padding_asymmetricrerrmax_lenrp) self in_channels out_channelsrMrSr|rTr}rzrHrquantize __class__r r!rsh   * $ zMaskedConv1d.__init__cCs|js|jr|S|jdur3tj|d|jjd|jjd|jjddd|jj ddddStj|t |j |jjd|jjddd|jj ddddS)NrJrr/trunc) rounding_mode) rrrrdivr3r|rTrMrSrer)rrpr r r! get_seq_lenDs&  2  , zMaskedConv1d.get_seq_lencCs|jr|d|jkr|j|d|jd|||}||}|jdur+||}|j}|j dkr=| d|j |d}| |}|j dkrQ| |d|j d}||fS)NrJdevicer`r) rrkrupdate_masked_lengthr mask_inputrrrdrzviewr3ry)rr]rpshoutr r r!forward[s       zMaskedConv1d.forwardNcCsH|durt|j||j\|_|_|j||_dS||_t||_dSrs)rwrprrfrrrr seq_rangerr r r!rts z!MaskedConv1d.update_masked_lengthcCsL|d}|jd|d|j|dk}||dj|jd}|S)NrJrr/r)rkrp unsqueezerfr)rr]rprr_r r r!r|s &zMaskedConv1d.mask_input)r/rr/r/r`FTF)NN) __name__ __module__ __qualname__ __constants__rrrrr __classcell__r r rr!r1sS r1cs$eZdZfddZddZZS) GroupShufflecs"tt|||_|||_dSrs)rrrr}channels_per_group)rr}channelsrr r!rszGroupShuffle.__init__cCsN|j}|d|j|j|d}t|dd}|d|j|j|d}|S)Nr`r/rJ)rdrr}rr transpose contiguous)rr]rr r r!rs zGroupShuffle.forward)rrrrrrr r rr!rs rc s~eZdZ    ddedededed eed ef fd d Zd dZ ddZ ddZ dddZ dddZ defddZZS) SqueezeExciter`nearestNFrreduction_ratior^interpolation_mode activationrc stt|||_||_d|_|durtjdd}tr7|r7t t j |||dd|t j |||dd|_ n ts?|r?t dt tj|||dd|tj|||dd|_ td|_|j|d|d dS) a Squeeze-and-Excitation sub-module. Args: channels: Input number of channels. reduction_ratio: Reduction ratio for "squeeze" layer. context_window: Integer number of timesteps that the context should be computed over, using stride 1 average pooling. If value < 1, then global context is computed. interpolation_mode: Interpolation mode of timestep dimension. Used only if context window is > 1. The modes available for resizing are: `nearest`, `linear` (3D-only), `bilinear`, `area` activation: Intermediate activation function used. Must be a callable activation function. NT)inplaceF)rHr~r/)r^)rrrr _quantizepoolrReLUr Sequentialr QuantLinearfcrr5AdaptiveAvgPool1dgapchange_context_window set_max_len)rrrr^rrrrr r!rs0   zSqueezeExcite.__init__cCs |||Srs)forward_for_export)rr]lengthsr r r!rs zSqueezeExcite.forwardcCs|jd}||jkr|||j}tjj|jjdd[|j |||jd}|}|j dj jtj kr8|j |jtj krB| }||d}|||}|dd}| |}|dd}t|}||}||}Wd||fS1szwY||fS)Nr`F)enabled)max_audio_lengthrrr#r/)rdrrrgrampautocastrr make_pad_maskrr8float32float masked_fill _se_pool_steprsigmoidrf)rr]rrrgr_rir r r!rs.             z SqueezeExcite.forward_for_exportcCsd|}|jdkrtj|ddd|jddd|j}|S|jr)t||j|}|St||j|}|S)Nrr`Tra)r^rrerrgtrainingrorj)rr]r_rir r r!rs & zSqueezeExcite._se_pool_stepcCsV||_|durt|j}tjd|j|d}t|dr!||_dS|jd|dddS)zSSets maximum input length. Pre-calculates internal seq_range mask. NrrrF) persistent) rnext parametersrrrthasattrrregister_bufferrr r r!rs  zSqueezeExcite.set_max_lencCsn|r|jj|kr|j||_|jj|jkr||jj}|jd||dd|dk}|d}|S)zMake masking for padding.Nrr`r/)rrrfexpandrkr)rseq_lensrrr_r r r!rs& zSqueezeExcite.make_pad_maskcCs,t|drtd|jd|||_dS)a Update the context window of the SqueezeExcitation module, in-place if possible. Will update the pooling layer to either nn.AdaptiveAvgPool1d() (for global SE) or nn.AvgPool1d() (for limited context SE). If only the context window is changing but still a limited SE context block - then the earlier instance of nn.AvgPool1d() will be updated. Args: context_window: An integer representing the number of input timeframes that will be used to compute the context. Each timeframe corresponds to a single window stride of the STFT features. Say the window_stride = 0.01s, then a context window of 128 represents 128 * 0.01 s of context to compute the Squeeze step. r^z0Changing Squeeze-Excitation context window from z to N)rr infor^)rr^r r r!r!s  z#SqueezeExcite.change_context_window)r`rNFrs)rrrrLstrrrboolrrrrrrrrr r rr!rs0;"   rcseZdZdZgdZddddddddd dd d d dd gd d dd dd d d d fdedeffdd Z      dddZ       d ddZd!ddZ de e e e e fde e e e e ffddZZS)" JasperBlocka Constructs a single "Jasper" block. With modified parameters, also constructs other blocks for models such as `QuartzNet` and `Citrinet`. - For `Jasper` : `separable` flag should be False - For `QuartzNet` : `separable` flag should be True - For `Citrinet` : `separable` flag and `se` flag should be True Note that above are general distinctions, each model has intricate differences that expand over multiple such blocks. For further information about the differences between models which use JasperBlock, please review the configs for ASR models found in the ASR examples directory. Args: inplanes: Number of input channels. planes: Number of output channels. repeat: Number of repeated sub-blocks (R) for this block. kernel_size: Convolution kernel size across all repeated sub-blocks. kernel_size_factor: Floating point scale value that is multiplied with kernel size, then rounded down to nearest odd integer to compose the kernel size. Defaults to 1.0. stride: Stride of the convolutional layers. dilation: Integer which defined dilation factor of kernel. Note that when dilation > 1, stride must be equal to 1. padding: String representing type of padding. Currently only supports "same" padding, which symmetrically pads the input tensor with zeros. dropout: Floating point value, determins percentage of output that is zeroed out. activation: String representing activation functions. Valid activation functions are : {"hardtanh": nn.Hardtanh, "relu": nn.ReLU, "selu": nn.SELU, "swish": Swish}. Defaults to "relu". residual: Bool that determined whether a residual branch should be added or not. All residual branches are constructed using a pointwise convolution kernel, that may or may not perform strided convolution depending on the parameter `residual_mode`. groups: Number of groups for Grouped Convolutions. Defaults to 1. separable: Bool flag that describes whether Time-Channel depthwise separable convolution should be constructed, or ordinary convolution should be constructed. heads: Number of "heads" for the masked convolution. Defaults to -1, which disables it. normalization: String that represents type of normalization performed. Can be one of "batch", "group", "instance" or "layer" to compute BatchNorm1D, GroupNorm1D, InstanceNorm or LayerNorm (which are special cases of GroupNorm1D). norm_groups: Number of groups used for GroupNorm (if `normalization` == "group"). residual_mode: String argument which describes whether the residual branch should be simply added ("add") or should first stride, then add ("stride_add"). Required when performing stride on parallel branch as well as utilizing residual add. residual_panes: Number of residual panes, used for Jasper-DR models. Please refer to the paper. conv_mask: Bool flag which determines whether to utilize masked convolutions or not. In general, it should be set to True. se: Bool flag that determines whether Squeeze-and-Excitation layer should be used. se_reduction_ratio: Integer value, which determines to what extend the hidden dimension of the SE intermediate step should be reduced. Larger values reduce number of parameters, but also limit the effectiveness of SE layers. se_context_window: Integer value determining the number of timesteps that should be utilized in order to compute the averaged context window. Defaults to -1, which means it uses global context - such that all timesteps are averaged. If any positive integer is used, it will utilize limited context window of that size. se_interpolation_mode: String used for interpolation mode of timestep dimension for SE blocks. Used only if context window is > 1. The modes available for resizing are: `nearest`, `linear` (3D-only), `bilinear`, `area`. stride_last: Bool flag that determines whether all repeated blocks should stride at once, (stride of S^R when this flag is False) or just the last repeated block should stride (stride of S when this flag is True). future_context: Int value that determins how many "right" / "future" context frames will be utilized when calculating the output of the conv kernel. All calculations are done for odd kernel sizes only. By default, this is -1, which is recomputed as the symmetric padding case. When future_context >= 0, will compute the asymmetric padding as follows : (left context, right context) = [K - 1 - future_context, future_context] Determining an exact formula to limit future context is dependent on global layout of the model. As such, we provide both "local" and "global" guidelines below. Local context limit (should always be enforced) - future context should be <= half the kernel size for any given layer - future context > kernel size defaults to symmetric kernel - future context of layer = number of future frames * width of each frame (dependent on stride) Global context limit (should be carefully considered) - future context should be layed out in an ever reducing pattern. Initial layers should restrict future context less than later layers, since shallow depth (and reduced stride) means each frame uses less amounts of future context. - Beyond a certain point, future context should remain static for a given stride level. This is the upper bound of the amount of future context that can be provided to the model on a global scale. - future context is calculated (roughly) as - (2 ^ stride) * (K // 2) number of future frames. This resultant value should be bound to some global maximum number of future seconds of audio (in ms). Note: In the special case where K < future_context, it is assumed that the kernel is too small to limit its future context, so symmetric padding is used instead. Note: There is no explicit limitation on the amount of future context used, as long as K > future_context constraint is maintained. This might lead to cases where future_context is more than half the actual kernel size K! In such cases, the conv layer is utilizing more of the future context than its current and past context to compute the output. While this is possible to do, it is not recommended and the layer will raise a warning to notify the user of such cases. It is advised to simply use symmetric padding for such cases. Example: Say we have a model that performs 8x stride and receives spectrogram frames with stride of 0.01s. Say we wish to upper bound future context to 80 ms. Layer ID, Kernel Size, Stride, Future Context, Global Context 0, K=5, S=1, FC=8, GC= 2 * (2^0) = 2 * 0.01 ms (special case, K < FC so use symmetric pad) 1, K=7, S=1, FC=3, GC= 3 * (2^0) = 3 * 0.01 ms (note that symmetric pad here uses 3 FC frames!) 2, K=11, S=2, FC=4, GC= 4 * (2^1) = 8 * 0.01 ms (note that symmetric pad here uses 5 FC frames!) 3, K=15, S=1, FC=4, GC= 4 * (2^1) = 8 * 0.01 ms (note that symmetric pad here uses 7 FC frames!) 4, K=21, S=2, FC=2, GC= 2 * (2^2) = 8 * 0.01 ms (note that symmetric pad here uses 10 FC frames!) 5, K=25, S=2, FC=1, GC= 1 * (2^3) = 8 * 0.01 ms (note that symmetric pad here uses 14 FC frames!) 6, K=29, S=1, FC=1, GC= 1 * (2^3) = 8 * 0.01 ms ... quantize: Bool flag whether to quantize the Convolutional blocks. layer_idx (int, optional): can be specified to allow layer output capture for InterCTC loss. Defaults to -1. ) conv_mask separable residual_moderesmconv r/same皙?NTFr`batchaddrrrX layer_idxc%sRtt||dkrtdtt|tr"fdd|D}nt|g}|dkr9t|d|d|d}n t |d|d|d|}||_ ||_ ||_ | |_ ||_||_||_||_d|_|}t}t|dD]*}|rudg} n|} ||j|||| ||| || |||d ||j| | d|}qm||j||||||| || |||d |r|t||||| |d ||_|}!| |_| rt}"|d kr|} ndg} t|dkr|g}!d |_|!D]}#t|j|#|d||| |d }$|"|$q|"|_t r|jrt!"t!j#j$|_%nt s|rt&d nd|_tj'|j| | d|_(dS)Nrz*currently only 'same' padding is supportedcsg|]}t|qSr )rP).0kkernel_size_factorr r! sz(JasperBlock.__init__..rr/) rMrSrTr|r}rzr normalization norm_groupsr) drop_probr)rr^rrr stride_addF)rMrrrSrr~))rrrr<rr0rrPrUr\inplanesplanesrrrserrinterctc_should_capturer ModuleListrangeextend_get_conv_bn_layer_get_act_dropout_layerappendrrcopydense_residuallenrrrTensorQuantizer QuantConv2ddefault_quant_desc_inputresidual_quantizerrrmout)%rrrrepeatrMrrSrTr|dropoutrresidualr}rrzrrrresidual_panesrrse_reduction_ratiose_context_windowse_interpolation_mode stride_lastrXrr padding_val inplanes_loopr3_ stride_val res_panesres_listiprrrr!rs      zJasperBlock.__init__rc Cst|j} | rt||||||||| | | d Str%| r%tj||||||||dSts-| r-tdtj||||||||dS)N)rSrTr|rHr}rzrr)rSrTr|rHr}r~)rr1rrrrrr4) rrrrMrSrTr|rHr}rzrrrr r r! _get_convNsP zJasperBlock._get_convc Cs| dkr|} | r&|j||||||||| | d |j||dddd||| d g}n|j||||||||| d g}| dkrD|tj| |dn6| d krS|tj||dn'| d krb|tjd|dn| d krr|tj|d d dntd| d|dkr|t|||S)Nr`)rSrTr|rHr}rzrr/r)rMrSrTr|rHr}r)rSrTr|rHr}rgroup) num_groups num_channelsinstancelayerrgMbP?g?)epsmomentumzNormalization method (z8) does not match one of [batch, layer, group, instance].)r rr GroupNormr>r<r)rrrrMrSrTr|rHr}rzrrrrlayersr r r!rsj  zJasperBlock._get_conv_bn_layercCs*|dur tjddd}|tj|dg}|S)Nr#g4@)min_valmax_val)p)rHardtanhDropout)rrrrr r r!rsz"JasperBlock._get_act_dropout_layerinput_rQcCsd}|d}t|dkr|\}}|d}|}t|jD]\}}t|ttfr.|||\}}q||}q|jdurt|jD]M\}}||} t|D]\} } t| trZ| | |\} } qI| | } qI|jdksi|jdkrtrv|j rv| || }q=ts|j rt d|| }q=t || }q=||}|r|} t| dkr|dd}||}|dd}|t|d dr|jd d r|jd |d |jdur|jdidg}|j|v|_|jr|jd|j|d |jd|j|d |jdur|jr||g|fS|g|fS)a  Forward pass of the module. Args: input_: The input is a tuple of two values - the preprocessed audio signal as well as the lengths of the audio signal. The audio signal is padded to the shape [B, D, T] and the lengths are a torch vector of length B. Returns: The output of the block after processing the input through `repeat` number of sub-blocks, as well as the lengths of the encoded audio after padding/striding. NrrJr`rrr~r/ model_guidsave_encoder_tensorsFencoder)namerinterctccapture_layerszinterctc/layer_output_zinterctc/layer_length_)r enumeraterr0r1rrrrrrrrrKris_adapter_availableget_enabled_adaptersrforward_enabled_adaptersis_access_enabledgetattr access_cfggetregister_accessible_tensorrrr)rr lens_origxsrrpilrres_outj res_layerr adapter_namesr"r r r!rsZ                 zJasperBlock.forward) rr/r/rFr/r`FF) rr/r/rFr/r`Frr/F)rN)rrr__doc__rrLrr rrrrrrrrr r rr!r9stq& > M:rc sheZdZdZ     ddedededed ef fd d Zd d Zdee e e e ffddZ Z S) ParallelBlocka Computational module that computes several `blocks` independently from each other and aggregates the outputs. It expects audio inputs to be passed together with lengths, just like Jasper blocks, and all outputs to have the same dimensions but it does not impose any additional requirements on the structure of the blocks themselves. Args: blocks: List of Jasper blocks that will be computed concurently. It is expected that they accept the same input and return outputs with the same number of channels. aggregation_mode: an optional string, indicating how the outputs will be aggregated. Supported values are ['sum', 'dropout']. "sum" value forces outputs to be summed together. "dropout" value enables tower dropout training with different blocks being dropped out during training. block_dropout_prob: a probability of dropping any individual block during training with "dropout" aggregation mode. Acts as a regularization technique. residual_mode: an optional string indicating how residuals will be applied. Supported values are ['sum', 'conv']. In 'sum' mode input features are summed together with the output. This will fail if the number of channels in the input is different from the number of channels in an output tensor. In 'conv' mode inputs are passed through pointwise convolution to make input channel dimension match output channel dimension. In this mode `in_filters` and `out_filters` params are required. in_filters: number of filters (channels) in the input tensor of each block. out_filters: number of filters (channels) in the output tensor of each block. rer#Naggregation_modeblock_dropout_probr in_filters out_filterscstt||_ddg|_||jvr!td|d|jd||_|dkr;tjt t |dd|_ t ||_ddg|_||jvrQtd |d|jd||_|dkrp|dus`|durdtd t||d dd d |_dSdS)Nrerz$Got non-supported aggregation mode: z. Supported values are .F) requires_gradr3z!Got non-supported residual mode: zPin_filters and out_filters have to be specified when using 'conv' residual mode.r/T)rMrHr)rrrrblockssupported_aggregationsr<r6 Parameterronesrweightsrrsupported_residualsrr1res_conv)rr<r6r7rr8r9rr r!rMs,      zParallelBlock.__init__cCsP||j}t|dkr&|jjdkr&||j}t|dkr&|jjdks|S)Nrr')rr@rrer)rr@r r r!get_dropout_maskps  zParallelBlock.get_dropout_maskr]c Cs t|jdkr|jd|Sd}d}d}|jdkr|}t|jD];\}}||\}}|d} |jdkr=|||d} |durD| }n|| }|durO|}q"tjt||gddd}q"|dd} |d} |jdkrr|| }n|jdkr|| | | d}|g|fS) a Forward pass computing aggregated output. Args: x: tuple of padded signal and lengths the signal. The shape of the signal is [B, D, T]. The lengths are 1D torch tensor of length B. Returns: torch tensor after passing input throught each block and aggregating these outputs according to the aggregation mode. r/rNrr`)rbrer3) rr<r6rCr#rrKstackrrB) rr]resultmax_maskscaling_weightsr.blockoutputr_weighted_output input_featrpr r r!rvs2         zParallelBlock.forward)rer#reNN)rrrr4rrLrrCrrrrrrr r rr!r56s(#&r5)r)r&)6rtypingrrrrrrtorch.nnrtorch.nn.functional functionalFr torch.nn.initrtorch.nn.modules.utilsr #nemo.collections.common.parts.utilsr nemo.core.classes.mixinsr 'nemo.core.classes.mixins.adapter_mixinsr nemo.utilsr pytorch_quantizationrrr!pytorch_quantization.tensor_quantrrrjasper_activationsr"r%rr2rPrLrUr\jitscriptrjrorwModuler1rrrr5r r r r!sZ               $ $