o }oi. @sddlmZddlZddlmZddlmZddlmZddl m Z m Z Gdddej Z ejj d'd ejd ejd ededejf ddZd(deededefddZd)dejdejdedefddZGdd d ej ZGd!d"d"ej ZGd#d$d$ej ZGd%d&d&ej ZdS)*)ListN)inf)nn) functional)get_same_padding init_weightsc s>eZdZdZddedededeffd d Zdd d Z Z S)StatsPoolLayeraStatistics and time average pooling (TAP) layer This computes mean and, optionally, standard deviation statistics across the time dimension. Args: feat_in: Input features with shape [B, D, T] pool_mode: Type of pool mode. Supported modes are 'xvector' (mean and standard deviation) and 'tap' (time average pooling, i.e., mean) eps: Epsilon, minimum value before taking the square root, when using 'xvector' mode. unbiased: Whether to use the biased estimator for the standard deviation when using 'xvector' mode. The default for torch.Tensor.std() is True. Returns: Pooled statistics with shape [B, D]. Raises: ValueError if an unsupported pooling mode is specified. xvector绽|=Tfeat_in pool_modeepsunbiasedcshtddh}||vrtd|d|d||_||_||_||_|jdkr2|jd9_dSdS)Nr tapzPool mode must be one of z; got '')super__init__ ValueErrorr r r r)selfr r r rsupported_modes __class__h/home/ubuntu/.local/lib/python3.10/site-packages/nemo/collections/asr/parts/submodules/tdnn_attention.pyr-s  zStatsPoolLayer.__init__Nc Cs|dur1|jdd}|jdkr-|jrdnd}|jd|dj|jd}tj||gdd}|S|}|St||dd }| |d }|jdd}||j d| d}|jdkr|jrZdnd}| | d |d  d d|dd |j|jd} tj|| fdd}|S|}|S) Ndimr r)r correction)minF)likelengths valid_onesgg@)meanr rstdclampr torchcatmake_seq_mask_like masked_fillshape unsqueezesubpowsumdivviewsqrt) rencoder_outputlengthr$rr%pooledmaskmeansstdsrrrforward:s8      zStatsPoolLayer.forward)r r TN) __name__ __module__ __qualname____doc__intstrfloatboolrr9 __classcell__rrrrrs  rTrr!r"r#time_dimreturncCsvtj|j||jd|jdd|d}t||D]}|d}q"|dkr4| |d}|s9|}|S)Ndevicerrr) r'aranger+rGrepeatltr,ranger transpose)r!r"r#rDr6_rrrr)[s0  r)lensmax_lenrGcCsNt||}|d|d|dk}|d}tj|ddd}||fS)aQ outputs masking labels for list of lengths of audio features, with max length of any mask as max_len input: lens: list of lens max_len: max length of any audio feature output: mask: masked labels num_values: sum of mask values for each feature (useful for computing statistics later) NrrrTrkeepdim)r'rHtor,r/)rNrOrGlens_matr6 num_valuesrrr lens_to_maskks   rUrr xmrr cCsDtj|||d}t||||d||}||fS)a compute mean and standard deviation of input(x) provided with its masking labels (m) input: x: feature input m: averaged mask labels output: mean: mean of input features std: stadard deviation of input features rr)r'r/r2r,r.r&)rVrWrr r$r%rrrget_statistics_with_mask}s *rXc sNeZdZdZ    ddededededed ef fd d Zdd d ZZS) TDNNModulea Time Delayed Neural Module (TDNN) - 1D input: inp_filters: input filter channels for conv layer out_filters: output filter channels for conv layer kernel_size: kernel weight size for conv layer dilation: dilation for conv layer stride: stride for conv layer padding: padding for conv layer (default None: chooses padding value such that input and output feature shape matches) output: tdnn layer output rN inp_filters out_filters kernel_sizedilationstridepaddingcsPt|durt|||d}tj|||||d|_t|_t||_ dS)N)r^r]) in_channels out_channelsr\r]r_) rrrrConv1d conv_layerReLU activation BatchNorm1dbn)rrZr[r\r]r^r_rrrrs  zTDNNModule.__init__cCs||}||}||Sr:)rcrerg)rrVr4rrrr9s   zTDNNModule.forward)rrrNr:r;r<r=r>r?rr9rCrrrrrYs(rYc sBeZdZdZd dededededef fdd Zdd d ZZS)MaskedSEModulea Squeeze and Excite module implementation with conv1d layers input: inp_filters: input filter channel size se_filters: intermediate squeeze and excite channel output and input size out_filters: output filter channel size kernel_size: kernel_size for both conv1d layers dilation: dilation size for both conv1d layers output: squeeze and excite layer output rrZ se_filtersr[r\r]c sLtttj||||dtt|tj||||dt|_dS)Nr\r]) rrr SequentialrbrdrfSigmoidse_layer)rrZrjr[r\r]rrrrs$  zMaskedSEModule.__init__NcCsb|dur tj|ddd}n|d}t|||jd\}}tj||ddd|}||}||S)NrT)rkeep_dimrOrGrP)r'r$sizerUrGr/rn)rinputr4rVrOr6rToutrrrr9s  zMaskedSEModule.forward)rrr:rhrrrrris$ ricsTeZdZdZ     ddededed ed ed ed effd d ZdddZZS) TDNNSEModuleaj Modified building SE_TDNN group module block from ECAPA implementation for faster training and inference Reference: ECAPA-TDNN Embeddings for Speaker Diarization (https://arxiv.org/pdf/2104.01466.pdf) inputs: inp_filters: input filter channel size out_filters: output filter channel size group_scale: scale value to group wider conv channels (deafult:8) se_channels: squeeze and excite output channel size (deafult: 1024/8= 128) kernel_size: kernel_size for group conv1d layers (default: 1) dilation: dilation size for group conv1d layers (default: 1) rxavier_uniformrZr[ group_scale se_channelsr\r] init_modec st||_t||dd}tj||||||d} tt||ddd| tt |t||ddd|_ t ||||_ | fdddS)Nr)r\r]r^)r\r]r_groupsrkcs t|dS)N)mode)r)rVrzrrs z'TDNNSEModule.__init__..)rrr[rrrbrlrYrdrfgroup_tdnn_blockrirnapply) rrZr[rxryr\r]rz padding_val group_convrr}rrs( zTDNNSEModule.__init__NcCs||}|||}||Sr:)rrn)rrrr4rVrrrr9s  zTDNNSEModule.forward)rurvrrrwr:) r;r<r=r>r?r@rr9rCrrrrrts."rtc sJeZdZdZ    ddedededed ef fd d Zdd dZZS)AttentivePoolLayera Attention pooling layer for pooling speaker embeddings Reference: ECAPA-TDNN Embeddings for Speaker Diarization (https://arxiv.org/pdf/2104.01466.pdf) inputs: inp_filters: input feature channel length from encoder attention_channels: intermediate attention channel size kernel_size: kernel_size for TDNN and attention conv1d layers (default: 1) dilation: dilation size for TDNN and attention conv1d layers (default: 1) rvrr rZattention_channelsr\r]r c sPtd||_tt|d|||dttj||||d|_||_ dS)Nrrk)r`rar\r]) rrr rrlrYTanhrbattention_layerr )rrZrr\r]r rrrr&s   zAttentivePoolLayer.__init__Nc Cs|d}|durtj|jd|jd}t|||jd\}}t|||\}}|ddd|}|ddd|}tj |||gdd}| |}| |dkt }t j|dd} t|| \} } tj | | fdddS)NrrrFrprr)rqr'onesr+rGrUrXr,rIr(rr*rFsoftmax) rrVr4rOr6rTr$r%attnalphamusgrrrr9>s  zAttentivePoolLayer.forward)rvrrr r:) r;r<r=r>r?rArr9rCrrrrrs$ r)Trr:)rr )typingrr'numpyrrtorch.nnrr,nemo.collections.asr.parts.submodules.jasperrrModulerjitscript_if_tracingTensorrBr?r)r@rUrArXrYrirtrrrrrs4    B  ,.5