o }oi-@stddlmZmZddlZddlmZddlmZddlm Z m Z ddl m Z m Z mZddlmZGdd d e ZdS) )DictOptionalN)calculate_mean) wrap_to_pi) NeuralModule typecheck) LengthsType NeuralTypeSpectrogramType)loggingcseZdZdZ       d*dedeedeedeed ed eed eed effd d Z e de ee ffddZ e de ee ffddZe defddZe defddZed+dejdeejdejfddZe d,dejdeejd edejfddZee dee ed ed!d"e deid#dejdejdejfd$d%Zee dee ed ed!d"e deid#dejdejdejfd&d'Zedejdejdejfd(d)ZZS)-!SpectrogramToMultichannelFeaturesaConvert a complex-valued multi-channel spectrogram to multichannel features. Args: num_subbands: Expected number of subbands in the input signal num_input_channels: Optional, provides the number of channels of the input signal. Used to infer the number of output channels. mag_reduction: Reduction across channels. Default `None`, will calculate magnitude of each channel. mag_power: Optional, apply power on the magnitude. use_ipd: Use inter-channel phase difference (IPD). mag_normalization: Normalization for magnitude features ipd_normalization: Normalization for IPD features eps: Small regularization constant. NF:0yE> num_subbandsnum_input_channels mag_reduction mag_poweruse_ipdmag_normalizationipd_normalizationepsc st||_||_||_|dvrtd|||_|dvr'td|||_|jr6d||_||_ n ||_|jdur@|nd|_ ||_ t d|j jt d|t d|jt d |jt d |jt d |jt d |jt d |j t d|jt d|j dS)N)Nmeanmean_varz Unknown magnitude normalization zUnknown ipd normalization zInitialized %s withz num_subbands: %dz mag_reduction: %sz mag_power: %sz use_ipd: %sz mag_normalization: %sz ipd_normalization: %sz eps: %fz _num_features: %sz _num_channels: %s)super__init__rrrNotImplementedErrorrr _num_features _num_channelsrr debug __class____name__) selfrrrrrrrrr [/home/ubuntu/.local/lib/python3.10/site-packages/nemo/collections/audio/modules/features.pyr,s4   z*SpectrogramToMultichannelFeatures.__init__returncCtdttdtdS)+Returns definitions of module output ports.BCDTr*input input_lengthr r rr"r$r$r% input_typesX  z-SpectrogramToMultichannelFeatures.input_typescCr')r(r)r.)output output_lengthr2r3r$r$r% output_types`r5z.SpectrogramToMultichannelFeatures.output_typescCs|jS)zConfigured number of features)rr3r$r$r% num_featureshsz.SpectrogramToMultichannelFeatures.num_featurescCs|jdur|jStd)zConfigured number of channelsNzvNum channels is not configured. To configure this, `num_input_channels` must be provided when constructing the object.)r ValueErrorr3r$r$r% num_channelsms z.SpectrogramToMultichannelFeatures.num_channelsr0r1cCsZ|jdks Jd|j|durtj|ddd}|St||ddd}tj|ddd}|S) a,Calculate mean across time and channel dimensions. Args: input: tensor with shape (B, C, F, T) input_length: tensor with shape (B,) Returns: Mean of `input` calculated across time and channel dimension with shape (B, 1, F, 1) )Expected input to have 4 dimensions, got NTdimkeepdimr?r@)ndimtorchrr)r0r1rr$r$r%get_mean_time_channelxs z7SpectrogramToMultichannelFeatures.get_mean_time_channel绽|=cCs|jdks Jd|j|dur tj|dddd\}}||fS|||}||d}t||d dd }tj|d dd }t||}||fS) aeCalculate mean and standard deviation across time and channel dimensions. Args: input: tensor with shape (B, C, F, T) input_length: tensor with shape (B,) Returns: Mean and standard deviation of the `input` calculated across time and channel dimension, each with shape (B, 1, F, 1). r<r=Nr>FT)rBunbiasedrCrr?rAr@) rDrEstd_meanrFpowrrsqrtclamp)clsr0r1rstdrr$r$r%get_mean_std_time_channels z;SpectrogramToMultichannelFeatures.get_mean_std_time_channelr)r*r/r6)r4r8cCs|j||d}||}|S)zMean normalization for the input tensor. Args: input: input tensor input_length: valid length for each example Returns: Mean normalized input. r/)rF)r"r0r1rr6r$r$r%normalize_meansz0SpectrogramToMultichannelFeatures.normalize_meancCs&|j|||jd\}}|||}|S)zMean and variance normalization for the input tensor. Args: input: input tensor input_length: valid length for each example Returns: Mean and variance normalized input. )r0r1r)rOr)r"r0r1rrNr6r$r$r%normalize_mean_vars z4SpectrogramToMultichannelFeatures.normalize_mean_varcCs|d}|jdurt|}n@|jdkr!ttj|ddd}n/|jdkr2tjt|ddd}n|jdkrHttjt|dddd}ntd |j|jdur\t||j}|j d kri|j ||d }n |j d kru|j ||d }|}|j r|dkrtj ||j|jd }n/tj|ddd}t|t|}t|}|jd kr|j ||d }n |jd kr|j ||d }tj||j|gdd}|jdur|d|jkrtd|dd|j||fS)a\Convert input batch of C-channel spectrograms into a batch of time-frequency features with dimension num_feat. The output number of channels may be the same as input, or reduced to 1, e.g., if averaging over magnitude and not appending individual IPDs. Args: input: Spectrogram for C channels with F subbands and N time frames, (B, C, F, N) input_length: Length of valid entries along the time dimension, shape (B,) Returns: num_feat_channels channels with num_feat features, shape (B, num_feat_channels, num_feat, N) rNabs_meanT)axisrCmean_absrmsrzUnexpected magnitude reduction rr/r)dtypedevice)rSzNumber of channels in features z5 is different than the configured number of channels )sizerrEabsrrKr:rrJrrPrQr zeros_likerVrWanglerrcatexpandshaper RuntimeError)r"r0r1rmagfeaturesipd spec_meanr$r$r%forwardsB      "     z)SpectrogramToMultichannelFeatures.forward)NNNFNNr )N)NrG)r! __module__ __qualname____doc__intrstrfloatboolrpropertyrr r4r8r9r; staticmethodrETensorrF classmethodrOrr tuplerrPrQrd __classcell__r$r$r#r%r s , $       &r )typingrrrE#nemo.collections.audio.losses.audior(nemo.collections.audio.parts.utils.audiornemo.core.classesrrnemo.core.neural_typesrr r nemo.utilsr r r$r$r$r%s