o piY@sBdZddlZddlZddlmZddlmZmZmZm Z ddl m Z ddl m Z e eZGdddejjZGd d d ejjZ d#dededefddZGdddejjZGdddejjZGdddejjZGdddejjZe GdddejjZGdddejjZGdd d ejjZGd!d"d"ejjZdS)$a&Low-level feature pipeline components This library gathers functions that compute popular speech features over batches of data. All the classes are of type nn.Module. This gives the possibility to have end-to-end differentiability and to backpropagate the gradient through them. Our functions are a modified version the ones in torch audio toolkit (https://github.com/pytorch/audio). Example ------- >>> import torch >>> from speechbrain.dataio.dataio import read_audio >>> signal =read_audio('tests/samples/single-mic/example1.wav') >>> signal = signal.unsqueeze(0) >>> compute_STFT = STFT( ... sample_rate=16000, win_length=25, hop_length=10, n_fft=400 ... ) >>> features = compute_STFT(signal) >>> features = spectral_magnitude(features) >>> compute_fbanks = Filterbank(n_mels=40) >>> features = compute_fbanks(features) >>> compute_mfccs = DCT(input_size=40, n_out=20) >>> features = compute_mfccs(features) >>> compute_deltas = Deltas(input_size=20) >>> delta1 = compute_deltas(features) >>> delta2 = compute_deltas(delta1) >>> features = torch.cat([features, delta1, delta2], dim=2) >>> compute_cw = ContextWindow(left_frames=5, right_frames=5) >>> features = compute_cw(features) >>> norm = InputNormalization() >>> features = norm(features, torch.tensor([1]).float()) Authors * Mirco Ravanelli 2020 N)length_to_mask)mark_as_loader mark_as_savermark_as_transferregister_checkpoint_hooks)FilterProperties) get_loggercsJeZdZdZdddejddddffdd Zd d Zd efd dZ Z S)STFTucomputes the Short-Term Fourier Transform (STFT). This class computes the Short-Term Fourier Transform of an audio signal. It supports multi-channel audio inputs (batch, time, channels). Arguments --------- sample_rate : int Sample rate of the input audio signal (e.g 16000). win_length : float Length (in ms) of the sliding window used to compute the STFT. hop_length : float Length (in ms) of the hope of the sliding window used to compute the STFT. n_fft : int Number of fft point of the STFT. It defines the frequency resolution (n_fft should be <= than win_len). window_fn : function A function that takes an integer (number of samples) and outputs a tensor to be multiplied with each window before fft. normalized_stft : bool If True, the function returns the normalized STFT results, i.e., multiplied by win_length^-0.5 (default is False). center : bool If True (default), the input will be padded on both sides so that the t-th frame is centered at time t×hop_length. Otherwise, the t-th frame begins at time t×hop_length. pad_mode : str It can be 'constant','reflect','replicate', 'circular', 'reflect' (default). 'constant' pads the input tensor boundaries with a constant value. 'reflect' pads the input tensor using the reflection of the input boundary. 'replicate' pads the input tensor using replication of the input boundary. 'circular' pads using circular replication. onesided : True If True (default) only returns nfft/2 values. Note that the other samples are redundant due to the Fourier transform conjugate symmetry. Example ------- >>> import torch >>> compute_STFT = STFT( ... sample_rate=16000, win_length=25, hop_length=10, n_fft=400 ... ) >>> inputs = torch.randn([10, 16000]) >>> features = compute_STFT(inputs) >>> features.shape torch.Size([10, 101, 201, 2])  FTconstantc s~t||_||_||_||_||_||_||_| |_ t t |jd|j|_t t |jd|j|_||j|_ dSNg@@) super__init__ sample_rate win_length hop_lengthn_fftnormalized_stftcenterpad_modeonesidedintroundwindow) selfrrrr window_fnrrrr __class__]/home/ubuntu/SoloSpeech/.venv/lib/python3.10/site-packages/speechbrain/processing/features.pyris z STFT.__init__c Cs|j}t|dkr|dd}||d|d|d}tj||j|j|j|j |j |j |j |j|jdd }t|}t|dkrd||d|d|jd|jd|jd}|ddddd}|S|dd}|S)zReturns the STFT generated from the input waveforms. Arguments --------- x : torch.Tensor A batch of audio signals to transform. Returns ------- stft : torch.Tensor rT)return_complex)shapelen transposereshapetorchstftrrrrtodevicerrrr view_as_realpermute)rxor_shaper,r r r!forwards:     z STFT.forwardreturncCs|jstdt|j|jdS)Nz]ValueProperties cannot model a non-centered STFT, as it assumes either centering or causality) window_sizestride)r ValueErrorrrrrr r r!get_filter_propertiesszSTFT.get_filter_properties) __name__ __module__ __qualname____doc__r+hamming_windowrr3rr9 __classcell__r r rr!r 6s5 1r cs>eZdZdZdddejddddffdd Zd d d ZZS) ISTFTaFComputes the Inverse Short-Term Fourier Transform (ISTFT) This class computes the Inverse Short-Term Fourier Transform of an audio signal. It supports multi-channel audio inputs (batch, time_step, n_fft, 2, n_channels [optional]). Arguments --------- sample_rate : int Sample rate of the input audio signal (e.g. 16000). n_fft : int Number of points in FFT. win_length : float Length (in ms) of the sliding window used when computing the STFT. hop_length : float Length (in ms) of the hope of the sliding window used when computing the STFT. window_fn : function A function that takes an integer (number of samples) and outputs a tensor to be used as a window for ifft. normalized_stft : bool If True, the function assumes that it's working with the normalized STFT results. (default is False) center : bool If True (default), the function assumes that the STFT result was padded on both sides. onesided : True If True (default), the function assumes that there are n_fft/2 values for each time frame of the STFT. epsilon : float A small value to avoid division by 0 when normalizing by the sum of the squared window. Playing with it can fix some abnormalities at the beginning and at the end of the reconstructed signal. The default value of epsilon is 1e-12. Example ------- >>> import torch >>> compute_STFT = STFT( ... sample_rate=16000, win_length=25, hop_length=10, n_fft=400 ... ) >>> compute_ISTFT = ISTFT( ... sample_rate=16000, win_length=25, hop_length=10 ... ) >>> inputs = torch.randn([10, 16000]) >>> outputs = compute_ISTFT(compute_STFT(inputs)) >>> outputs.shape torch.Size([10, 16000]) Nr r FTg-q=c s~t||_||_||_||_||_||_||_| |_ t t |jd|j|_t t |jd|j|_||j|_ dSr) rrrrrrrrrepsilonrrr) rrrrrrrrrrArr r!rs zISTFT.__init__c Cs|j}|jdur|jr|jddd}n|jdur#|js#|jd}n|j}t|dkrG|ddddd}|d|jd|jd|jd}nt|dkrU|dddd}t|d |d }tj|||j |j |j |j |j|j|d }t|dkr||d|dd}|dd}|S) aReturns the ISTFT generated from the input signal. Arguments --------- x : torch.Tensor A batch of audio signals in the frequency domain to transform. sig_length : int The length of the output signal in number of samples. If not specified will be equal to: (time_step - 1) * hop_length + n_fft Returns ------- istft : torch.Tensor Nr$r#rr&r").r).r#)inputrrrrrrlength)r'rrr(r0r*r+complexistftrrrr-r.rr))rr1 sig_lengthr2rrGr r r!r3s4  $    z ISTFT.forwardN) r:r;r<r=r+r>rr3r?r r rr!r@s5!r@r#F+=powerlogepscCs@|dd}|dkr||}||}|rt||S|S)ayReturns the magnitude of a complex spectrogram. Arguments --------- stft : torch.Tensor A tensor, output from the stft function. power : int What power to use in computing the magnitude. Use power=1 for the power spectrogram. Use power=0.5 for the magnitude spectrogram. log : bool Whether to apply log to the spectral features. eps : float A small value to prevent square root of zero. Returns ------- spectr : torch.Tensor Example ------- >>> a = torch.Tensor([[3, 4]]) >>> spectral_magnitude(a, power=0.5) tensor([5.]) r$rCr#)powsumr+rL)r,rKrLrMspectrr r r!spectral_magnitudeSs rQcseZdZdZ        d fdd ZddZeddZeddZddZ ddZ e d fddZ ddZddZZS)! Filterbankacomputes filter bank (FBANK) features given spectral magnitudes. Arguments --------- n_mels : float Number of Mel filters used to average the spectrogram. log_mel : bool If True, it computes the log of the FBANKs. filter_shape : str Shape of the filters ('triangular', 'rectangular', 'gaussian'). f_min : int Lowest frequency for the Mel filters. f_max : int Highest frequency for the Mel filters. n_fft : int Number of fft points of the STFT. It defines the frequency resolution (n_fft should be<= than win_len). sample_rate : int Sample rate of the input audio signal (e.g, 16000) power_spectrogram : float Exponent used for spectrogram computation. amin : float Minimum amplitude (used for numerical stability). ref_value : float Reference value used for the dB scale. top_db : float Minimum negative cut-off in decibels. param_change_factor : bool If freeze=False, this parameter affects the speed at which the filter parameters (i.e., central_freqs and bands) can be changed. When high (e.g., param_change_factor=1) the filters change a lot during training. When low (e.g. param_change_factor=0.1) the filter parameters are more stable during training param_rand_factor : float This parameter can be used to randomly change the filter parameters (i.e, central frequencies and bands) during training. It is thus a sort of regularization. param_rand_factor=0 does not affect, while param_rand_factor=0.15 allows random variations within +-15% of the standard values of the filter parameters (e.g., if the central freq is 100 Hz, we can randomly change it from 85 Hz to 115 Hz). freeze : bool If False, it the central frequency and the band of each filter are added into nn.parameters. If True, the standard frozen features are computed. Example ------- >>> import torch >>> compute_fbanks = Filterbank() >>> inputs = torch.randn([10, 101, 201]) >>> features = compute_fbanks(inputs) >>> features.shape torch.Size([10, 101, 40]) (T triangularr@r >r$绽|=?T@cst||_||_||_||_||_||_||_||_ | |_ | |_ | |_ ||_ |jdd|_tt|j |j |_td|_| |_| |_|j dkrQd|_nd|_|j|jkrid|j|jf}tj|ddt||j||j|jd}||}|dd|dd }|dd |_|dd |_|j stj !|j|j|j|_tj !|j|j|j|_td |jd|j}|"|jj#d d|_$dS) Nr$r#cpur zRequire f_min: %f < f_max: %fT)exc_inforCr)%rrn_melslog_mel filter_shapef_minf_maxrrpower_spectrogramamin ref_valuetop_dbfreezen_stftmathlog10max db_multiplierr+r. device_inpparam_change_factorparam_rand_factor multiplierloggererrorlinspace_to_mel_to_hzband f_centralnn Parameterrepeatr' all_freqs_mat)rr^r_r`rarbrrrcrdrerfrnrorgerr_msgmelhzrv all_freqsrr r!rsV     zFilterbank.__init__c Csp|j|jjdddd}|j|jjdddd}|js8||j|j|j}||j|j|j}n#|j dkr[|j r[dt dd|j |j }||d}||d}| |||j}|j}t|dkr|dddd}||d|d|d|d}t ||}|jr||}t|dkr|j}||d|d|d|d}|dddd}|S)zReturns the FBANks. Arguments --------- spectrogram : torch.Tensor A batch of spectrogram tensors. Returns ------- fbanks : torch.Tensor r#rrXr$r&r")rwrzr{r'r)rvrgrrnrotrainingr+rand_create_fbank_matrixr-r.r(r0r*matmulr__amplitude_to_DB) r spectrogram f_central_matband_mat rand_change fbank_matrixsp_shapefbanksfb_shaper r r!r3sf         zFilterbank.forwardcCsdtd|dS)zReturns mel-frequency value corresponding to the input frequency value in Hz. Arguments --------- hz : float The frequency point in Hz. Returns ------- The mel-frequency value # r#)rirj)r~r r r!rtJszFilterbank._to_melcCsdd|ddS)aReturns hz-frequency value corresponding to the input mel-frequency value. Arguments --------- mel : float The frequency point in the mel-scale. Returns ------- The hz-frequency value rr rr#r )r}r r r!ruZszFilterbank._to_hzc CsN|||}|d}| d}tjd|jd}t|t||dd}|S)aReturns fbank matrix using triangular filters. Arguments --------- all_freqs : torch.Tensor torch.Tensor gathering all the frequency points. f_central : torch.Tensor torch.Tensor gathering central frequencies of each filter. band : torch.Tensor torch.Tensor gathering the bands of each filter. Returns ------- fbank_matrix : torch.Tensor rXr#r.r)r+zerosrmrkminr)) rrrwrvslope left_side right_sidezerorr r r!_triangular_filtersjs   zFilterbank._triangular_filtersc Cs@||}||}||}}||}||dd}|S)aReturns fbank matrix using rectangular filters. Arguments --------- all_freqs : torch.Tensor torch.Tensor gathering all the frequency points. f_central : torch.Tensor torch.Tensor gathering central frequencies of each filter. band : torch.Tensor torch.Tensor gathering the bands of each filter. Returns ------- fbank_matrix : torch.Tensor rr#)gelefloatr)) rrrwrvlow_hzhigh_hzr right_sizerr r r!_rectangular_filterss  zFilterbank._rectangular_filterscCs*td||||ddd}|S)aXReturns fbank matrix using gaussian filters. Arguments --------- all_freqs : torch.Tensor torch.Tensor gathering all the frequency points. f_central : torch.Tensor torch.Tensor gathering central frequencies of each filter. band : torch.Tensor torch.Tensor gathering the bands of each filter. smooth_factor: torch.Tensor Smoothing factor of the gaussian filter. It can be used to employ sharper or flatter filters. Returns ------- fbank_matrix : torch.Tensor gr$rr#)r+expr))rrrwrv smooth_factorrr r r!_gaussian_filterss zFilterbank._gaussian_filterscCsP|jdkr||j||}|S|jdkr||j||}|S||j||}|S)aReturns fbank matrix to use for averaging the spectrum with the set of filter-banks. Arguments --------- f_central_mat : torch.Tensor torch.Tensor gathering central frequencies of each filter. band_mat : torch.Tensor torch.Tensor gathering the bands of each filter. Returns ------- fbank_matrix : torch.Tensor rT rectangular)r`rr{rr)rrrrr r r!rs   zFilterbank._create_fbank_matrixcCs^|jttj||jd}||j|j8}|jdd|j}t|| |j ddd}|S)zConverts linear-FBANKs to log-FBANKs. Arguments --------- x : torch.Tensor A batch of linear FBANK tensors. Returns ------- x_db : torch.Tensor r)rCdimrr#) rpr+rjclamprdrlamaxrfrkviewr')rr1x_db new_x_db_maxr r r!rs  zFilterbank._amplitude_to_DB)rSTrTrrUr rVr$rWrXrYrXrZT)r:r;r<r=rr3 staticmethodrtrurrr+tensorrrrr?r r rr!rR{s89KL    rRc*eZdZdZdfdd ZddZZS) DCTawComputes the discrete cosine transform. This class is primarily used to compute MFCC features of an audio signal given a set of FBANK features as input. Arguments --------- input_size : int Expected size of the last dimension in the input. n_out : int Number of output coefficients. ortho_norm : bool Whether to use orthogonal norm. Example ------- >>> import torch >>> inputs = torch.randn([10, 101, 40]) >>> compute_mfccs = DCT(input_size=inputs.size(-1)) >>> features = compute_mfccs(inputs) >>> features.shape torch.Size([10, 101, 20]) r\Tcst||krtd||ftt|}tt|d}ttj t||d|}|rM|ddt d9<|t dt|9}n|d9}| |_ dS)NzCCannot select more DCT coefficients than inputs (n_out=%i, n_in=%i)r#g?rrX@) rrr7r+aranger unsqueezecosripisqrttdct_mat)r input_sizen_out ortho_normnkdctrr r!rs  z DCT.__init__cCs|j}t|dkr||jd|jd|jd|jd}t||j|j}t|dkrA||d|jd|jd|d}|S)zReturns the DCT of the input tensor. Arguments --------- x : torch.Tensor A batch of tensors to transform, usually fbank features. Returns ------- dct : torch.Tensor r&rr"r#r$)r'r(r*r+rrr-r.)rr1 input_shaperr r r!r3's * z DCT.forward)r\Tr:r;r<r=rr3r?r r rr!rsrc*eZdZdZdfdd ZddZZS)DeltasaComputes delta coefficients (time derivatives). Arguments --------- input_size : int The expected size of the inputs for parameter initialization. window_length : int Length of the window used to compute the time derivatives. Example ------- >>> inputs = torch.randn([10, 101, 20]) >>> compute_deltas = Deltas(input_size=inputs.size(-1)) >>> features = compute_deltas(inputs) >>> features.shape torch.Size([10, 101, 20]) rBcsnt|dd|_|j|jdd|jdd|_|dtj|j |jdtjd|dddS)Nr#r$r"kernel)dtype) rrrdenomregister_bufferr+rfloat32rz)rr window_lengthrr r!rWs $ zDeltas.__init__cCs|dddd}|j}t|dkr$||d|d|d|d}tjjj||j|jfdd}tjjj ||j |j |jdd |j }t|dkr]||d|d|d|d}|dddd}|S) zReturns the delta coefficients. Arguments --------- x : torch.Tensor A batch of tensors. Returns ------- delta_coeff : torch.Tensor r#r$rCr&rr" replicate)mode)groups)r)r'r(r*r+rx functionalpadrconv1drr-r.r)rr1r2 delta_coeffr r r!r3es"  " zDeltas.forward)rBrr r rr!rDsrcr) ContextWindowaComputes the context window. This class applies a context window by gathering multiple time steps in a single feature vector. The operation is performed with a convolutional layer based on a fixed kernel designed for that. Arguments --------- left_frames : int Number of left frames (i.e, past frames) to collect. right_frames : int Number of right frames (i.e, future frames) to collect. Example ------- >>> import torch >>> compute_cw = ContextWindow(left_frames=5, right_frames=5) >>> inputs = torch.randn([10, 101, 20]) >>> features = compute_cw(inputs) >>> features.shape torch.Size([10, 101, 220]) rcst||_||_|j|jd|_dt|j|jd|_t|j|j|_ |j|jkr>|j|j}t |j |d|_ d|_ dS)Nr#r$T) rr left_frames right_frames context_lenrk kernel_lenr+eyerroll first_call)rrrlagrr r!rs    zContextWindow.__init__cCs|dd}|jdur(d|_|j|jddd|jd|j|jd|_|j}t |dkrB| |d|d|d|d}t j j j||j|j|jdt|j|jd}t |dkrr| |d|jd|d|jd }|dd}|S) zReturns the tensor with the surrounding context. Arguments --------- x : torch.Tensor A batch of tensors. Returns ------- cw_x : torch.Tensor The context-enriched tensor r#r$TFr&rr")rpaddingrC)r)rrrzr'rrrrr(r*r+rxrrr-r.rkrr)rr1r2cw_xr r r!r3s,  "    zContextWindow.forward)rrrr r rr!rsrcseZdZUdZddlmZeeejfe d<eeejfe d<eeefe d<    dfd d Z e gdfddZ ddZ ddZddZfddZeddZeedddZZS)InputNormalizationaPerforms mean and variance normalization of the input tensor. Arguments --------- mean_norm : True If True, the mean will be normalized. std_norm : True If True, the standard deviation will be normalized. norm_type : str It defines how the statistics are computed ('sentence' computes them at sentence level, 'batch' at batch level, 'speaker' at speaker level, while global computes a single normalization vector for all the sentences in the dataset). Speaker and global statistics are computed with a moving average approach. avg_factor : float It can be used to manually set the weighting factor between current statistics and accumulated ones. requires_grad : bool Whether this module should be updated using the gradient during training. update_until_epoch : int The epoch after which updates to the norm stats should stop. Example ------- >>> import torch >>> norm = InputNormalization() >>> inputs = torch.randn([10, 101, 20]) >>> inp_len = torch.ones([10]) >>> features = norm(inputs, inp_len) r)Dict spk_dict_mean spk_dict_stdspk_dict_countTglobalNFr"csrt||_||_||_||_||_tdg|_ tdg|_ i|_ i|_ i|_ d|_d|_d|_||_dS)NrrXrW)rr mean_normstd_norm norm_type avg_factor requires_gradr+r glob_meanglob_stdrrrweightcountrMupdate_until_epoch)rrrrrrrrr r!r s  zInputNormalization.__init__cCsJ|jd}g}g}|jdks|jdkrt|}t|D]} t|| |jd} ||| d| df\} } || || |jdkrU|| | j | j || <|jdkrt|| d} |j r| |j vrz| |j | <| |j | <d|j | <nT|j | d|j | <|jdurd|j | |_n|j|_d|j|j | | |j| |j | <d|j|j | | |j| |j | <|j | |j | |j | j }|j | j }n| |j vr|j | j }|j | j }n| j }| j }|| |||| <q|jdks |jdkrtjt|dd } tjt|dd } |jdkr,|| j | j }|jdkr|j r|jdkrC| |_| |_n@|dusN||jkr|jdur]d|jd|_n|j|_d|j|j| |j| |_d|j|j| |j| |_|j|j|jd|_||jj ||jj |}|S) aReturns the tensor with the surrounding context. Arguments --------- x : torch.Tensor A batch of tensors. lengths : torch.Tensor A batch of tensors containing the relative length of each sentence (e.g, [0.7, 0.9, 1.0]). It is used to avoid computing stats on zero-padded steps. spk_ids : torch.Tensor containing the ids of each speaker (e.g, [0 10 6]). It is used to perform per-speaker normalization when norm_type='speaker'. epoch : int The epoch count. Returns ------- x : torch.Tensor The normalized tensor. rsentencespeakerr#.Nbatchrr)r'rr+ empty_likerangerr_compute_current_statsappenddatarrrrrrr-detachmeanstackrrrr)rr1lengthsspk_idsepoch N_batches current_means current_stdsoutsnt_id actual_size current_mean current_stdspk_id speaker_mean speaker_stdr r r!r3#s                            zInputNormalization.forwardcCs||jrtj|ddj}n tjdg|jd}|jr%tj|ddj}n tjdg|jd}t ||j t |}||fS)aMComputes mean and std Arguments --------- x : torch.Tensor A batch of tensors. Returns ------- current_mean : torch.Tensor The average of x along dimension 0 current_std : torch.Tensor The standard deviation of x along dimension 0 rrrZrrX) rr+rrrrr.rstdrkrM ones_like)rr1rrr r r!rsz)InputNormalization._compute_current_statscCsDi}|j|d<|j|d<|j|d<|j|d<|j|d<|j|d<|S)z=Fills the dictionary containing the normalization statistics.rrrrrr)rrrrrr)rstater r r!_statistics_dicts      z#InputNormalization._statistics_dictcCs|d|_t|dtr|d|_|d|_n |d|_|d|_i|_|dD] }|d||j|<q(i|_|dD] }|d||j|<q;|d|_|S)zLoads the dictionary containing the statistics. Arguments --------- state : dict A dictionary containing the normalization statistics. Returns ------- state : dict rrrrrr)r isinstancerrrrrr)rrspkr r r!_load_statistics_dicts        z(InputNormalization._load_statistics_dictcshtt||}|j||_|j||_|jD]}|j|||j|<|j|||j|<q|S)z,Puts the needed tensors in the right device.)rrr-rrrr)rr.rrr r!r-s zInputNormalization.tocCs|}t||dS)zSave statistic dictionary. Arguments --------- path : str A path where to save the dictionary. N)rr+save)rpathstatsr r r!_saves zInputNormalization._savecCs"~d}tj||d}||dS)a Load statistic dictionary. Arguments --------- path : str The path of the statistic dictionary end_of_epoch : bool Whether this is the end of an epoch. Here for compatibility, but not used. r[) map_locationN)r+loadr)rr  end_of_epochr.r r r r!_loads zInputNormalization._load)TTrNFr")F)r:r;r<r=typingrrr+Tensor__annotations__rrr3rrrr-rr rrrr?r r rr!rs0  " &  rcs^eZdZdZ     dfdd Zdd d Zd d Zd dZddZddZ ddZ Z S) GlobalNormaNA global normalization module - computes a single mean and standard deviation for the entire batch across unmasked positions and uses it to normalize the inputs to the desired mean and standard deviation. This normalization is reversible - it is possible to use the .denormalize() method to recover the original values. Arguments --------- norm_mean: float the desired normalized mean norm_std: float the desired normalized standard deviation update_steps: float the number of steps over which statistics will be collected length_dim: int the dimension used to represent the length mask_value: float the value with which to fill masked positions without a mask_value, the masked positions would be normalized, which might not be desired Example ------- >>> import torch >>> from speechbrain.processing.features import GlobalNorm >>> global_norm = GlobalNorm( ... norm_mean=0.5, ... norm_std=0.2, ... update_steps=3, ... length_dim=1 ... ) >>> x = torch.tensor([[1., 2., 3.]]) >>> x_norm = global_norm(x) >>> x_norm tensor([[0.3000, 0.5000, 0.7000]]) >>> x = torch.tensor([[5., 10., -4.]]) >>> x_norm = global_norm(x) >>> x_norm tensor([[0.6071, 0.8541, 0.1623]]) >>> x_denorm = global_norm.denormalize(x_norm) >>> x_denorm tensor([[ 5.0000, 10.0000, -4.0000]]) >>> x = torch.tensor([[100., -100., -50.]]) >>> global_norm.freeze() >>> global_norm(x) tensor([[ 5.3016, -4.5816, -2.1108]]) >>> global_norm.denormalize(x_norm) tensor([[ 5.0000, 10.0000, -4.0000]]) >>> global_norm.unfreeze() >>> global_norm(x) tensor([[ 5.3016, -4.5816, -2.1108]]) >>> global_norm.denormalize(x_norm) tensor([[ 5.0000, 10.0000, -4.0000]]) rZrXNr$c szttd}td}td}|d||d||d|||_||_||_d|_||_ ||_ d|_ dS)NrZ running_mean running_stdrrF) rrr+rr norm_meannorm_std mask_value step_count update_steps length_dimfrozen) rrrrrrrrrrr r!r\s        zGlobalNorm.__init__Fc Cs |dur tt|}|dur|j}|||}|s^|js^|jdus(|j|jkr^||}| }| }| } |j | } |j |j | || |j _|j |j| || |j_| |j _||}t|sptj||jd}||} ||| }|jd7_|S)aNormalizes the tensor provided Arguments --------- x: torch.Tensor the tensor to normalize lengths: torch.Tensor a tensor of relative lengths (padding will not count towards normalization) mask_value: float the value to use for masked positions skip_update: false whether to skip updates to the norm Returns ------- result: torch.Tensor the normalized tensor Nrr#)r+onesr(rget_maskrrr masked_selectrrrOrrrr normalize is_tensorrr. masked_fill) rr1rr skip_updatemaskx_maskedrrr new_weightmask_value_normr r r!r3ts:      zGlobalNorm.forwardcC$||j|j}||j|j}|S)a&Performs the normalization operation against the running mean and standard deviation Arguments --------- x: torch.Tensor the tensor to normalize Returns ------- result: torch.Tensor the normalized tensor )rrrrrr1r r r!r!szGlobalNorm.normalizecCsT||j}t|||}td|D] }||jkr ||}q||}|S)aFReturns the length mask for the specified tensor Arguments --------- x: torch.Tensor the tensor for which the mask will be obtained lengths: torch.Tensor the length tensor Returns ------- mask: torch.Tensor the mask tensor r#)sizerrrrr expand_asbool)rr1rmax_lenr%rr r r!rs   zGlobalNorm.get_maskcCr))zReverses the normalization process Arguments --------- x: torch.Tensor a normalized tensor Returns ------- result: torch.Tensor a denormalized version of x )rrrrr*r r r! denormalizes zGlobalNorm.denormalizecC d|_dS)z%Stops updates to the running mean/stdTNrr8r r r!rg zGlobalNorm.freezecCr0)z'Resumes updates to the running mean/stdFNr1r8r r r!unfreezer2zGlobalNorm.unfreeze)rZrXNr$rZ)NNF) r:r;r<r=rr3r!rr/rgr3r?r r rr!r#s: 8rcs0eZdZdZfddZddZddZZS) MinLevelNorma0A commonly used normalization for the decibel scale The scheme is as follows x_norm = (x - min_level_db)/-min_level_db * 2 - 1 The rationale behind the scheme is as follows: The top of the scale is assumed to be 0db. x_rel = (x - min) / (max - min) gives the relative position on the scale between the minimum and the maximum where the minimum is 0. and the maximum is 1. The subsequent rescaling (x_rel * 2 - 1) puts it on a scale from -1. to 1. with the middle of the range centered at zero. Arguments --------- min_level_db: float the minimum level Example ------- >>> norm = MinLevelNorm(min_level_db=-100.) >>> x = torch.tensor([-50., -20., -80.]) >>> x_norm = norm(x) >>> x_norm tensor([ 0.0000, 0.6000, -0.6000]) cst||_dSrI)rr min_level_db)rr5rr r!rs  zMinLevelNorm.__init__cCs4||j|j }|d9}|d}t|dd}|S)a Normalizes audio features in decibels (usually spectrograms) Arguments --------- x: torch.Tensor input features Returns ------- normalized_features: torch.Tensor the normalized features rrXrCr#)r5r+clipr*r r r!r3s  zMinLevelNorm.forwardcCs4t|dd}|dd}||j 9}||j7}|S)zReverses the min level normalization process Arguments --------- x: torch.Tensor the normalized tensor Returns ------- result: torch.Tensor the denormalized tensor rCr#rXr)r+r6r5r*r r r!r/&s    zMinLevelNorm.denormalize)r:r;r<r=rr3r/r?r r rr!r4s  r4cr) DynamicRangeCompressionaqDynamic range compression for audio signals - clipped log scale with an optional multiplier Arguments --------- multiplier: float the multiplier constant clip_val: float the minimum accepted value (values below this minimum will be clipped) Example ------- >>> drc = DynamicRangeCompression() >>> x = torch.tensor([10., 20., 0., 30.]) >>> drc(x) tensor([ 2.3026, 2.9957, -11.5129, 3.4012]) >>> drc = DynamicRangeCompression(2.) >>> x = torch.tensor([10., 20., 0., 30.]) >>> drc(x) tensor([ 2.9957, 3.6889, -10.8198, 4.0943]) r#h㈵>cst||_||_dSrI)rrrpclip_val)rrpr9rr r!rRs  z DynamicRangeCompression.__init__cCsttj||jd|jS)zPerforms the forward pass Arguments --------- x: torch.Tensor the source signal Returns ------- result: torch.Tensor the result r)r+rLrr9rpr*r r r!r3Ws zDynamicRangeCompression.forward)r#r8rr r rr!r7:sr7)r#FrJ) r=rir+speechbrain.dataio.dataiorspeechbrain.utils.checkpointsrrrr!speechbrain.utils.filter_analysisrspeechbrain.utils.loggerrr:rqrxModuler r@rr-rrQrRrrrrrr4r7r r r r!sD$    (LHWANJ