o %ݫiI@sdZddlZddlZdZedddZddZeefd d Zed d d Zd ddZ e     d!ddZ d"ddZ d#ddZ d$ddZ dS)%a5 Functions for analyzing vocal characteristics: jitter, shimmer, HNR, and GNE. These are typically used for analysis of dysarthric voices using more traditional approaches (i.e. not deep learning). Often useful as a baseline for e.g. pathology detection. Inspired by PRAAT. Authors * Peter Plantinga, 2024 Nc Csjt|}|dddd||fjdd\}}tjjj|dd}|d|djdd\}}||}||fS)aBCompute features based on autocorrelation Arguments --------- frames: torch.Tensor The audio frames to be evaluated for autocorrelation, shape [batch, frame, sample] min_lag: int The minimum number of samples to consider for potential period length. max_lag: int The maximum number of samples to consider for potential period length. neighbors: int The number of neighbors to use for rolling median -- to avoid octave errors. Returns ------- harmonicity: torch.Tensor The highest autocorrelation score relative to the 0-lag score. Used to compute HNR best_lags: torch.Tensor The lag corresponding to the highest autocorrelation score, an estimate of period length. Example ------- >>> audio = torch.rand(1, 16000) >>> frames = audio.unfold(-1, 800, 200) >>> frames.shape torch.Size([1, 77, 800]) >>> harmonicity, best_lags = compute_autocorr_features(frames, 100, 200) >>> harmonicity.shape torch.Size([1, 77]) >>> best_lags.shape torch.Size([1, 77]) Ndim)r)pad) autocorrelatemaxtorchnn functionalrunfoldmedian) framesmin_lagmax_lag neighborsautocorrelation harmonicitylags best_lags_rY/home/ubuntu/.local/lib/python3.10/site-packages/speechbrain/processing/vocal_features.pycompute_autocorr_featuress "&rcCsP|d}tj||jdddd}t||||}t||jdd}||S)aGenerate autocorrelation scores using circular convolution. Arguments --------- frames: torch.Tensor The audio frames to be evaluated for autocorrelation, shape [batch, frame, sample] Returns ------- autocorrelation: torch.Tensor The ratio of the best candidate lag's autocorrelation score against the theoretical maximum autocorrelation score at lag 0. Normalized by the autocorrelation_score of the window. Example ------- >>> audio = torch.rand(1, 16000) >>> frames = audio.unfold(-1, 800, 200) >>> frames.shape torch.Size([1, 77, 800]) >>> autocorrelation = autocorrelate(frames) >>> autocorrelation.shape torch.Size([1, 77, 401]) rdevicer 绽|=min)sizer hann_windowrviewcompute_cross_correlationclamp)r window_sizehannr norm_scorerrrr Bs r cCst|}tj|d|jd}|ddd|j}| d}| |}|d}tj |ddd\}} | |} || |k|| ||k@|| |k|| ||k@B} d|| <gg} } t |D]0}tj |ddd\}} || |dk|| |dk@} d|| <| |d| | dqctj| dd } tj| dd } | |} t| || } | | jddd }|jdd |}| jddd }| |}|jdd |djd d }||fS) aFunction to compute periodic features: jitter, shimmer Arguments --------- frames: torch.Tensor The framed audio to use for feature computation, dims [batch, frame, sample]. best_lags: torch.Tensor The estimated period length for each frame, dims [batch, frame]. neighbors: int Number of neighbors to use in comparison. Returns ------- jitter: torch.Tensor The average absolute deviation in period over the frame. shimmer: torch.Tensor The average absolute deviation in amplitude over the frame. Example ------- >>> audio = torch.rand(1, 16000) >>> frames = audio.unfold(-1, 800, 200) >>> frames.shape torch.Size([1, 77, 800]) >>> harmonicity, best_lags = compute_autocorr_features(frames, 100, 200) >>> jitter, shimmer = compute_periodic_features(frames, best_lags) >>> jitter.shape torch.Size([1, 77]) >>> shimmer.shape torch.Size([1, 77]) rrr rT)rkeepdimrrr)rkeepdimsrr )r clonedetacharanger"rr$expandshape unsqueeze remainderr rangeappendsqueezestackminimumfloatmeanabsr&)rrr masked_frames mask_indicesperiodsperiod_indices jitter_rangepeaklag lag_indicesmaskpeaksri jitter_framesjitteravg_ampsamp_diffshimmerrrrcompute_periodic_featuresgsF"         rKrc CsN|d}tjdd||jdddd}t||d}t||d|}t||d||d|}t||d||d|}|d}||| j dd }|| d } | |j dd|} |j dd|j dd|} |d d ddd d f} tj|d| d dj dd} tj|||||| | | fddS) aCompute statistical measures on spectral frames such as flux, skew, spread, flatness. Reference page for computing values: https://www.mathworks.com/help/audio/ug/spectral-descriptors.html Arguments --------- spectrum: torch.Tensor The spectrum to use for feature computation, dims [batch, frame, freq]. eps: float A small value to avoid division by 0. Returns ------- features: torch.Tensor A [batch, frame, 8] tensor of spectral features for each frame: * centroid: The mean of the spectrum. * spread: The stdev of the spectrum. * skew: The spectral balance. * kurtosis: The spectral tailedness. * entropy: The peakiness of the spectrum. * flatness: The ratio of geometric mean to arithmetic mean. * crest: The ratio of spectral maximum to arithmetic mean. * flux: The average delta-squared between one spectral value and it's successor. Example ------- >>> audio = torch.rand(1, 16000) >>> window_size = 800 >>> frames = audio.unfold(-1, window_size, 200) >>> frames.shape torch.Size([1, 77, 800]) >>> hann = torch.hann_window(window_size).view(1, 1, -1) >>> windowed_frames = frames * hann >>> spectrum = torch.abs(torch.fft.rfft(windowed_frames)) >>> spectral_features = compute_spectral_features(spectrum) >>> spectral_features.shape torch.Size([1, 77, 8]) rrr rrrrN)rprepend)r"r linspacerr$ spec_normr1sqrtr5logr9expamaxsumdiffpowr6)spectrumepsnfreqfreqscentroidspreadskewkurtentropygeomeanflatnesscrestrfluxrrrcompute_spectral_featuress + "rdcCs ||jdd|jdd|S)z*Normalize the given value by the spectrum.rr)rT)valuerWrXrrrrOs rO>,Q?{Gz?cs|dks Jdd}tj||}t|}t|}tj||jdddd} |j d||d| } t | dd ddd} } t | | |fd d Dfd d D} tj | ddj ddS)ajAn algorithm for GNE computation from the original paper: "Glottal-to-Noise Excitation Ratio - a New Measure for Describing Pathological Voices" by D. Michaelis, T. Oramss, and H. W. Strube. This algorithm divides the signal into frequency bands, and compares the correlation between the bands. High correlation indicates a relatively low amount of noise in the signal, whereas lower correlation could be a sign of pathology in the vocal signal. Godino-Llorente et al. in "The Effectiveness of the Glottal to Noise Excitation Ratio for the Screening of Voice Disorders." explore the goodness of the bandwidth and frequency shift parameters, the defaults here are the ones recommended in that work. Arguments --------- audio : torch.Tensor The batched audio signal to use for GNE computation, [batch, sample] sample_rate : float The sample rate of the input audio. bandwidth : float The width of the frequency bands used for computing correlation. fshift : float The shift between frequency bands used for computing correlation. frame_len : float Length of each analysis frame, in seconds. hop_len : float Length of time between the start of each analysis frame, in seconds. Returns ------- gne : torch.Tensor The glottal-to-noise-excitation ratio for each frame of the audio signal. Example ------- >>> sample_rate = 16000 >>> audio = torch.rand(1, sample_rate) # 1s of audio >>> gne = compute_gne(audio, sample_rate=sample_rate) >>> gne.shape torch.Size([1, 98]) rz3Expected audio to be 2-dimensional, [batch, sample]'rr r) dimensionr"step ) lpc_ordercsi|] }|t|qSr)compute_hilbert_envelopes).0 center_freq) bandwidthexcitation_frames sample_raterr Qs zcompute_gne..cs<g|]}D]}||dkrt||ddqqS)rrLwidth)r%)rqfreq_ifreq_j)rs center_freqs envelopesrr Yszcompute_gne..r)rrL)r torchaudiorresampleintr r#rr$rinverse_filterr3r6rS)audiorursfshift frame_lenhop_lenold_sample_rate frame_sizehop_sizewindowrmin_freqmax_freq correlationsr)rsr{r|rtrur compute_gnes&6     rrnc Cst|||d}|j\}}}|||d}|||d}d|dd|f<|ddddfd|djdd}|dd|ddf}tj||} tjj j | ddd } t | } d| ddd f<t j j || | d d } | ||dS) aPerform inverse filtering on frames to estimate glottal pulse train. Uses autocorrelation method and Linear Predictive Coding (LPC). Algorithm from https://course.ece.cmu.edu/~ece792/handouts/RS_Chap_LPC.pdf Arguments --------- frames : torch.Tensor The audio frames to filter using inverse filter. lpc_order : int The size of the filter to compute and use on the frames. Returns ------- filtered_frames : torch.Tensor The frames after the inverse filter is applied Example ------- >>> audio = torch.rand(1, 10000) >>> frames = audio.unfold(-1, 300, 100) >>> frames.shape torch.Size([1, 98, 300]) >>> filtered_frames = inverse_filter(frames) >>> filtered_frames.shape torch.Size([1, 98, 300]) rwrg?Nr )r )dims)r r)rerF)r&)r%r0r$rflipr linalgsolver rr zeros_liker~lfilter) rrorbatch frame_countrreshaped_framesRrlpc lpc_coeffsa_coeffsinverse_filteredrrrrds & rrkc Cs||d}||d}tj|}tj|dd|}tj|tjd}||k||k@} tj| |jd} | |dddd| f<tj ||} | S)aCompute the hilbert envelope of the signal in a specific frequency band using FFT. Arguments --------- frames : torch.Tensor A set of frames from a signal for which to compute envelopes. center_freq : float The target frequency for the envelope. bandwidth : float The size of the band to use for the envelope. sample_rate : float The number of samples per second in the frame signals. Returns ------- envelopes : torch.Tensor The computed envelopes. Example ------- >>> audio = torch.rand(1, 10000) >>> frames = audio.unfold(-1, 300, 100) >>> frames.shape torch.Size([1, 98, 300]) >>> envelope = compute_hilbert_envelopes(frames, 1000) >>> envelope.shape torch.Size([1, 98, 300]) rrr )dtyperN) r fftfftfreqr"rr8r#rTrifftr:) rrrrsrulow_freq high_freqspectrarZrC window_binsranalytic_signalrrrrps !  rpc Cs|j\}}}|durd|dfn||f}tjjj||dd}||}|d|d} ||dd} tjjj| | |d} | ||d} t|djdd |djdd } | | dj d d } | S) aComputes the correlation between two sets of frames. Arguments --------- frames_a : torch.Tensor frames_b : torch.Tensor The two sets of frames to compare using cross-correlation, shape [batch, frame, sample] width : int, default is None The number of samples before and after 0 lag. A width of 3 returns 7 results. If None, 0 lag is put at the front, and the result is 1/2 the original length + 1, a nice default for autocorrelation as there are no repeated values. Returns ------- The cross-correlation between frames_a and frames_b. Example ------- >>> frames = torch.arange(10).view(1, 1, -1).float() >>> compute_cross_correlation(frames, frames, width=3) tensor([[[0.6316, 0.7193, 0.8421, 1.0000, 0.8421, 0.7193, 0.6316]]]) >>> compute_cross_correlation(frames, frames) tensor([[[1.0000, 0.8421, 0.7193, 0.6316, 0.5789, 0.5614]]]) Nrrcircular)moder r)inputweightgroupsrrr ) r0r r rrr$conv1drPrTr1r&) frames_aframes_brx batch_sizerrrpadded_frames_a merged_size reshaped_a reshaped_bcross_correlationnormrrrr%s &r%)r)r)rfrgrhrirj)rn)rgrk)N)__doc__r r~PERIODIC_NEIGHBORSno_gradrr rKrdrOrrrpr%rrrrs.  0%R  H [@ 5