o
i[3 @ s d dl mZmZ d dlZd dlZddlmZ ddlm Z ddl
m Z m Z m
Z
ddlmZ e r3d dlZdZd Ze
eZed
d G d d
d
eZd
gZdS ) )OptionalUnionN )SequenceFeatureExtractor) BatchFeature)
TensorTypeis_librosa_availablelogging)requiresgh㈵>g p>)torchlibrosa)backendsc s e Zd ZdZddgZ
d! fd d Zd"ddZ
d#deej e
e e
ej e
e
e f de de
e de
eeef de
e de
e de
e de
e de
e d e
e d e
e d efd d Z ZS )$ParakeetFeatureExtractora
Constructs a Parakeet feature extractor.
This feature extractor inherits from [`~feature_extraction_sequence_utils.SequenceFeatureExtractor`] which contains
most of the main methods. Users should refer to this superclass for more information regarding those methods.
This class extracts mel-filter bank features from raw speech using a custom numpy implementation of the `Short Time
Fourier Transform` which should match pytorch's `torch.stft` equivalent.
Args:
feature_size (`int`, *optional*, defaults to 80):
The feature dimension of the extracted features.
sampling_rate (`int`, *optional*, defaults to 16000):
The sampling rate at which the audio files should be digitalized expressed in hertz (Hz).
hop_length (`int`, *optional*, defaults to 160):
Length of the overlapping windows for the STFT used to obtain the Mel Frequency coefficients.
n_fft (`int`, *optional*, defaults to 512):
Size of the Fourier transform.
win_length (`int`, *optional*, defaults to 400):
The window length for the STFT computation.
preemphasis (`float`, *optional*, defaults to 0.97):
A preemphasis filter coefficient. 0.0 means no preemphasis filter.
padding_value (`float`, *optional*, defaults to 0.0):
Padding value used to pad the audio. Should correspond to silences.
input_featuresattention_maskP >
ףp=
? c
sf t jd|||d| || _|| _|| _|| _tjj|||d|d dd} t
| t j | _
d S )N) feature_size
sampling_rate
padding_valuer slaney)srn_fftn_melsfminfmaxnorm )super__init__
hop_lengthr
win_length preemphasisr filtersmelr
from_numpytofloat32 mel_filters)
selfr r r& r r' r( r kwargsr. __class__r# l/home/ubuntu/.local/lib/python3.10/site-packages/transformers/models/parakeet/feature_extraction_parakeet.pyr% C s
z!ParakeetFeatureExtractor.__init__cpuc C s t j| jd|d}t j|| j| j| j|ddd}t |}t |d d}|d}| j
|}|| }t |t
}|ddd }|S )
NF)periodicdeviceTconstant)r& r' windowreturn_complexpad_moder r )r hann_windowr' stftr r& view_as_realsqrtpowsumr. r, logLOG_ZERO_GUARD_VALUEpermute)r/ waveformr6 r8 r>
magnitudesr. mel_specr# r# r3 _torch_extract_fbank_featurese s$
z6ParakeetFeatureExtractor._torch_extract_fbank_featuresFNlongest
raw_speech
truncationpad_to_multiple_ofreturn_tensorsreturn_attention_maskpadding
max_lengthr do_normalizer6 return_token_timestampsreturnc
K s |dur || j kr td| jj d| j d| j d| d n td| jj d t|tjr6t
|}nt|t t frLt|d tjrLd
d |D }t|t j
oXt|jd k}
|
rrt|jd
krrtd| jj d |d}t|t t f}|r|D ]}t|jd krtd| jj d |d}q}|
s|rdd |D }n
|dddf t jg}dd |D }t||d}| j|||||dd}|jd}| jdurt j|jd |jdd |jd k }t j |dddd f |ddd df | j|ddddf gd d}| | d}| ||
}t |j| j d
d
| j | j!}t j|jd |
ddddf |dddf k }|d}|| }|j"d d|d }|d }|| d
| j"d d|d d }t #|d }|| |t$ }||9 }t||d|dS )a
Main method to featurize and prepare for the model one or several sequence(s). Implementation uses PyTorch for
the STFT computation if available, otherwise a slower NumPy based one.
Args:
raw_speech (`np.ndarray`, `list[float]`, `list[np.ndarray]`, `list[list[float]]`):
The sequence or batch of sequences to be padded. Each sequence can be a numpy array, a list of float
values, a list of numpy arrays or a list of list of float values. Must be mono channel audio, not
stereo, i.e. single float per timestep.
truncation (`bool`, *optional*, default to `True`):
Activates truncation to cut input sequences longer than *max_length* to *max_length*.
pad_to_multiple_of (`int`, *optional*, defaults to None):
If set will pad the sequence to a multiple of the provided value.
This is especially useful to enable the use of Tensor Cores on NVIDIA hardware with compute capability
`>= 7.5` (Volta), or on TPUs which benefit from having sequence lengths be a multiple of 128.
return_attention_mask (`bool`, *optional*):
Whether to return the attention mask. If left to the default, will return the attention mask according
to the specific feature_extractor's default.
[What are attention masks?](../glossary#attention-mask)
For Parakeet models, `attention_mask` should always be passed for batched inference, to avoid subtle
bugs.
return_tensors (`str` or [`~utils.TensorType`], *optional*):
If set, will return tensors instead of list of python integers. Acceptable values are:
- `'tf'`: Return TensorFlow `tf.constant` objects.
- `'pt'`: Return PyTorch `torch.Tensor` objects.
- `'np'`: Return Numpy `np.ndarray` objects.
sampling_rate (`int`, *optional*):
The sampling rate at which the `raw_speech` input was sampled. It is strongly recommended to pass
`sampling_rate` at the forward call to prevent silent errors and allow automatic speech recognition
pipeline.
padding_value (`float`, *optional*, defaults to 0.0):
The value that is used to fill the padding values / vectors.
do_normalize (`bool`, *optional*, defaults to `False`):
Whether or not to zero-mean unit-variance normalize the input. Normalizing can help to significantly
improve the performance of the model.
device (`str`, *optional*, defaults to `'cpu'`):
Specifies the device for computation of the log-mel spectrogram of audio signals in the
`_torch_extract_fbank_features` method. (e.g., "cpu", "cuda")
return_token_timestamps (`bool`, *optional*, defaults to `None`):
Deprecated. Use `return_attention_mask` instead from which the number of frames can be inferred.
Whether or not to return the number of frames of the input raw_speech.
These num_frames can be used by the model to compute word level timestamps.
Nz3The model corresponding to this feature extractor: z& was trained using a sampling rate of zI. Please make sure that the provided `raw_speech` input was sampled with z and not .zDIt is strongly recommended to pass the `sampling_rate` argument to `zN()`. Failing to do so can result in silent errors that might be hard to debug.r c S s g | ]}t |qS r# )r tensor.0speechr# r# r3
s z5ParakeetFeatureExtractor.__call__..r< r z2Only mono-channel audio is supported for input to z;. We will take the mean of the channels to convert to mono.r; c S s$ g | ]}|d d d f tjqS )N)r, r r- rW r# r# r3 rZ s $ c S s g | ]}t |qS r# )lenrW r# r# r3 rZ s )r
audio_lengthspt)rP rQ rL rM rN )r6 )dimr )r r )data tensor_type)%r
ValueErrorr2 __name__loggerwarning
isinstancenpndarrayr rV listtupleTensorr[ shapemeanr, r- r padr squeezer( aranger6 unsqueezer\ cat masked_fillrI floor_divider r& rB r@ EPSILON) r/ rK rL rM rN rO rP rQ r rR r6 rS r0 is_batched_torchis_batched_sequencerY r\ batched_speech
padded_inputsr timemaskfeatures_lengthsr maskinput_features_maskedrl variancestdr# r# r3 __call__ s D
B 0
&z!ParakeetFeatureExtractor.__call__)r r r r r r r )r4 )
FNNNrJ NNNr4 N)rb
__module__ __qualname____doc__model_input_namesr% rI r rf rg rh floatboolr intstrr r r
__classcell__r# r# r1 r3 r % s^
" "
r )typingr r numpyrf r !feature_extraction_sequence_utilsr feature_extraction_utilsr utilsr r r utils.import_utilsr
r rt rD
get_loggerrb rc r __all__r# r# r# r3 s
z