o
eiI @ sf d Z ddlZddlmZmZ ddlmZmZ ddl m
Z
m Z m Z m
Z
mZ e
eZG dd deZdS ) zZ
Sequence feature extraction class for common feature extractors to preprocess sequences.
N )is_valid_audio
load_audio) BatchFeatureFeatureExtractionMixin)PaddingStrategy
TensorTypeis_torch_tensorloggingto_numpyc sf e Zd ZdZdededef fdd Z d d
eee B e e
ef B e e
ee f B ee e
ef B d e e
B e B d edB d
e dedB de dB de
e
B dB defddZde jddfd
e e
ejf eB d edB de dedB de dB de f ddZ d d
e e
ejf eB d edB dedB d
e dB fddZd ddZde
ee
B eee
B fd d Z ZS )!SequenceFeatureExtractora
This is a general feature extraction class for speech recognition.
Args:
feature_size (`int`):
The feature dimension of the extracted features.
sampling_rate (`int`):
The sampling rate at which the audio files should be digitalized expressed in hertz (Hz).
padding_value (`float`):
The value that is used to fill the padding values / vectors.
feature_size
sampling_rate
padding_valuec sD || _ || _|| _|dd| _|dd| _t jdi | d S )N padding_siderightreturn_attention_maskT )r
r r popr r super__init__)selfr
r r kwargs __class__r l/home/ubuntu/transcripts/venv/lib/python3.10/site-packages/transformers/feature_extraction_sequence_utils.pyr ) s z!SequenceFeatureExtractor.__init__TNFprocessed_featurespadding
max_length
truncationpad_to_multiple_ofr return_tensorsreturnc s t ttfr t d ttfr fddd D jd vr5tdjd dt jd }|durB|nj}t |dkrS|rQg d< S |d } t | ttfrd}
t ||
dkrt|
d7 }
t ||
dksh|
t |k r||
d } |du rt
| rd }nt | t t ttt
jfrd
}n
td | d t| d
D ] \} } t | d t t frt| | < qdd | D | < qj||d}
jd }t | t fdd D stdg }t D ]fdd D }j||||d}|| q|
tjkr!tfdd|D }tj}
i }t D ]9j | ||
||d}| D ]%\} } | |vrFg || < | j t
t
j u rV| t
j } || | q9q't||dS )a2
Pad input values / input vectors or a batch of input values / input vectors up to predefined length or to the
max sequence length in the batch.
Padding side (left/right) padding values are defined at the feature extractor level (with `self.padding_side`,
`self.padding_value`)
If the `processed_features` passed are dictionary of numpy arrays or PyTorch tensors the
result will use the same type unless you provide a different tensor type with `return_tensors`. In the case of
PyTorch tensors, you will lose the specific device of your tensors however.
Args:
processed_features ([`BatchFeature`], list of [`BatchFeature`], `dict[str, list[float]]`, `dict[str, list[list[float]]` or `list[dict[str, list[float]]]`):
Processed inputs. Can represent one input ([`BatchFeature`] or `dict[str, list[float]]`) or a batch of
input values / vectors (list of [`BatchFeature`], *dict[str, list[list[float]]]* or *list[dict[str,
list[float]]]*) so you can use this method during preprocessing as well as in a PyTorch Dataloader
collate function.
Instead of `list[float]` you can have tensors (numpy arrays or PyTorch tensors),
see the note above for the return type.
padding (`bool`, `str` or [`~utils.PaddingStrategy`], *optional*, defaults to `True`):
Select a strategy to pad the returned sequences (according to the model's padding side and padding
index) among:
- `True` or `'longest'`: Pad to the longest sequence in the batch (or no padding if only a single
sequence if provided).
- `'max_length'`: Pad to a maximum length specified with the argument `max_length` or to the maximum
acceptable input length for the model if that argument is not provided.
- `False` or `'do_not_pad'` (default): No padding (i.e., can output a batch with sequences of different
lengths).
max_length (`int`, *optional*):
Maximum length of the returned list and optionally padding length (see above).
truncation (`bool`):
Activates truncation to cut input sequences longer than `max_length` to `max_length`.
pad_to_multiple_of (`int`, *optional*):
If set will pad the sequence to a multiple of the provided value.
This is especially useful to enable the use of Tensor Cores on NVIDIA hardware with compute capability
`>= 7.5` (Volta), or on TPUs which benefit from having sequence lengths be a multiple of 128.
return_attention_mask (`bool`, *optional*):
Whether to return the attention mask. If left to the default, will return the attention mask according
to the specific feature_extractor's default.
[What are attention masks?](../glossary#attention-mask)
return_tensors (`str` or [`~utils.TensorType`], *optional*):
If set, will return tensors instead of list of python integers. Acceptable values are:
- `'pt'`: Return PyTorch `torch.Tensor` objects.
- `'np'`: Return Numpy `np.ndarray` objects.
r c s i | ] fd dD qS )c s g | ]}| qS r r ).0examplekeyr r
| z;SequenceFeatureExtractor.pad...r )r# )r r% r
{ s z0SequenceFeatureExtractor.pad..zYou should supply an instance of `transformers.BatchFeature` or list of `transformers.BatchFeature` to this method that includes z, but you provided Nattention_maskr ptnpztype of z
unknown: z6. Should be one of a python, numpy, or pytorch object.c S s g | ]}t |qS r )r r# vr r r r' r( z0SequenceFeatureExtractor.pad..)r r c 3 s | ] }t | kV qd S )N)lenr- )
batch_sizer r s z/SequenceFeatureExtractor.pad..zLSome items in the output dictionary have a different batch size than others.c s i | ] \}}|| qS r r )r# kr. )ir r r) s )r r r c 3 s" | ] }t | jd V qdS )r N)r/ model_input_names)r# input_slicer r r r1 s )r padding_strategyr r ) tensor_type)!
isinstancelisttupledictr keysr4
ValueErrorr r/ r intfloatr, ndarraytypeitemsr _get_padding_strategiesallvaluesrange _truncateappendr LONGESTmax
MAX_LENGTH_paddtypefloat64astypefloat32)r r r r r r r r! required_input
first_elementindexr& valuer7 truncated_inputsinputs inputs_slice
batch_outputsoutputsr )r0 r3 r r r pad3 s F
z SequenceFeatureExtractor.padr7 c
C sx || j d }|tjkrt|}|dur&|dur&|| dkr&|| d | }|tjko0t||k }|rCd|vrCtjt|tjd|d< |r|t| }| jdkr~|r^t |d d|f|d< | j
dkrid|fdfnd|f} tj || d| j d || j d < |S | jd
kr|rt |d |df|d< | j
dkr|dfdfn|df} tj || d| j d || j d < |S t d t
| j |S ) a
Pad inputs (on left/right and up to predefined length or max length in the batch)
Args:
processed_features (`Union[dict[str, np.ndarray], BatchFeature]`):
Dictionary of input values (`np.ndarray[float]`) / input vectors (`list[np.ndarray[float]]`) or batch
of inputs values (`list[np.ndarray[int]]`) / input vectors (`list[np.ndarray[int]]`)
max_length (`int`, *optional*):
Maximum length of the returned list and optionally padding length (see below)
padding_strategy (`PaddingStrategy`, *optional*, default to `PaddingStrategy.DO_NOT_PAD`):
PaddingStrategy to use for padding.
- PaddingStrategy.LONGEST Pad to the longest sequence in the batch
- PaddingStrategy.MAX_LENGTH: Pad to the max length (default)
- PaddingStrategy.DO_NOT_PAD: Do not pad
The feature_extractor padding sides are defined in self.padding_side:
- 'left': pads on the left of the sequences
- 'right': pads on the right of the sequences
pad_to_multiple_of (`int`, *optional*):
Integer if set will pad the sequence to a multiple of the provided value. This is especially useful to
enable the use of Tensor Core on NVIDIA hardware with compute capability `>= 7.5` (Volta), or on TPUs
which benefit from having sequence lengths be a multiple of 128.
return_attention_mask (`bool`, *optional*):
Set to False to avoid returning attention mask (default: set to model specifics)
r Nr r* )rN r )r r constant)constant_valuesleftzInvalid padding strategy:)r4 r rJ r/
DO_NOT_PADr, onesint32r r[ r
r r> str)
r r r r7 r r rR needs_to_be_padded
difference
padding_shaper r r rM s@ "
z SequenceFeatureExtractor._padc C s |s|S |r|du rt d|| jd }|dur+|dur+|| dkr+|| d | }t||k}|rQ|| jd d| || jd < d|v rQ|d d| |d< |S )a
Truncate inputs to predefined length or max length in the batch
Args:
processed_features(`Union[dict[str, np.ndarray], BatchFeature]`):
Dictionary of input values (`np.ndarray[float]`) / input vectors (`list[np.ndarray[float]]`) or batch
of inputs values (`list[np.ndarray[int]]`) / input vectors (`list[np.ndarray[int]]`)
max_length (`int`, *optional*):
maximum length of the returned list and optionally padding length (see below)
pad_to_multiple_of (`int`, *optional*) :
Integer if set will pad the sequence to a multiple of the provided value. This is especially useful to
enable the use of Tensor Core on NVIDIA hardware with compute capability `>= 7.5` (Volta), or on TPUs
which benefit from having sequence lengths be a multiple of 128.
truncation (`bool`, *optional*):
Activates truncation to cut input sequences longer than `max_length` to `max_length`.
NzKWhen setting ``truncation=True``, make sure that ``max_length`` is defined.r r r* )r> r4 r/ )r r r r r rR needs_to_be_truncatedr r r rH % s z"SequenceFeatureExtractor._truncatec C s |dur |du r t j}nt|t st |}n t|t r |}nt j}|du r3|t jkr3tdt j d|t jkrA| jdu rAtd|S )z3
Find the correct padding strategy
FTNzWhen setting ``padding=z(``, make sure that max_length is definedzAsking to pad but the feature_extractor does not have a padding value. Please select a value to use as `padding_value`. For example: `feature_extractor.padding_value = 0.0`.)r rJ r9 r_ rL r> r )r r r r7 r r r rD P s&
z0SequenceFeatureExtractor._get_padding_strategiesaudio_url_or_urlsc sL t |tr fdd|D S t |trt|S t|r |S tdt| )z
Convert a single or a list of urls into the corresponding `np.ndarray` objects.
If a single url is passed, the return value will be a single object. If a list is passed a list of objects is
returned.
c s g | ]} |qS r ) fetch_audio)r# xr6 r r r' x s z8SequenceFeatureExtractor.fetch_audio..z=only a single or a list of entries is supported but got type=)r9 r: rb r r TypeErrorrB )r rg r r6 r rh p s
z$SequenceFeatureExtractor.fetch_audio)TNFNNN)NNN)FN)__name__
__module__ __qualname____doc__r? r@ r r r: r< rb boolr r r[ r_ r, rA rM rH rD rh
__classcell__r r r r r s
.
K
+* r )rn numpyr, audio_utilsr r feature_extraction_utilsr r utilsr r r r
r
get_loggerrk loggerr r r r r s