o c۷i/ @sxddlZddlZddlZddlZddlmZddlmZddlm Z ddl m Z ddl Z ddlZddlmZmZmZmZe rDddlmZeeZer_ddlZeZedd d eDZnedd d iZermddlZd d ejDZ GdddZ!de"e j#Bde"e j#BfddZ$de j#de j#fddZ%de j#de j#de j#fddZ&ede'de'de"de"de'de j#f d d!Z(dS)"N)Enum)cache)Path) TYPE_CHECKING)assert_soundfile_installedassert_soxr_installedis_soundfile_installedis_soxr_installed)RawAudio AudioFormatcCsi|]}||qSr ).0 format_namer r J/home/ubuntu/vllm_env/lib/python3.10/site-packages/mistral_common/audio.py srnonecCsg|]}|jqSr )valuelower)r vr r r 'src@seZdZdejdededdfddZdefdd Zd'd d Z e de fd d Z e d(dededdfddZe d(dededdfddZe d(dededdfddZe d(dededdfddZd)dededefddZe d*d"d#Zd$eddfd%d&ZdS)+Audio audio_array sampling_rateformatreturnNcCs||_||_||_|dS)a Initialize an Audio instance with audio data, sampling rate, and format. Args: audio_array: The audio data as a numpy array. sampling_rate: The sampling rate of the audio in Hz. format: The format of the audio file. N)rrr _check_valid)selfrrrr r r__init__+s zAudio.__init__cCs*d|jdt|j|jdd|jjS)NzAudio - sampling_rate=z Hz, duration=z.2fz s, shape=)rlenrshaperr r r__repr__8s zAudio.__repr__cCsdt|jtjsJttj|jjdksJd|jjt|jtvs0Jd|jdtdS)Nzself.audio_array.ndim=z self.format= not in EXPECTED_FORMAT_VALUES=) isinstancernpndarraytypendimrrEXPECTED_FORMAT_VALUESr r r rr?s$zAudio._check_validcCs|jjd|j}|S)z{Calculate the duration of the audio in seconds. Returns: The duration of the audio in seconds. r)rrr)rdurationr r rr*EszAudio.durationTurlstrictc Csxzt|}|tj|j|dWStjy'}ztd||d}~wty;}z td|d|d}~ww)zCreate an Audio instance from a URL. Args: url: The URL of the audio file. strict: Whether to strictly enforce mono audio. Returns: An instance of the Audio class. r,z#Failed to download audio from URL: Nz*Failed to create Audio instance from URL: z .) requestsgetraise_for_statusr from_bytescontentRequestException ValueError Exception)r+r,responseer r rfrom_urlPs zAudio.from_url audio_base64c Cs`ttd|r|dd}zt|}Wnty(}ztd|d}~wwtj ||dS)aCreate an Audio instance from a base64 encoded string. Args: audio_base64: The base64 encoded audio data. strict: Whether to strictly enforce mono audio. Defaults to True. Returns: An instance of the Audio class. z^data:audio/\w+;base64,,r"zHbase64 decoding failed. Please check the input string is a valid base64.Nr-) rrematchsplitbase64 b64decoder5r4rr1)r9r, audio_bytesr7r r r from_base64ds  zAudio.from_base64filecCstt|tr|dr|dd}t|s!td|dt|d }|}Wdn1s5wYt j ||dS)zCreate an Audio instance from an audio file. Args: file: Path to the audio file. strict: Whether to strictly enforce mono audio. Defaults to True. Returns: An instance of the Audio class. zfile://Nzfile=z does not existrbr-) rr$str startswithrexistsFileNotFoundErroropenreadrr1)rBr,fr@r r r from_file{s     zAudio.from_filer@c Cst|)}t|}|jdd}|j}|j}Wdn1s"wYWdn1s1wYt|}|j }|j dkrT|rNt d|j |j dd}t |||dS)zCreate an Audio instance from bytes. Args: audio_bytes: The audio data as bytes. strict: Whether to strictly enforce mono audio. Defaults to True. Returns: An instance of the Audio class. float32)dtypeNr"zaudio_array.ndim=axis)rrr)ioBytesIOsf SoundFilerJ sampleraterr rrr(r4meanr) r@r, audio_filerKrr audio_format format_enumrr r rr1s     zAudio.from_bytesFprefixcCst|tvsJd|dtt$}tj||j|j|d| dt |  d}Wdn1s b64encoderJdecoder)rrrZrW base64_strr r r to_base64s   zAudio.to_base64audior cCsDt|jtr t|jSt|jtrt|jStdt|j)zCreate an Audio instance from a RawAudio object. Args: audio: The RawAudio object containing audio data. Returns: An instance of the Audio class. zUnsupported audio data type: ) r$databytesrr1rErAr4r')rbr r rfrom_raw_audios   zAudio.from_raw_audionew_sampling_ratecCs6|j|krdSttj|j|j|dd|_||_dS)zResample audio data to a new sampling rate. Args: new_sampling_rate: The new sampling rate to resample the audio to. NHQ)quality)rrsoxrresampler)rrfr r rrjs  zAudio.resample)rN)T)F)rbr rr)__name__ __module__ __qualname__r%r&intrErr!rpropertyfloatr* staticmethodboolr8rArLrdr1rarerjr r r rr*s$   rfreqrcCsd}d}dtd}d|d}t|tjr8t|tjs#Jt|||k}|t||||||<|S||krG|t|||}|S)zConvert frequency from hertz to mels using the "slaney" mel-scale. Args: freq: The frequency, or multiple frequencies, in hertz (Hz). Returns: The frequencies on the mel scale. @@.@;@皙@@i@)r%logr$r&r')rs min_log_hertz min_log_mellogstepmels log_regionr r r hertz_to_mels   rr~cCsLd}d}tdd}d|d}||k}|t||||||<|S)zConvert frequency from mels to hertz using the "slaney" mel-scale. Args: mels: The frequency, or multiple frequencies, in mels. Returns: The frequencies in hertz. rtrurwrvryrx)r%rzexp)r~r{r|r}rsrr r r mel_to_hertzs  r fft_freqs filter_freqscCst|}t|dt|d}|ddddf |dd}|ddddf|dd}ttdt||}|S)a@Creates a triangular filter bank. Adapted from *torchaudio* and *librosa*. Args: fft_freqs: Discrete frequencies of the FFT bins in Hz. filter_freqs: Center frequencies of the triangular filters to create, in Hz. Returns: array of shape `(num_frequency_bins, num_mel_filters)` rr"N)r%diff expand_dimsmaximumzerosminimum)rr filter_diffslopes down_slopes up_slopes filter_bankr r r_create_triangular_filter_banks " rnum_frequency_bins num_mel_bins min_frequency max_frequencyrc Cs|dkr td|d||krtd|d|t|}t|}t|||d}t|}td|d|} t| |} d|d|d|d|} | t| d9} | jdd d krjtd |d |d | S)aVCreate a Mel filter bank matrix for converting frequency bins to the Mel scale. This function generates a filter bank matrix that can be used to transform a spectrum represented in frequency bins to the Mel scale. The Mel scale is a perceptual scale of pitches judged by listeners to be equal in distance from one another. Args: num_frequency_bins: The number of frequency bins in the input spectrum. num_mel_bins: The number of desired Mel bins in the output. min_frequency: The minimum frequency (in Hz) to consider. max_frequency: The maximum frequency (in Hz) to consider. sampling_rate: The sampling rate of the audio signal. Returns: A filter bank matrix of shape (num_mel_bins, num_frequency_bins) that can be used to project frequency bin energies onto Mel bins. rzRequire num_frequency_bins: z >= 2zRequire min_frequency: z <= max_frequency: rg@NrOgzNAt least one mel filter has all zero values. The value for `num_mel_filters` (z?) may be set too high. Or, the value for `num_frequency_bins` (z) may be set too low.) r4rr%linspacerrrmaxany) rrrrrmel_minmel_max mel_freqsrr mel_filtersenormr r rmel_filter_bank&s*  r))r>rQloggingr;enumr functoolsrpathlibrtypingrnumpyr%r.mistral_common.importsrrrr &mistral_common.protocol.instruct.chunkr getLoggerrklogger soundfilerSavailable_formatsr ri __members__valuesr)rrpr&rrrrnrr r r rsR      >