o }oi6@sddlmZmZddlmZddlZddlZddlZddl m Z ddl m Z ddl mZGdddeZGd d d eZGd d d eZ ddejdededeeeffddZdedededededeeeff ddZdS))ABCabstractmethod)TupleN)EncDecClassificationModel)normalize_volume)loggingc @s<eZdZdZedejdedede ejeeffddZ dS) AudioTrimmerz3Interface for silence trimming implementations audio sample_rateaudio_idreturncCst)aTrim starting and trailing silence from the input audio. Args: audio: Numpy array containing audio samples. Float [-1.0, 1.0] format. sample_rate: Sample rate of input audio. audio_id: String identifier (eg. file name) used for logging. Returns numpy array with trimmed audio, and integer sample indices representing the start and end of speech within the original audio array. )NotImplementedError)selfr r r rk/home/ubuntu/.local/lib/python3.10/site-packages/nemo/collections/tts/parts/preprocessing/audio_trimming.py trim_audios zAudioTrimmer.trim_audioN) __name__ __module__ __qualname____doc__rnparrayintstrrrrrrrrs.rc@speZdZ       dded ed ed ed ed ededdfddZddejdede de ejeeffddZ dS)EnergyAudioTrimmer2?皙?T db_threshold ref_amplitudespeech_frame_thresholdtrim_win_lengthtrim_hop_length pad_seconds volume_normr NcCsj|dksJ|dks J|dksJ|dksJ|dksJ||_||_||_||_||_||_||_dS)aEnergy/power based silence trimming using Librosa backend. Args: db_threshold: Audio frames at least db_threshold decibels below ref_amplitude will be considered silence. ref_amplitude: Amplitude threshold for classifying speech versus silence. speech_frame_threshold: Start and end of speech will be detected where there are at least speech_frame_threshold consecutive audio frames classified as speech. Setting this value higher is more robust to false-positives (silence detected as speech), but setting it too high may result in very short speech segments being cut out from the audio. trim_win_length: Length of audio frames to use when doing speech detection. This does not need to match the win_length used any other part of the code or model. trim_hop_length: Stride of audio frames to use when doing speech detection. This does not need to match the hop_length used any other part of the code or model. pad_seconds: Audio duration in seconds to keep before and after each speech segment. Set this to at least 0.1 to avoid cutting off any speech audio, with larger values being safer but increasing the average silence duration left afterwards. volume_norm: Whether to normalize the volume of audio before doing speech detection. rN)r!r"r#r$r%r&r')rr!r"r#r$r%r&r'rrr__init__.s      zEnergyAudioTrimmer.__init__r r r c Cs|jr t|dd}tjj||j|j|j|jd}t ||j |d\}}|s.|s.t gddfStj j||jd}tj j||jd}t|||jd||jd\}}|||} | ||fS)Nrr volume_level)ref frame_length hop_lengthtop_db is_speechr#r rr. start_sample end_sample max_sampler r&)r'rlibrosaeffects_signal_to_frame_nonsilentr"r$r%r!"get_start_and_end_of_speech_framesr#rrcoreframes_to_samplespad_sample_indicesshaper&) rr r r speech_frames start_frame end_framer4r5 trimmed_audiorrrrXs2    zEnergyAudioTrimmer.trim_audio)rrrrrr Tr)) rrrrfloatboolr(rrrrrrrrrr-s4 .*rc@seZdZ         d#d ed ed ed ededededededdfddZdej dej fddZ dededede eeffddZ d$dej ded ede ej eeffd!d"Z dS)%VadAudioTrimmervad_multilingual_marblenet>?cpurr T model_namevad_sample_rate vad_thresholddevicer#r$r%r&r'r Nc Cs|dksJ|dks J|dksJ|dksJ|dksJ||_tj|d|j|_||_||_||_||_ ||_ |j d|_ ||_ | |_ dS)a!Voice activity detection (VAD) based silence trimming. Args: model_name: NeMo VAD model to load. Valid configurations can be found with EncDecClassificationModel.list_available_models() vad_sample_rate: Sample rate used for pretrained VAD model. vad_threshold: Softmax probability [0, 1] of VAD output, above which audio frames will be classified as speech. device: Device "cpu" or "cuda" to use for running the VAD model. trim_win_length: Length of audio frames to use when doing speech detection. This does not need to match the win_length used any other part of the code or model. trim_hop_length: Stride of audio frames to use when doing speech detection. This does not need to match the hop_length used any other part of the code or model. pad_seconds: Audio duration in seconds to keep before and after each speech segment. Set this to at least 0.1 to avoid cutting off any speech audio, with larger values being safer but increasing the average silence duration left afterwards. volume_norm: Whether to normalize the volume of audio before doing speech detection. r)rMN)rPrfrom_pretrainedevalto vad_modelrNrOr#r$r% trim_shiftr&r') rrMrNrOrPr#r$r%r&r'rrrr(|s       zVadAudioTrimmer.__init__r c Cs|jd|jkr tgStjj||j|jd}|jd|jg}t j |t j |j d}t j |t j |j d}|j||d}t j|dd}|}|dddf}||jk} | S)Nr)r-r.)dtyperP) input_signalinput_signal_length)dimr)r>r$rrr7utilframer% transposetorchtensorfloat32rPint32rUsoftmaxdetachrJnumpyrO) rr audio_framesaudio_frame_lengths audio_signalaudio_signal_len log_probsprobs speech_probsr?rrr_detect_speechs    zVadAudioTrimmer._detect_speechr4r5r cCs*||j}t||}t||}||fS)N)rNr)rr4r5r sample_rate_ratiorrr_scale_sample_indicess   z%VadAudioTrimmer._scale_sample_indicesr)r c Cs||jkr|}n tj|||jd}|jrt|dd}|j|d}t||j|d\}}|s6|s6t gddfS|dkr=d}ntj j ||j d}||j 7}||jdkrX|jd} ntj j ||j d} | |j 7} ||jkru|j|| |d\}} t|| |jd||jd \}} ||| } | || fS) N)orig_sr target_srrr*)r r0rr2)r4r5r r3)rNr7resampler'rrmr:r#rrr;r<r%rVr>ror=r&) rr r r vad_audior?r@rAr4r5rBrrrrs@           zVadAudioTrimmer.trim_audio) rGrHrIrJrrKrLr TrC)rrrrrrDrEr(rrrmrrorrrrrrF{sD   2".rFr)r1r#r r c Cs|jd}d}td||dD]}||}t|||r"|}nqd}t||ddD]}||}t|||r?|}nq-|dusH|durStd|ddS||fS)aFinds the speech frames corresponding to the start and end of speech for an utterance. Args: is_speech: [num_frames] boolean array with true entries labeling speech frames. speech_frame_threshold: The number of consecutive speech frames required to classify the speech boundaries. audio_id: String identifier (eg. file name) used for logging. Returns integers representing the frame indices of the start (inclusive) and end (exclusive) of speech. rNrrZz+Could not find start or end of speech for '')rr)r>rangeallrwarning) r1r#r num_framesr@ihigh_irAlow_irrrr:s& r:r4r5r6r r&cCs8t||}||}||}td|}t||}||fS)aShift the input sample indices by pad_seconds in front and back within [0, max_sample] Args: start_sample: Start sample index end_sample: End sample index max_sample: Maximum sample index sample_rate: Sample rate of audio pad_seconds: Amount to pad/shift the indices by. Returns the sample indices after padding by the input amount. r)rmaxmin)r4r5r6r r& pad_samplesrrrr=&s   r=rC)abcrrtypingrr7rerr_nemo.collections.asr.modelsr2nemo.collections.tts.parts.utils.tts_dataset_utilsr nemo.utilsrrrrFrrrr:rDr=rrrrsF    N  %