o 3NPi2 @s>dZddlZddlZddlmZmZmZmZmZddl m Z m Z ddl Z ddlZedZdadaddZdee jeeefd eefd d Zd e jd ed eeeeffddZd#ded eeeeffddZd$deeded eefddZ d%deededed eefddZ d&deeded ed efd!d"ZdS)'an Voice Activity Detection with TRUE parallelism + Compute Monitoring. === OPTIMIZATION (v6.2) === Key improvements: - Persistent workers: Load Silero model ONCE per worker via initializer - Memory-efficient: Accept numpy array directly (no file re-read) - Adaptive worker count based on detected CPU cores - Optimized for 0.4s event detection (min_speech_ms = 200) N)ListDictTupleOptionalUnion)ProcessPoolExecutor as_completedzFastPipelineV6.VADc Cs d}t|D]|}ztjjdddddd\aaWdSty}z]||dkrHd d |}d dl}d dl}| d d }| ||WYd}~qd dl }|j d }|j |rsztjj|dd ddd\aaWWYd}~dSYtd|d|dd}~wwdS)a Worker initializer - loads Silero VAD model ONCE per worker process. === OPTIMIZATION (v6.2) === Previously: Each worker loaded model for EVERY chunk (~200MB x 32 workers = 6.4GB!) Now: Model loaded ONCE per worker at startup, reused for all chunks === EDGE CASE HANDLING (v6.7) === - Retry logic for network errors when multiple workers start - Graceful fallback to cached model - Exponential backoff on failures Benefit: ~50% VAD speedup, 80% less memory snakers4/silero-vad silero_vadFT repo_or_dirmodel force_reloadonnx trust_repoN?rz-~/.cache/torch/hub/snakers4_silero-vad_masterlocal)r rsourcerrzFailed to load VAD model after z attempts. Last error: z:. Try running the main script first to download the model.)rangetorchhubload _vad_model _vad_utils Exceptiontimerandomuniformsleepospath expanduserexists RuntimeError) max_retriesattempte wait_timerrjitterr" cache_dirr-M/home/ubuntu/.cursor/worktrees/maya3data__SSH__216.81.248.184_/zxg/src/vad.py_init_vad_workersR         r/argsreturnc Cs|\}}}}t}t\}}}}}t|}||||dd|dd|dd|dd|dd d |d } g} | D]} || d } || d } | t| t| t| | dq;| S)z Worker function for parallel VAD processing. Uses globally initialized model (loaded via _init_vad_worker). Accepts tuple for map() compatibility. vad_thresholdrvad_min_speech_msvad_min_silence_msvad_window_size_samplesivad_speech_pad_msT) thresholdmin_speech_duration_msmin_silence_duration_mswindow_size_samples speech_pad_msreturn_seconds sampling_ratestartendr@rAduration)rrr from_numpyfloatgetappend)r0 audio_chunk sample_rate config_dict chunk_startrget_speech_timestamps_ wav_tensorspeech_timestampssegmentsts global_start global_endr-r-r. _vad_workerZs2           rT waveform_nprIc sjt||}td|jdt}z tjjddddddWnty;}zt d|WYd }~nd }~wwt |j |}g}|j |j |j|j|jd }td t||D]} || | |} | |} || ||| fqZtd t|d |j dg} d } t|jtd`fddt|D}t|D]H}z,|jdd}| || d7} | tdt|dd krtd| dt|dWqty}ztd|WYd }~qd }~wwWd n1swY| jdddt| dd}tdd |D}t|}td!|d"d#t|d$|d"d%td&||d"d'|j|S)(a* Run Silero VAD on pre-loaded audio buffer with persistent workers. === OPTIMIZATION (v6.2) === - Accepts numpy array directly (no file re-read) - Uses initializer to load model ONCE per worker === EDGE CASE HANDLING (v6.7) === - Pre-download model before starting workers to avoid race conditions Args: waveform_np: Audio samples as numpy array (mono) sample_rate: Sample rate config: Pipeline configuration Returns: List of speech segments with start/end/duration u#🎤 Running Silero VAD (parallel: z workers, persistent model)...r r FTr z8 Pre-download attempt failed (will retry in workers): N)r2r3r5r6r7rz Processing z chunks (z s each)...) max_workers initializercsi|] \}}t||qSr-)submitrT).0ir0executorr-r. sz'run_vad_from_buffer..x)timeoutr z VAD progress: /z chunkszVAD chunk failed: cSs|dS)Nr@r-)xr-r-r.sz%run_vad_from_buffer..)key皙?) merge_gapcss|]}|dVqdS)rCNr-)rYsr-r-r. sz&run_vad_from_buffer..u ✅ VAD: z.1fzs | z segments | zs speechz Speedup: zx realtime | Workers: )lenloggerinfo vad_workersrrrrrwarningintvad_chunk_sizer2r3r5r6r7rrGrr/ enumeraterresultextendmaxerrorsort_merge_adjacent_vad_segmentssum)rUrIconfigtotal_durationr@r)chunk_size_samples chunk_argsrJrZ chunk_datachunk_start_time all_segments completedfuturesfuturerPmerged total_speechelapsedr-r[r.run_vad_from_buffersr       &r audio_pathcCsH|dur t|j|j|Sddl}||\}}|d}t|||S)a Run Silero VAD with TRUE parallelism and persistent workers. === OPTIMIZATION (v6.2) === - If audio_buffer provided, uses pre-loaded numpy array (no file re-read) - Uses initializer to load Silero model ONCE per worker (not per chunk!) Args: audio_path: Path to audio file (used if audio_buffer is None) config: Pipeline configuration audio_buffer: Optional AudioBuffer with pre-loaded waveform Returns: List of speech segments with start/end/duration Nr)rrUrI torchaudiorsqueezenumpy)rrx audio_bufferrwaveformsrrUr-r-r.run_vad_parallels  rrerPrfcCs|sgSg}|d}|ddD](}|d|d}||kr1|d|d<|d|d|d<q|||}q|||S)z Merge VAD segments that were split at chunk boundaries. Args: segments: Sorted list of VAD segments merge_gap: Maximum gap (seconds) to merge across rrNr@rArC)copyrG)rPrfrcurrentseggapr-r-r.rvs     rv333333? vad_segmentsry min_silencecCsg}|r|dd|kr|d|dd|dddtt|dD]!}||d}||dd}||}||krF||||dq%|rf||dd|krf||dd|||ddd|S)a Find silence regions (gaps between VAD segments). Useful for: - Finding optimal chunk boundaries - Marking non-speech regions Args: vad_segments: List of speech segments from VAD total_duration: Total audio duration min_silence: Minimum silence duration to report Returns: List of silence segments with start/end/duration rr@rBrrA)rGrri)rryrsilencesrZ gap_startgap_end gap_durationr-r-r.find_silence_boundariess2    r>@ target_timewindowc Csr|}d}tt|dD]*}||d}||dd}||}||d} t| ||kr6||kr6|}| }q |S)u Find the best silence point near target_time for chunking. Strategy: Find the largest gap within ±window of target_time. This ensures we don't cut in the middle of speech. Args: vad_segments: List of speech segments from VAD target_time: Ideal cut point window: Search window (seconds) around target Returns: Best cut point (center of largest gap, or target if no gaps) rrrAr@r)rriabs) rrrbest_cut best_gap_sizerZrrgap_size gap_centerr-r-r.find_optimal_cut_pointQs  r)N)re)r)r)__doc__rloggingtypingrrrrrconcurrent.futuresrrrnpr getLoggerrjrrr/ndarrayrndictrErTstrrrrvrrr-r-r-r.s>  $?$, _   6