o q^i3 @sdZddlZddlZeedsdde_ddlZddlZddlZddlZz'ddl Z ddl Z ee drJe j Z e e ZdejvrJdddd d Zee _ Wn eyTYnwejrjd ejj_d ejjj_d ejj_ejZd d Zee_zddlmmZej Z!dddZ"e"e_ Wn e#yYnwzddl$m%m&Z&e&j'Z(dddZ)e)e&_'Wn e#e*fyYnwe+dZ,GdddZ-e-Z.dS)a Model management - Hot inference with adaptive loading. Key improvements: - community-1 FIRST (better per pyannote docs), 3.1 as fallback - Lazy loading with compute-aware batching - Better GPU memory management === v7.0 OPTIMIZATION: cuDNN Benchmark + TF32 === Enables CUDA optimizations for faster inference: - cudnn.benchmark=True: Auto-tune convolution algorithms - TF32 precision: 3x faster matmul on Ampere+ GPUs Expected gain: 5-10% on diarization (conv-heavy model) Nlist_audio_backendscCsddgS)N soundfilesoxrrrL/home/ubuntu/.cursor/worktrees/maya3__SSH__216.81.248.184_/nmo/src/models.pysrhf_hub_downloaduse_auth_token)r tokencOs&|dur |dur |}t|d|i|S)Nr )_orig_hf_hub_download)r r argskwargsrrr_patched_hf_hub_download)srTcOsd|d<t|i|S)NF weights_only_original_torch_load)r r rrr_patched_torch_load?srcKt||ddSNF) map_locationrr) path_or_urlrr rrr_patched_pl_loadHrcCrrr)rrrrr_patched_fabric_loadQrrzFastPipelineV6.ModelscsleZdZdZdZfddZddZdejfdd Z d d Z dd dZ de fddZ ddefddZZS) ModelManagera Singleton model manager - loads all models once, keeps them hot. Model priority (per user request): 1. community-1 (better performance per pyannote docs) 2. speaker-diarization-3.1 (fallback) Key optimization: Load once, inference many times. Ncs&|jdurt||_d|j_|jS)NF) _instancesuper__new__ _initialized)cls __class__rrrfs zModelManager.__new__cCsP|jrdSd|_d|_d|_d|_d|_d|_d|_d|_d|_d|_ d|_ dS)NTF) rdevice silero_vad silero_utilsdiarization_pipelinesegmentation_modelembedding_model panns_model_loaded _panns_loaded _model_nameselfrrr__init__ls zModelManager.__init__returncCs*|jdurttjrdnd|_|jS)z#Get CUDA device or fallback to CPU.Ncudacpu)r"torchr0 is_availabler,rrr get_device{s zModelManager.get_devicec s|jrtd|jddSt|ddptjdd}|s"td||_ tdtd tdt }| }td |t j rbt j d jt j d }td |d ddtdt jjdddddd\}}||_||_tdd dlm}d dlm} d dl} td| jt| j fdd} | | _ddg} | D]B\} }z!td|d|j!| |j d|_"||_td |d!Wnt#y}zt$d"|d#|WYd}~qd}~ww|j"durtd$|j"%|td%d|_&td&d d'l'm(}|j)d(d)d*t*|id+|_+td,d-|_t |}t j rSt j ,d }t j d }td.|dd/|dd0td1|d2d3tddS)4z Load all models once into GPU memory. Order: VAD -> Diarization (community-1 first) -> Segmentation -> Embedding u✅ Models already loaded: z (hot inference)Nhf_tokenHF_TOKENzHF_TOKEN is required to load pyannote models. - Set env var HF_TOKEN (do not hard-code it) - Ensure you've accepted the model terms on Hugging Face: - pyannote/speaker-diarization-community-1 - pyannote/segmentation-3.0 (if needed) zF======================================================================u 🚀 LOADING ALL MODELS INTO GPUzDevice: rzGPU Memory before: geAz.2fzGB freezLoading Silero VAD...zsnakers4/silero-vadr#F) repo_or_dirmodel force_reloadonnxverbosezSilero VAD loaded)Pipeline)SpeakerDiarizationzLoading PyAnnote Diarization...csXfdd|D}t|t|}|r!td||g|Ri|S)Ncsi|] \}}|vr||qSrr).0kv) _valid_paramsrr szCModelManager.load_all.._patched_sd_init..z!Filtered out unsupported params: )itemssetkeysloggerdebug)r-r r filtered_kwargsremoved_original_sd_initrBrr_patched_sd_inits z/ModelManager.load_all.._patched_sd_init)z(pyannote/speaker-diarization-community-1z community-1)z pyannote/speaker-diarization-3.1zspeaker-diarization-3.1zTrying z...)r u ✅ Loaded z (preferred model)zFailed to load z: z'Failed to load any diarization pipelineu&✅ PyAnnote Diarization loaded on GPUzLoading ECAPA-TDNN...)EncoderClassifierz!speechbrain/spkrec-ecapa-voxcelebz/ephemeral/models/ecapar")sourcesavedirrun_optsu✅ ECAPA-TDNN loadedTz GPU Memory: zGB allocated, z GB reservedu✅ All models loaded in .1fzs (STAYING HOT))-r)rGinfor+getattrosenvironget RuntimeErrorr5timer4r2r0r3get_device_properties total_memorymemory_reservedrHhubloadr#r$pyannote.audior=pyannote.audio.pipelinesr>inspectr.rE signature parametersrFfrom_pretrainedr% Exceptionwarningtor&speechbrain.inference.speakerrN from_hparamsstrr'memory_allocated)r-configr5startr"freer9utilsr=r>rarM models_to_try model_name short_nameerNelapsed allocatedreservedrrKrload_alls                     zModelManager.load_allFcCs@tjr|rttjtjdStjdSdS)z Clear GPU cache between operations. Args: aggressive: If True, also empty reserved memory and run gc N)r2r0r3gccollect empty_cachereset_peak_memory_stats)r- aggressiverrr clear_caches  zModelManager.clear_cachecCs4tjrtjdj}tjd}||dSdS)zGet current free VRAM in GB.ri@g)r2r0r3rZr[r\)r-totalrvrrrget_vram_free_gbs   zModelManager.get_vram_free_gbr0r"c Cs0|jr|jdurtd|jStdt}zTddlm}|dkr0tj s0d}t dt j d }|rNt j|rNtd ||||d |_n||d |_d |_t|}td|d|dd|jWSty}z tdtd|d}~wty}z td|d}~ww)a Load PANNs CNN14 model for music detection (lazy loading). === v7.4: Pre-baked model support === If PANNS_MODEL_PATH env var is set, loads from that path (Docker pre-baked). Otherwise downloads from default location. Args: device: 'cuda' or 'cpu' Returns: Loaded AudioTagging model Nu.✅ PANNs CNN14 already loaded (hot inference)z*Loading PANNs CNN14 for music detection...r) AudioTaggingr0r1z'CUDA not available, using CPU for PANNsPANNS_MODEL_PATHz Using pre-baked model: )checkpoint_pathr")r"Tu✅ PANNs CNN14 loaded on z in rRsu9❌ PANNs not installed. Run: pip install panns-inferencez_panns-inference package required for music detection. Install with: pip install panns-inferenceu❌ Failed to load PANNs: )r*r(rGrSrYpanns_inferencerr2r0r3rfrUrVrWpathexists ImportErrorerrorre)r-r"rmrrrtrsrrr load_pannss<        zModelManager.load_panns)F)r0)__name__ __module__ __qualname____doc__rrr.r2r"r4rwr}floatrrjr __classcell__rrr rrZs  ~r)N)/rrU torchaudiohasattrrrYloggingrxr2rahuggingface_hubrr rb_hf_sigrcrrer0r3backendscudnn benchmarkmatmul allow_tf32r^rrpytorch_lightning.core.savingcoresaving pl_savingpl_load _orig_pl_loadrr#lightning_fabric.utilities.cloud_io utilitiescloud_io_load_orig_fabric_loadrAttributeError getLoggerrGrMODELSrrrrsb                s