o 3NPiw* @sdZddlZddlZddlZddlZejr'dejj_ dejjj _ dejj_ ej Z ddZee_ zddlmmZejZd ddZee_Wn eyPYnwzddlmmZejZd dd Zee_Wn eefyrYnwed ZGd d d Z e Z!dS)a Model management - Hot inference with adaptive loading. Key improvements: - community-1 FIRST (better per pyannote docs), 3.1 as fallback - Lazy loading with compute-aware batching - Better GPU memory management === v7.0 OPTIMIZATION: cuDNN Benchmark + TF32 === Enables CUDA optimizations for faster inference: - cudnn.benchmark=True: Auto-tune convolution algorithms - TF32 precision: 3x faster matmul on Ampere+ GPUs Expected gain: 5-10% on diarization (conv-heavy model) NTcOsd|d<t|i|S)NF weights_only_original_torch_load)argskwargsrP/home/ubuntu/.cursor/worktrees/maya3data__SSH__216.81.248.184_/zxg/src/models.py_patched_torch_load"sr cKt||ddSNF) map_locationrr) path_or_urlr rrrr_patched_pl_load+rcCr r r)r r rrr_patched_fabric_load4rrzFastPipelineV6.ModelscsleZdZdZdZfddZddZdejfdd Z d d Z dd dZ de fddZ ddefddZZS) ModelManagera Singleton model manager - loads all models once, keeps them hot. Model priority (per user request): 1. community-1 (better performance per pyannote docs) 2. speaker-diarization-3.1 (fallback) Key optimization: Load once, inference many times. Ncs&|jdurt||_d|j_|jS)NF) _instancesuper__new__ _initialized)cls __class__rrrIs zModelManager.__new__cCsP|jrdSd|_d|_d|_d|_d|_d|_d|_d|_d|_d|_ d|_ dS)NTF) rdevice silero_vad silero_utilsdiarization_pipelinesegmentation_modelembedding_model panns_model_loaded _panns_loaded _model_nameselfrrr__init__Os zModelManager.__init__returncCs*|jdurttjrdnd|_|jS)z#Get CUDA device or fallback to CPU.Ncudacpu)rtorchr' is_availabler#rrr get_device^s zModelManager.get_devicec s|jrtd|jddStdtdtdt}|}td|tjrLtj dj tj d}td|d d d td tj j d dddd\}}||_||_tdddlm}ddlm}ddl} td|jt| jfdd} | |_ddg} | D]B\} } z!td| d|j| |jd|_| |_td| dWnty}ztd| d|WYd}~qd}~ww|jdurtd |j |td!d|_!td"dd#l"m#}|j$d$d%d&t%|id'|_&td(d)|_t|}tjr Diarization (community-1 first) -> Segmentation -> Embedding u✅ Models already loaded: z (hot inference)NzF======================================================================u 🚀 LOADING ALL MODELS INTO GPUzDevice: rzGPU Memory before: geAz.2fzGB freezLoading Silero VAD...zsnakers4/silero-vadrF) repo_or_dirmodel force_reloadonnxu✅ Silero VAD loaded)Pipeline)SpeakerDiarizationzLoading PyAnnote Diarization...csXfdd|D}t|t|}|r!td||g|Ri|S)Ncsi|] \}}|vr||qSrr).0kv) _valid_paramsrr szCModelManager.load_all.._patched_sd_init..z!Filtered out unsupported params: )itemssetkeysloggerdebug)r$rrfiltered_kwargsremoved_original_sd_initr5rr_patched_sd_inits z/ModelManager.load_all.._patched_sd_init)z(pyannote/speaker-diarization-community-1z community-1)z pyannote/speaker-diarization-3.1zspeaker-diarization-3.1zTrying z...)tokenu ✅ Loaded z (preferred model)zFailed to load z: z'Failed to load any diarization pipelineu&✅ PyAnnote Diarization loaded on GPUzLoading ECAPA-TDNN...)EncoderClassifierz!speechbrain/spkrec-ecapa-voxcelebz/ephemeral/models/ecapar)sourcesavedirrun_optsu✅ ECAPA-TDNN loadedTz GPU Memory: zGB allocated, z GB reservedu✅ All models loaded in .1fzs (STAYING HOT))(r r:infor"timer+r)r'r*get_device_properties total_memorymemory_reservedhubloadrrpyannote.audior0pyannote.audio.pipelinesr1inspectr%r8 signature parametersr9from_pretrainedhf_tokenr Exceptionwarning RuntimeErrortorspeechbrain.inference.speakerrB from_hparamsstrrmemory_allocated)r$configstartrfreer-utilsr0r1rPr@ models_to_try model_name short_nameerBelapsed allocatedreservedrr>rload_allds                     zModelManager.load_allFcCs@tjr|rttjtjdStjdSdS)z Clear GPU cache between operations. Args: aggressive: If True, also empty reserved memory and run gc N)r)r'r*gccollect empty_cachereset_peak_memory_stats)r$ aggressiverrr clear_caches  zModelManager.clear_cachecCs4tjrtjdj}tjd}||dSdS)zGet current free VRAM in GB.ri@g)r)r'r*rIrJrK)r$totalrgrrrget_vram_free_gbs   zModelManager.get_vram_free_gbr'rc Cs|jr|jdurtd|jStdt}z6ddlm}|dkr0tj s0d}t d||d |_d |_t|}td |d |d d|jWSt ye}z t dt d|d}~wt yy}z t d|d}~ww)a~ Load PANNs CNN14 model for music detection (lazy loading). === v6.7 FEATURE: Music Detection === PANNs (Pretrained Audio Neural Networks) trained on AudioSet. Used to detect music/instruments in audio chunks. Args: device: 'cuda' or 'cpu' Returns: Loaded AudioTagging model Nu.✅ PANNs CNN14 already loaded (hot inference)z*Loading PANNs CNN14 for music detection...r) AudioTaggingr'r(z'CUDA not available, using CPU for PANNs)rTu✅ PANNs CNN14 loaded on z in rFsu9❌ PANNs not installed. Run: pip install panns-inferencez_panns-inference package required for music detection. Install with: pip install panns-inferenceu❌ Failed to load PANNs: )r!rr:rGrHpanns_inferencerqr)r'r*rV ImportErrorerrorrU)r$rr^rqrerdrrr load_pannss4       zModelManager.load_panns)F)r')__name__ __module__ __qualname____doc__rrr%r)rr+rhrnfloatrpr[rv __classcell__rrrrr=s  pr)N)"rzrHloggingrir)r'r*backendscudnn benchmarkmatmul allow_tf32rMrr pytorch_lightning.core.savingcoresaving pl_savingpl_load _orig_pl_loadrrt#lightning_fabric.utilities.cloud_io utilitiescloud_io_load_orig_fabric_loadrAttributeError getLoggerr:rMODELSrrrrs@           _