o 9wi 6@sTdZddlZddlmZddlmZddlmZddlm Z m Z m Z m Z ddl ZddlmZmZddlmZdd lmZmZdd lmZmZmZmZdd lmZdd lmZdd l m!Z!ddl"m#Z#ddl$m%Z%ddl&m'Z'ddl(m)Z)ddl*m+Z+m,Z,m-Z-m.Z.m/Z/m0Z0ddl1m2Z3ddl4m5Z5Gddde)Z6Gddde)Z2Gddde)Z7dS)z"Voice activity detection pipelinesN)deepcopy)partial) MethodType)CallableOptionalTextUnion) AnnotationSlidingWindowFeature)SpeakerDiarizationProtocol)DetectionErrorRate DetectionPrecisionRecallFMeasure) CategoricalInteger LogUniformUniform)Trainer)SGD)BaseWaveformTransform) Inference)GraduallyUnfreeze) AudioFile)Pipeline)PipelineAugmentationPipelineInference PipelineModelget_augmentation get_inference get_model)VoiceActivityDetection)Binarizec@s&eZdZdZededefddZdS)OracleVoiceActivityDetectionz(Oracle voice activity detection pipelinefilereturncCs|d}|jdddS)a Return groundtruth voice activity detection Parameter --------- file : AudioFile Must provide a "annotation" key. Returns ------- hypothesis : `pyannote.core.Annotation` Speech regions annotationstringspeech) generatormodality) get_timelinesupport to_annotation)r"r&r,n/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/pyannote/audio/pipelines/voice_activity_detection.pyapply>sz"OracleVoiceActivityDetection.applyN)__name__ __module__ __qualname____doc__ staticmethodrr r.r,r,r,r-r!;sr!c seZdZdZ   ddededeedfffdd Zd d Z d d Z ddZ dZ dde deedefddZdeeeffddZddZZS)ra*Voice activity detection pipeline Parameters ---------- segmentation : Model, str, or dict, optional Pretrained segmentation (or voice activity detection) model. Defaults to "pyannote/segmentation". See pyannote.audio.pipelines.utils.get_model for supported format. fscore : bool, optional Optimize (precision/recall) fscore. Defaults to optimizing detection error rate. use_auth_token : str, optional When loading private huggingface.co models, set `use_auth_token` to True or to a string containing your hugginface.co authentication token that can be obtained by running `huggingface-cli login` inference_kwargs : dict, optional Keywords arguments passed to Inference. Hyper-parameters ---------------- onset, offset : float Onset/offset detection thresholds min_duration_on : float Remove speech regions shorter than that many seconds. min_duration_off : float Fill non-speech regions shorter than that many seconds. pyannote/segmentationFN segmentationfscoreuse_auth_tokenc st||_||_t||d}dd|d<t|fi||_|jjr+d|_ |_ n t dd|_ t dd|_ t dd|_ t dd|_ dS)N)r7cSstj|dddS)NTaxiskeepdims)npmax)scoresr,r,r-}sz1VoiceActivityDetection.__init__..pre_aggregation_hook?g?)super__init__r5r6rr _segmentationspecificationspowersetonsetoffsetrmin_duration_onmin_duration_off)selfr5r6r7inference_kwargsmodel __class__r,r-rDns      zVoiceActivityDetection.__init__cCs2|jdkr dddddS|jdkrddd St) Nr4g%C?gT㥛 ?g rh?gx&?rHrIrJrKzpyannote/segmentation-3.0.0rB)rJrK)r5NotImplementedErrorrLr,r,r-default_parameterss  z)VoiceActivityDetection.default_parameterscCsdgS)NSPEECHr,rSr,r,r-classesszVoiceActivityDetection.classescCst|j|j|j|jd|_dS)z2Initialize pipeline with current set of parametersrQN)r rHrIrJrK _binarizerSr,r,r- initializes  z!VoiceActivityDetection.initializezcache/segmentation/inferencer"hookr#cCs|j||d}|jr&|j|vr||j}n|j|t|ddd}|||j<n |j|t|ddd}|d|||}|d|_|dd|DS)aApply voice activity detection Parameters ---------- file : AudioFile Processed file. hook : callable, optional Callback called after each major steps of the pipeline as follows: hook(step_name, # human-readable name of current step step_artefact, # artifact generated by current step file=file) # file being processed Time-consuming steps call `hook` multiple times with the same `step_name` and additional `completed` and `total` keyword arguments usable to track progress of current step. Returns ------- speech : Annotation Speech regions. )rYr5NuricSsi|]}|dqS)rUr,).0labelr,r,r- sz0VoiceActivityDetection.apply..) setup_hooktrainingCACHED_SEGMENTATIONrErrWrZ rename_labelslabels)rLr"rY segmentationsr&r,r,r-r.s        zVoiceActivityDetection.applycC|jr tdddStdddSz'Return new instance of detection metricrBF)collar skip_overlapr6r r rSr,r,r- get_metric  z!VoiceActivityDetection.get_metriccC|jrdSdSNmaximizeminimizer6rSr,r,r- get_directionz$VoiceActivityDetection.get_direction)r4FNN)r/r0r1r2rboolrrrDrTrVrXr`rrrr r.r r rirp __classcell__r,r,rOr-rQs&   .rcsjeZdZdZ   ddedeedeffdd Zd e d e fd d Z d e e effddZddZZS)AdaptiveVoiceActivityDetectiona<Adaptive voice activity detection pipeline Let M be a pretrained voice activity detection model. For each file f, this pipeline starts by applying the model to obtain a first set of speech/non-speech labels. Those (automatic, possibly erroneous) labels are then used to fine-tune M on the very same file f into a M_f model, in a self-supervised manner. Finally, the fine-tuned model M_f is applied to file f to obtain the final (and hopefully better) speech/non-speech labels. During fine-tuning, frames where the pretrained model M is very confident are weighted more than those with lower confidence: the intuition is that the model will use these high confidence regions to adapt to recording conditions (e.g. background noise) and hence will eventually be better on the parts of f where it was initially not quite confident. Conversely, to avoid overfitting too much to those high confidence regions, we use data augmentation and freeze all but the final few layers of the pretrained model M. Parameters ---------- segmentation : Model, str, or dict, optional Pretrained segmentation model. Defaults to "hbredin/VoiceActivityDetection-PyanNet-DIHARD". augmentation : BaseWaveformTransform, or dict, optional torch_audiomentations waveform transform, used during fine-tuning. Defaults to no augmentation. fscore : bool, optional Optimize (precision/recall) fscore. Defaults to optimizing detection error rate. Hyper-parameters ---------------- num_epochs : int Number of epochs (where one epoch = going through the file once). batch_size : int Batch size. learning_rate : float Learning rate. See also -------- pyannote.audio.pipelines.utils.get_inference -hbredin/VoiceActivityDetection-PyanNet-DIHARDNFr5 augmentationr6csNtt||_t||_||_tdd|_t gd|_ t dd|_ dS)Nr ) gư>ry) rCrDr inferencerrwr6r num_epochsr batch_sizer learning_rate)rLr5rwr6rOr,r-rDs    z'AdaptiveVoiceActivityDetection.__init__r"r#c sdtd<tjddddd<tdddddd}|d <jd kr2d Stjtddddddd <Gfd d d t }t |jj d j j d}tjj}||_fdd}t|||_t}tjddtjddgd|d}||Wdn1swYt|jjjj d} | d<|S)NsegryTr9vadrArBrQr$r confidencecseZdZdZfddZdS)z;AdaptiveVoiceActivityDetection.apply..DummyProtocol DummyProtocolc3s VdSrrr,rSr"r,r- train_iterQs zFAdaptiveVoiceActivityDetection.apply..DummyProtocol.train_iterN)r/r0r1namerr,rr,r-rNsr)durationweightrrwcst|jdS)N)lr)r parametersr)rNrSr,r-configure_optimizers_szBAdaptiveVoiceActivityDetection.apply..configure_optimizersgpu)epochs_per_stageF) max_epochs acceleratordevices callbacksenable_checkpointingdefault_root_dir)devicer)dictrr<r=r instantiaterminabsr VoiceActivityDetectionTaskrrrwrrNtaskrrtempfileTemporaryDirectoryrrfitrr) rLr" vad_pipelinervad_task vad_modelrrtrainerrr,)r"rLr-r..s\          z$AdaptiveVoiceActivityDetection.applycCrdrerhrSr,r,r-rixrjz)AdaptiveVoiceActivityDetection.get_metriccCrkrlrorSr,r,r-rprqz,AdaptiveVoiceActivityDetection.get_direction)rvNF)r/r0r1r2rrrrsrDrr r.rr r rirprtr,r,rOr-rus2Jru)8r2rcopyr functoolsrtypesrtypingrrrrnumpyr< pyannote.corer r pyannote.database.protocolr pyannote.metrics.detectionr r pyannote.pipeline.parameterrrrrpytorch_lightningr torch.optimr/torch_audiomentations.core.transforms_interfacerpyannote.audiorpyannote.audio.core.callbackrpyannote.audio.core.iorpyannote.audio.core.pipelinerpyannote.audio.pipelines.utilsrrrrrrpyannote.audio.tasksrrpyannote.audio.utils.signalr r!rur,r,r,r-s2