o pi'@sddlmZmZmZmZmZmZddlZddl m Z m Z m Z ddl mZddlmZmZddlmZddlmZddlmZGd d d eZdS) )DictOptionalSequenceTextTupleUnionN)Problem ResolutionSpecifications)SegmentationTask)SegmentSlidingWindowFeature)Protocol)BaseWaveformTransform)MetriccseZdZdZ          ddedeeedfd ed eee eeffd ee e d ee d e dee de deedeee eeeeffffdd Zde ded efddZZS)VoiceActivityDetectiona Voice activity detection Voice activity detection (or VAD) is the task of detecting speech regions in a given audio recording. It is addressed as a binary (0 or 1) sequence labeling task. A frame is marked as "speech" (1) as soon as at least one speaker is active. Parameters ---------- protocol : Protocol pyannote.database protocol cache : str, optional As (meta-)data preparation might take a very long time for large datasets, it can be cached to disk for later (and faster!) re-use. When `cache` does not exist, `Task.prepare_data()` generates training and validation metadata from `protocol` and save them to disk. When `cache` exists, `Task.prepare_data()` is skipped and (meta)-data are loaded from disk. Defaults to a temporary path. duration : float, optional Chunks duration. Defaults to 2s. warm_up : float or (float, float), optional Use that many seconds on the left- and rightmost parts of each chunk to warm up the model. While the model does process those left- and right-most parts, only the remaining central part of each chunk is used for computing the loss during training, and for aggregating scores during inference. Defaults to 0. (i.e. no warm-up). balance: Sequence[Text], optional When provided, training samples are sampled uniformly with respect to these keys. For instance, setting `balance` to ["database","subset"] will make sure that each database & subset combination will be equally represented in the training samples. weight: str, optional When provided, use this key to as frame-wise weight in loss function. batch_size : int, optional Number of training samples per batch. Defaults to 32. num_workers : int, optional Number of workers used for generating training samples. Defaults to multiprocessing.cpu_count() // 2. pin_memory : bool, optional If True, data loaders will copy tensors into CUDA pinned memory before returning them. See pytorch documentation for more details. Defaults to False. augmentation : BaseWaveformTransform, optional torch_audiomentations waveform transform, used by dataloader during training. metric : optional Validation metric(s). Can be anything supported by torchmetrics.MetricCollection. Defaults to AUROC (area under the ROC curve). N@ Fprotocolcachedurationwarm_upbalanceweight batch_size num_workers pin_memory augmentationmetricc sPtj|||||| | | |d ||_||_ttjtj|j |j |j dgd|_ dS)N)rrrrrrrrspeech)problem resolutionr min_durationrclasses) super__init__rrr rBINARY_CLASSIFICATIONr FRAMErr#rspecifications) selfrrrrrrrrrrr __class__n/home/ubuntu/.local/lib/python3.10/site-packages/pyannote/audio/tasks/segmentation/voice_activity_detection.pyr&Vs,  zVoiceActivityDetection.__init__file_id start_timecs||}t|||}t}|jj||\|d<}|jd|jdd|k}||d|jk|d|jk@} |jj j } d|jj j } t | d|j|j| } t dt | | t} t | d|j|j| }t || t}|jt||jjj}t j|dft jd }t| |D]\} }d|| |ddf<qt||jj d gd |d <|jd |fddjjD|d<||dd<|S)a6Prepare chunk for voice activity detection Parameters ---------- file_id : int File index start_time : float Chunk start time duration : float Chunk duration. Returns ------- sample : dict Dictionary containing the chunk data with the following keys: - `X`: waveform - `y`: target as a SlidingWindowFeature instance - `meta`: - `database`: database index - `file`: file index Xzannotations-segmentsr/startendg?r)dtyper )labelsyzaudio-metadatacsi|]}||qSr-r-).0keymetadatar-r. sz8VoiceActivityDetection.prepare_chunk..metafile)get_filer dictmodelaudiocrop prepared_datar3r2receptive_fieldsteprnpmaximumroundastypeintminimum num_frameshparams sample_ratezerosuint8zipr r5names)r*r/r0rr>chunksample_ annotationschunk_annotationsrFhalfr2 start_idxr3end_idxrMr7r-r:r. prepare_chunk~s:     z$VoiceActivityDetection.prepare_chunk) NrrNNrNFNN)__name__ __module__ __qualname____doc__rrrstrfloatrrrrKboolrrrr&r\ __classcell__r-r-r+r.r#sH5     (r)typingrrrrrrnumpyrGpyannote.audio.core.taskrr r (pyannote.audio.tasks.segmentation.mixinsr pyannote.corer r pyannote.databaser/torch_audiomentations.core.transforms_interfacer torchmetricsrrr-r-r-r.s