o piT@sddlmZmZddlmZmZddlmZmZddl m Z m Z ddl m Z dZdZd Zd ZGd d d e eZd ZdZdZdZdZGdddeZdZdZdZGdddeZdZdZdZGdddeZ dZ!dZ"dZ#dZ$Gddde eZ%d Z&d!Z'd"Z(d Z)d Z*Gd#d$d$e eZ+d%S)&)OptionalTuple) AnnotationTimeline) BaseMetric f_measure)DetailsMetricComponents)UEMSupportMixinzdetection error ratetotalz false alarmmissc seZdZdZedefddZedefddZdd e d e ffd d Z dde de de edefddZdede fddZZS)DetectionErrorRateaDetection error rate This metric can be used to evaluate binary classification tasks such as speech activity detection, for instance. Inputs are expected to only contain segments corresponding to the positive class (e.g. speech regions). Gaps in the inputs considered as the negative class (e.g. non-speech regions). It is computed as (fa + miss) / total, where fa is the duration of false alarm (e.g. non-speech classified as speech), miss is the duration of missed detection (e.g. speech classified as non-speech), and total is the total duration of the positive class in the reference. Parameters ---------- collar : float, optional Duration (in seconds) of collars removed from evaluation around boundaries of reference segments (one half before, one half after). skip_overlap : bool, optional Set to True to not evaluate overlap regions. Defaults to False (i.e. keep overlap regions). returncCtSN)DER_NAMEclsrN/home/ubuntu/.local/lib/python3.10/site-packages/pyannote/metrics/detection.py metric_nameCzDetectionErrorRate.metric_namecC tttgSr) DER_TOTALDER_FALSE_ALARMDER_MISSrrrrmetric_componentsG z$DetectionErrorRate.metric_componentsFcollar skip_overlapc s&tt|jdi|||_||_dSNr)superr__init__r r!)selfr r!kwargs __class__rrr$Ks zDetectionErrorRate.__init__N reference hypothesisuemcKs|j||||j|jdd\}}}|jdd}|jdd}|j|d}|j|d}d}||D] \}} ||| @j7}q3d} ||D] \} } | | | @j7} qFi} | | t<|| t <|| t <| SNT)r+r r! returns_uemF)copy)supportr) uemifyr r! get_timeliner/gapsco_iterdurationrrr)r%r)r*r+r& reference_ hypothesis_false_positiver_hfalse_negativerh_detailrrrcompute_componentsPs(    z%DetectionErrorRate.compute_componentsr=cCs@d|t|t}d|t}|dkr|dkrdSdS||SN?rr)rrr)r%r=errorr rrrcompute_metricps z!DetectionErrorRate.compute_metric)rFr)__name__ __module__ __qualname____doc__ classmethodstrrr rfloatboolr$rrrr r>rB __classcell__rrr'rr+s$  rzdetection accuracyz true positivez true negativezfalse positivezfalse negativec @beZdZdZeddZedefddZ dded ed e e de fd d Z d e de fddZdS)DetectionAccuracyaDetection accuracy This metric can be used to evaluate binary classification tasks such as speech activity detection, for instance. Inputs are expected to only contain segments corresponding to the positive class (e.g. speech regions). Gaps in the inputs considered as the negative class (e.g. non-speech regions). It is computed as (tp + tn) / total, where tp is the duration of true positive (e.g. speech classified as speech), tn is the duration of true negative (e.g. non-speech classified as non-speech), and total is the total duration of the input signal. Parameters ---------- collar : float, optional Duration (in seconds) of collars removed from evaluation around boundaries of reference segments (one half before, one half after). skip_overlap : bool, optional Set to True to not evaluate overlap regions. Defaults to False (i.e. keep overlap regions). cCrr) ACCURACY_NAMErrrrrrzDetectionAccuracy.metric_namercC ttttgSr)ACCURACY_TRUE_POSITIVEACCURACY_TRUE_NEGATIVEACCURACY_FALSE_POSITIVEACCURACY_FALSE_NEGATIVErrrrrsz#DetectionAccuracy.metric_componentsNr)r*r+cKs|j||||j|jdd\}}}|jdd}|jdd}|j|d}|j|d}d}||D] \}} ||| @j7}q3d} ||D] \} } | | | @j7} qFd} ||D] \} } | | | @j7} qYd}||D] \}} ||| @j7}qli}| |t<||t <||t <| |t <|Sr,) r0r r!r1r/r2r3r4rQrPrSrR)r%r)r*r+r&r5r6 true_positiver;r9 true_negativer8r<r7r:r=rrrr>s6   z$DetectionAccuracy.compute_componentsr=cCsLd|t|t}d|t|t|t|t}|dkr"dS||SNr@r)rQrPrSrR)r%r= numerator denominatorrrrrBsz DetectionAccuracy.compute_metricrrCrDrErFrGrr rrrrr r>rIrBrrrrrMs"  )rMzdetection precision retrievedzrelevant retrievedc @rL)DetectionPrecisionaDetection precision This metric can be used to evaluate binary classification tasks such as speech activity detection, for instance. Inputs are expected to only contain segments corresponding to the positive class (e.g. speech regions). Gaps in the inputs considered as the negative class (e.g. non-speech regions). It is computed as tp / (tp + fp), where tp is the duration of true positive (e.g. speech classified as speech), and fp is the duration of false positive (e.g. non-speech classified as speech). Parameters ---------- collar : float, optional Duration (in seconds) of collars removed from evaluation around boundaries of reference segments (one half before, one half after). skip_overlap : bool, optional Set to True to not evaluate overlap regions. Defaults to False (i.e. keep overlap regions). cCrr)PRECISION_NAMErrrrrrzDetectionPrecision.metric_namercCttgSr)PRECISION_RETRIEVEDPRECISION_RELEVANT_RETRIEVEDrrrrrz$DetectionPrecision.metric_componentsNr)r*r+c Ks|j||||j|jdd\}}}|jdd}|jdd}|j|d}d}||D] \}}|||@j7}q-d} ||D] \} }| | |@j7} q@i} || | t<|| t <| Sr,) r0r r!r1r/r2r3r4r^r_) r%r)r*r+r&r5rTr;r9r7r8r=rrrr>$   z%DetectionPrecision.compute_componentsr=cCs,d|t}d|t}|dkrdS||SrV)r_r^)r%r=relevant_retrievedrZrrrrBs  z!DetectionPrecision.compute_metricrrYrrrrr["  r[zdetection recallrelevantc @rL)DetectionRecalla~Detection recall This metric can be used to evaluate binary classification tasks such as speech activity detection, for instance. Inputs are expected to only contain segments corresponding to the positive class (e.g. speech regions). Gaps in the inputs considered as the negative class (e.g. non-speech regions). It is computed as tp / (tp + fn), where tp is the duration of true positive (e.g. speech classified as speech), and fn is the duration of false negative (e.g. speech classified as non-speech). Parameters ---------- collar : float, optional Duration (in seconds) of collars removed from evaluation around boundaries of reference segments (one half before, one half after). skip_overlap : bool, optional Set to True to not evaluate overlap regions. Defaults to False (i.e. keep overlap regions). cCrr) RECALL_NAMErrrrrBrzDetectionRecall.metric_namercCr]r)RECALL_RELEVANTRECALL_RELEVANT_RETRIEVEDrrrrrFr`z!DetectionRecall.metric_componentsNr)r*r+c Ks|j||||j|jdd\}}}|jdd}|jdd}|j|d}d}||D] \}}|||@j7}q-d} ||D] \}} | || @j7} q@i} || | t<|| t <| Sr,) r0r r!r1r/r2r3r4rgrh) r%r)r*r+r&r6rTr;r9r:r<r=rrrr>Jraz"DetectionRecall.compute_componentsr=cCs8d|t}d|t}|dkr|dkrdSdS||Sr?)rhrg)r%r=rbrdrrrrBhs  zDetectionRecall.compute_metricrrYrrrrre+rcrezF[precision|recall]c seZdZdZeddZeddZ  dd ed ed effd d Z dde de de e de fddZde defddZdde e deeeeffddZZS) DetectionPrecisionRecallFMeasureaCompute detection precision and recall, and return their F-score Parameters ---------- collar : float, optional Duration (in seconds) of collars removed from evaluation around boundaries of reference segments (one half before, one half after). skip_overlap : bool, optional Set to True to not evaluate overlap regions. Defaults to False (i.e. keep overlap regions). beta : float, optional When beta > 1, greater importance is given to recall. When beta < 1, greater importance is given to precision. Defaults to 1. See also -------- pyannote.metrics.detection.DetectionPrecision pyannote.metrics.detection.DetectionRecall pyannote.metrics.base.f_measure cCrr)DFS_NAMErrrrrrz,DetectionPrecisionRecallFMeasure.metric_namecCrr)DFS_PRECISION_RETRIEVEDDFS_RECALL_RELEVANTDFS_RELEVANT_RETRIEVEDrrrrrrz2DetectionPrecisionRecallFMeasure.metric_componentsrFr@r r!betac s,tt|jdi|||_||_||_dSr")r#rir$r r!rn)r%r r!rnr&r'rrr$s z)DetectionPrecisionRecallFMeasure.__init__Nr)r*r+rcKs|j||||j|jdd\}}}|jdd}|jdd}|j|d}|j|d}d}||D] \}} ||| @j7}q3d} ||D] \} } | | | @j7} qFd} ||D] \}} | || @j7} qYt|| t || t |i}|Sr,) r0r r!r1r/r2r3r4rkrlrm)r%r)r*r+r&r5r6rTr;r9r7r8r:r<r=rrrr>s.   z3DetectionPrecisionRecallFMeasure.compute_componentsr=cCs|j|d\}}}|S)N)r=)compute_metrics)r%r=_valuerrrrBsz/DetectionPrecisionRecallFMeasure.compute_metriccCsz|dur|jn|}|t}|t}|t}|dkrd}n||}|dkr.|dkr+d}nd}n||}||t|||jdfS)Nrrrr@)rn) accumulated_rkrlrmrrn)r%r=precision_retrievedrecall_relevantrb precisionrecallrrrrosz0DetectionPrecisionRecallFMeasure.compute_metrics)rFr@r)rCrDrErFrGrrrIrJr$rrrr r>rBrrorKrrr'rrizs2     &  rizdetection cost functionzpositive class totalznegative class totalc sxeZdZdZ  dfdd Zedd Zed efd d Z dde de de e d e fddZ de d efddZZS)DetectionCostFunctionaDetection cost function. This metric can be used to evaluate binary classification tasks such as speech activity detection. Inputs are expected to only contain segments corresponding to the positive class (e.g. speech regions). Gaps in the inputs considered as the negative class (e.g. non-speech regions). Detection cost function (DCF), as defined by NIST for OpenSAT 2019, is 0.25*far + 0.75*missr, where far is the false alarm rate (i.e., the proportion of non-speech incorrectly classified as speech) and missr is the miss rate (i.e., the proportion of speech incorrectly classified as non-speech. Parameters ---------- collar : float, optional Duration (in seconds) of collars removed from evaluation around boundaries of reference segments (one half before, one half after). Defaults to 0.0. skip_overlap : bool, optional Set to True to not evaluate overlap regions. Defaults to False (i.e. keep overlap regions). fa_weight : float, optional Weight for false alarm rate. Defaults to 0.25. miss_weight : float, optional Weight for miss rate. Defaults to 0.75. kwargs Keyword arguments passed to :class:`pyannote.metrics.base.BaseMetric`. References ---------- "OpenSAT19 Evaluation Plan v2." https://www.nist.gov/system/files/documents/2018/11/05/opensat19_evaluation_plan_v2_11-5-18.pdf rF??c s2tt|jdi|||_||_||_||_dSr")r#rwr$r r! fa_weight miss_weight)r%r r!rzr{r&r'rrr$s  zDetectionCostFunction.__init__cCrr)DCF_NAMErrrrrrz!DetectionCostFunction.metric_namercCrOr) DCF_POS_TOTAL DCF_NEG_TOTALDCF_FALSE_ALARMDCF_MISSrrrrr!s z'DetectionCostFunction.metric_componentsNr)r*r+cKs|j||||j|jdd\}}}|jdd}|jdd}|j|d}|j|d}|}|}d} ||D] \} } | | | @j7} q;d} ||D] \} }| | |@j7} qNt|t |t | t | i}|Sr,) r0r r!r1r/r2r4r3r}r~rr)r%r)r*r+r&r5r6pos_durneg_durmiss_durr;r<fa_durr8r9 componentsrrrr>%s.   z(DetectionCostFunction.compute_componentsrc CsPdd}|t}|t}|||}|t}|t}|||}|j||j|S)NcSs |dkr |dkr dSdS||S)Nrr@r)numdenomrrr _compute_rateOs z;DetectionCostFunction.compute_metric.._compute_rate)r~rr}rrzr{) r%rrrrfa_raterr miss_raterrrrBNs  z$DetectionCostFunction.compute_metric)rFrxryr)rCrDrErFr$rGrr rrrrr r>rIrBrKrrr'rrws((  )rwN),typingrr pyannote.corerrbaserrtypesr r utilsr rrrrrrNrPrQrRrSrMr\r^r_r[rfrgrhrerjrkrlrmrir|r}r~rrrwrrrrsF QXFIk