o si\@sdZddlZddlZddlZddlZddlZddlm Z ddl m Z ddZ dd Z d#d d Zd dZddZddZddZd$ddZddZ    d%ddZ  d&dd Zd!d"ZdS)'aEvaluation criteria for hierarchical structure analysis. Hierarchical structure analysis seeks to annotate a track with a nested decomposition of the temporal elements of the piece, effectively providing a kind of "parse tree" of the composition. Unlike the flat segmentation metrics defined in :mod:`mir_eval.segment`, which can only encode one level of analysis, hierarchical annotations expose the relationships between short segments and the larger compositional elements to which they belong. Conventions ----------- Annotations are assumed to take the form of an ordered list of segmentations. As in the :mod:`mir_eval.segment` metrics, each segmentation itself consists of an n-by-2 array of interval times, so that the ``i`` th segment spans time ``intervals[i, 0]`` to ``intervals[i, 1]``. Hierarchical annotations are ordered by increasing specificity, so that the first segmentation should contain the fewest segments, and the last segmentation contains the most. Metrics ------- * :func:`mir_eval.hierarchy.tmeasure`: Precision, recall, and F-measure of triplet-based frame accuracy for boundary detection. * :func:`mir_eval.hierarchy.lmeasure`: Precision, recall, and F-measure of triplet-based frame accuracy for segment labeling. References ---------- .. [#mcfee2015] Brian McFee, Oriol Nieto, and Juan P. Bello. "Hierarchical evaluation of segment boundary detection", International Society for Music Information Retrieval (ISMIR) conference, 2015. .. [#mcfee2017] Brian McFee, Oriol Nieto, Morwaread Farbood, and Juan P. Bello. "Evaluating hierarchical structure in music annotations", Frontiers in Psychology, 2017. N)util)validate_structurecCs|t|t|S)aRound a time-stamp to a specified resolution. Equivalent to ``t - np.mod(t, frame_size)``. Examples -------- >>> _round(53.279, 0.1) 53.2 >>> _round(53.279, 0.25) 53.25 Parameters ---------- t : number or ndarray The time-stamp to round frame_size : number > 0 The resolution to round to Returns ------- t_round : number The rounded time-stamp )npmodfloat)t frame_sizer F/home/ubuntu/.local/lib/python3.10/site-packages/mir_eval/hierarchy.py_round6sr cCs(ttjttj|}t|t|fS)acCompute the covered time range of a hierarchical segmentation. Parameters ---------- intervals_hier : list of ndarray A hierarchical segmentation, encoded as a list of arrays of segment intervals. Returns ------- t_min : float t_max : float The minimum and maximum times spanned by the annotation )list itertoolschainminmax)intervals_hier boundariesr r r _hierarchy_boundsQsrcs(ddtfddt||DDS)aAlign a hierarchical annotation to span a fixed start and end time. Parameters ---------- int_hier : list of list of intervals lab_hier : list of list of str Hierarchical segment annotations, encoded as a list of list of intervals (int_hier) and list of list of strings (lab_hier) t_min : None or number >= 0 The minimum time value for the segmentation t_max : None or number >= t_min The maximum time value for the segmentation Returns ------- intervals_hier : list of list of intervals labels_hier : list of list of str `int_hier` `lab_hier` aligned to span `[t_min, t_max]`. cSsg|]}t|qSr )r ).0_r r r zsz$_align_intervals..cs(g|]\}}tjt||dqS))labelst_mint_max)radjust_intervalsrasarray)rivallabrrr r r}s )zip)int_hierlab_hierrrr r r _align_intervalses r$c Cst|}t|\}}tt||t|||}tjj||ftjd}t |dD]#\}}tt ||| tD]}t |d|d} ||| | f<q9q(| S)aCompute the (sparse) least-common-ancestor (LCA) matrix for a hierarchical segmentation. For any pair of frames ``(s, t)``, the LCA is the deepest level in the hierarchy such that ``(s, t)`` are contained within a single segment at that level. Parameters ---------- intervals_hier : list of ndarray An ordered list of segment interval arrays. The list is assumed to be ordered by increasing specificity (depth). frame_size : number The length of the sample frames (in seconds) Returns ------- lca_matrix : scipy.sparse.csr_matrix A sparse matrix such that ``lca_matrix[i, j]`` contains the depth of the deepest segment containing frames ``i`` and ``j``. dtyperr)rrintr scipysparse lil_matrixruint8 enumeraterastypeslicetocsr) rr n_startn_endn lca_matrixlevel intervalsridxr r r _lcas r7cCst|}t|\}}tt||t|||}tjj||ftjd}t t ||dD]M\}\}} t | d} t tj| | } t|||t} t t| D]$\} }tt| | }tt| |}||||f<| |krw||||f<qSq+tj|S)a}Compute the (sparse) least-common-ancestor (LCA) matrix for a hierarchical segmentation. For any pair of frames ``(s, t)``, the LCA is the deepest level in the hierarchy such that ``(s, t)`` are contained within a single segment at that level. Parameters ---------- intervals_hier : list of ndarray An ordered list of segment interval arrays. The list is assumed to be ordered by increasing specificity (depth). labels_hier : list of list of str ``labels_hier[i]`` contains the segment labels for the ``i``th layer of the annotations frame_size : number The length of the sample frames (in seconds) Returns ------- meet_matrix : scipy.sparse.csr_matrix A sparse matrix such that ``meet_matrix[i, j]`` contains the depth of the deepest segment label containing both ``i`` and ``j``. r%rr)rrr'r r(r)r*rr+r,r!r index_labelstriuequalouterr-wherer.r csr_matrix)r labels_hierr r0r1r2 meet_matrixr4r5rlab_enc int_agree int_framesseg_iseg_jidx_iidx_jr r r _meets"    rGcCs*|j|jkr td|jd}|dur|}d}d}t|D]i}ttd||t|||}|||f} |||f} | } | } t||} t | d| | | ddf} t | d| | | ddf} t | | |d\} } | r|d| t | 7}|d7}q|r|t |}|Sd}|S)aGeneralized area under the curve (GAUC) This function computes the normalized recall score for correctly ordering triples ``(q, i, j)`` where frames ``(q, i)`` are closer than ``(q, j)`` in the reference annotation. Parameters ---------- ref_lca : scipy.sparse est_lca : scipy.sparse The least common ancestor matrices for the reference and estimated annotations transitive : bool If True, then transitive comparisons are counted, meaning that ``(q, i)`` and ``(q, j)`` can differ by any number of levels. If False, then ``(q, i)`` and ``(q, j)`` can differ by exactly one level. window : number or None The maximum number of frames to consider for each query. If `None`, then all frames are considered. Returns ------- score : number [0, 1] The percentage of reference triples correctly ordered by the estimation. Raises ------ ValueError If ``ref_lca`` and ``est_lca`` have different shapes z=Estimated and reference hierarchies must have the same shape.rNrr) transitive?) shape ValueErrorranger.rrtoarraysqueezer concatenate_compare_frame_rankingsr)ref_lcaest_lcarHwindowr2score num_framesqueryresults ref_score est_scorer6 inversions normalizerr r r _gaucs< '       ""  r\cCstj|dd\}}tj|dd\}}d}d}d}|t|krX|t|krX||||kr1|d7}n||||krL|t||d||7}|d7}|t|krX|t|ks$|S)awCount the number of inversions in two numpy arrays: # points i, j where a[i] >= b[j] Parameters ---------- a, b : np.ndarray, shape=(n,) (m,) The arrays to be compared. This implementation is optimized for arrays with many repeated values. Returns ------- inversions : int The number of detected inversions T) return_countsrrN)runiquelensum)aba_countsb_countsrZijr r r _count_inversionsJs rgFcs4t|}||}||}tj|ddd\}}}t|}|t|tdd} tddt|||dd|ddD]\} } } } t | | | | <| | <q@|r[t |d }nd d |D}t |\}}t tfd d |D}|d kr|dSd }|D]\}}|t|| ||| |7}q|t |fS)aCompute the number of ranking disagreements in two lists. Parameters ---------- ref : np.ndarray, shape=(n,) est : np.ndarray, shape=(n,) Reference and estimate ranked lists. `ref[i]` is the relevance score for point `i`. transitive : bool If true, all pairs of reference levels are compared. If false, only adjacent pairs of reference levels are compared. Returns ------- inversions : int The number of pairs of indices `i, j` where `ref[i] < ref[j]` but `est[i] >= est[j]`. normalizer : float The total number of pairs (i, j) under consideration. If transitive=True, then this is |{(i,j) : ref[i] < ref[j]}| If transitive=False, then this is |{i,j) : ref[i] +1 = ref[j]}| T) return_indexr]cSstdSNr)r.r r r r sz)_compare_frame_rankings..cSsdSrir r r r r rjsNrcSsg|]}||dfqS)rr )rrer r r rsz+_compare_frame_rankings..cs g|] \}}||qSr r )rrerfref_mapr r rs r)rr)rargsortr^r appendr_ collections defaultdictr!r.r combinationsteerr`rg)refestrHr6 ref_sorted est_sortedlevels positionscountsindexr4cntstartend level_pairslcounterr[rZlevel_1level_2r rmr rPms4  *   rPcCst|d}tt|d}t|dddD])\}}t|}t|d|||tt|}||r>td|||O}qdS)a=Validate a hierarchical segment annotation. Parameters ---------- intervals_hier : ordered list of segmentations Raises ------ ValueError If any segmentation does not span the full duration of the top-level segmentation. If any segmentation does not start at 0. rrNz/Segment hierarchy is inconsistent at level {:d}) rgenerate_labelssetintervals_to_boundariesr,rwarningswarnformat)r label_toprr4r5 label_current new_boundsr r r validate_hier_intervalss  r.@皙?rIc Cs|dkr td||durd}n||krtd||tt|||}t|t|t||}t||}t||||} t||||} tj| | |d} | | | fS)arCompute the tree measures for hierarchical segment annotations. Parameters ---------- reference_intervals_hier : list of ndarray ``reference_intervals_hier[i]`` contains the segment intervals (in seconds) for the ``i`` th layer of the annotations. Layers are ordered from top to bottom, so that the last list of intervals should be the most specific. estimated_intervals_hier : list of ndarray Like ``reference_intervals_hier`` but for the estimated annotation transitive : bool whether to compute the t-measures using transitivity or not. window : float > 0 size of the window (in seconds). For each query frame q, result frames are only counted within q +- window. frame_size : float > 0 length (in seconds) of frames. The frame size cannot be longer than the window. beta : float > 0 beta parameter for the F-measure. Returns ------- t_precision : number [0, 1] T-measure Precision t_recall : number [0, 1] T-measure Recall t_measure : number [0, 1] F-beta measure for ``(t_precision, t_recall)`` Raises ------ ValueError If either of the input hierarchies are inconsistent If the input hierarchies have different time durations If ``frame_size > window`` or ``frame_size <= 0`` r.frame_size ({:.2f}) must be a positive number.Nz1frame_size ({:.2f}) cannot exceed window ({:.2f})beta) rKrr'r rr7r\r f_measure) reference_intervals_hierestimated_intervals_hierrHrSr r window_framesrQrRt_recall t_precision t_measurer r r tmeasures(1   rc Cst|dkr td|t|t|t|||}t|||}t||dd}t||dd} tj| ||d} | || fS)axCompute the tree measures for hierarchical segment annotations. Parameters ---------- reference_intervals_hier : list of ndarray ``reference_intervals_hier[i]`` contains the segment intervals (in seconds) for the ``i`` th layer of the annotations. Layers are ordered from top to bottom, so that the last list of intervals should be the most specific. reference_labels_hier : list of list of str ``reference_labels_hier[i]`` contains the segment labels for the ``i`` th layer of the annotations estimated_intervals_hier : list of ndarray estimated_labels_hier : list of ndarray Like ``reference_intervals_hier`` and ``reference_labels_hier`` but for the estimated annotation frame_size : float > 0 length (in seconds) of frames. The frame size cannot be longer than the window. beta : float > 0 beta parameter for the F-measure. Returns ------- l_precision : number [0, 1] L-measure Precision l_recall : number [0, 1] L-measure Recall l_measure : number [0, 1] F-beta measure for ``(l_precision, l_recall)`` Raises ------ ValueError If either of the input hierarchies are inconsistent If the input hierarchies have different time durations If ``frame_size > window`` or ``frame_size <= 0`` rrTNr)rKrrrGr\rr) rreference_labels_hierrestimated_labels_hierr rref_meetest_meetl_recall l_precision l_measurer r r lmeasure$s1   rcKst|\}}t||ddd\}}t||d|d\}}t}d|d<tjt||fi|\|d<|d<|d<d |d<tjt||fi|\|d <|d <|d <tjt||||fi|\|d <|d<|d<|S)a Compute all hierarchical structure metrics for the given reference and estimated annotations. Examples -------- A toy example with two two-layer annotations >>> ref_i = [[[0, 30], [30, 60]], [[0, 15], [15, 30], [30, 45], [45, 60]]] >>> est_i = [[[0, 45], [45, 60]], [[0, 15], [15, 30], [30, 45], [45, 60]]] >>> ref_l = [ ['A', 'B'], ['a', 'b', 'a', 'c'] ] >>> est_l = [ ['A', 'B'], ['a', 'a', 'b', 'b'] ] >>> scores = mir_eval.hierarchy.evaluate(ref_i, ref_l, est_i, est_l) >>> dict(scores) {'T-Measure full': 0.94822745804853459, 'T-Measure reduced': 0.8732458222764804, 'T-Precision full': 0.96569179094693058, 'T-Precision reduced': 0.89939075137018787, 'T-Recall full': 0.93138358189386117, 'T-Recall reduced': 0.84857799953694923} A more realistic example, using SALAMI pre-parsed annotations >>> def load_salami(filename): ... "load SALAMI event format as labeled intervals" ... events, labels = mir_eval.io.load_labeled_events(filename) ... intervals = mir_eval.util.boundaries_to_intervals(events)[0] ... return intervals, labels[:len(intervals)] >>> ref_files = ['data/10/parsed/textfile1_uppercase.txt', ... 'data/10/parsed/textfile1_lowercase.txt'] >>> est_files = ['data/10/parsed/textfile2_uppercase.txt', ... 'data/10/parsed/textfile2_lowercase.txt'] >>> ref = [load_salami(fname) for fname in ref_files] >>> ref_int = [seg[0] for seg in ref] >>> ref_lab = [seg[1] for seg in ref] >>> est = [load_salami(fname) for fname in est_files] >>> est_int = [seg[0] for seg in est] >>> est_lab = [seg[1] for seg in est] >>> scores = mir_eval.hierarchy.evaluate(ref_int, ref_lab, ... est_hier, est_lab) >>> dict(scores) {'T-Measure full': 0.66029225561405358, 'T-Measure reduced': 0.62001868041578034, 'T-Precision full': 0.66844764668949885, 'T-Precision reduced': 0.63252297209957919, 'T-Recall full': 0.6523334654992341, 'T-Recall reduced': 0.60799919710921635} Parameters ---------- ref_intervals_hier : list of list-like ref_labels_hier : list of list of str est_intervals_hier : list of list-like est_labels_hier : list of list of str Hierarchical annotations are encoded as an ordered list of segmentations. Each segmentation itself is a list (or list-like) of intervals (\*_intervals_hier) and a list of lists of labels (\*_labels_hier). **kwargs additional keyword arguments to the evaluation metrics. Returns ------- scores : OrderedDict Dictionary of scores, where the key is the metric name (str) and the value is the (float) score achieved. T-measures are computed in both the "full" (``transitive=True``) and "reduced" (``transitive=False``) modes. Raises ------ ValueError Thrown when the provided annotations are not valid. rN)rrFrHzT-Precision reducedzT-Recall reducedzT-Measure reducedTzT-Precision fullz T-Recall fullzT-Measure fullz L-PrecisionzL-Recallz L-Measure)rr$rq OrderedDictr filter_kwargsrr)ref_intervals_hierref_labels_hierest_intervals_hierest_labels_hierkwargsrt_endscoresr r r evaluateksD N   r)rN)F)FrrrI)rrI)__doc__rqrrnumpyr scipy.sparser(rsegmentrr rr$r7rGr\rgrPrrrrr r r r s4)   "+9_ #B& W G