o siI8@sdZddlZddlmZddlZddlmZddlm Z dej dej fdd Z d d Z dd dZ ddeefddZddZddZdS)u Alignment models are given a sequence of events along with a piece of audio, and then return a sequence of timestamps, with one timestamp for each event, indicating the position of this event in the audio. The events are listed in order of occurrence in the audio, so that output timestamps have to be monotonically increasing. Evaluation usually involves taking the series of predicted and ground truth timestamps and comparing their distance, usually on a pair-wise basis, e.g. taking the median absolute error in seconds. Conventions ----------- Timestamps should be provided in the form of a 1-dimensional array of onset times in seconds in increasing order. Metrics ------- * :func:`mir_eval.alignment.absolute_error`: Median absolute error and average absolute error * :func:`mir_eval.alignment.percentage_correct`: Percentage of correct timestamps, where a timestamp is counted as correct if it lies within a certain tolerance window around the ground truth timestamp * :func:`mir_eval.alignment.pcs`: Percentage of correct segments: Percentage of overlap between predicted segments and ground truth segments, where segments are defined by (start time, end time) pairs * :func:`mir_eval.alignment.perceptual_metric`: metric based on human synchronicity perception as measured in the paper "User-centered evaluation of lyrics to audio alignment", N. Lizé-Masclef, A. Vaglio, M. Moussallam, ISMIR 2021 References ---------- .. [#lizemasclef2021] N. Lizé-Masclef, A. Vaglio, M. Moussallam. "User-centered evaluation of lyrics to audio alignment", International Society for Music Information Retrieval (ISMIR) conference, 2021. .. [#mauch2010] M. Mauch, F: Hiromasa, M. Goto. "Lyrics-to-audio alignment and phrase-level segmentation using incomplete internet-style chord annotations", Frontiers in Proceedings of the Sound Music Computing Conference (SMC), 2010. .. [#dzhambazov2017] G. Dzhambazov. "Knowledge-Based Probabilistic Modeling For Tracking Lyrics In Music Audio Signals", PhD Thesis, 2017. .. [#fujihara2011] H. Fujihara, M. Goto, J. Ogata, H. Okuno. "LyricSynchronizer: Automatic synchronization system between musical audio signals and lyrics", IEEE Journal of Selected Topics in Signal Processing, VOL. 5, NO. 6, 2011 N)Optional)skewnorm) filter_kwargsreference_timestampsestimated_timestampscCs0t|tjstdt|t|tjstdt||jdkr,td|jd|jdkr:td|jd|jdkrCtd|j|jkrVtd |jd |jd t|dd |d d dksktdt|dd |d d dkstdt|dkstdt|dkstdd S)aICheck that the input annotations to a metric look like valid onset time arrays, and throws helpful errors if not. Parameters ---------- reference_timestamps : np.ndarray reference timestamp locations, in seconds estimated_timestamps : np.ndarray estimated timestamp locations, in seconds z7Reference timestamps need to be a numpy array, but got z7Estimated timestamps need to be a numpy array, but got zBReference timestamps need to be a one-dimensional vector, but got z dimensionszBEstimated timestamps need to be a one-dimensional vector, but got rzReference timestamps are empty.zPNumber of timestamps must be the same in prediction and ground truth, but found z in prediction and z in ground truthNz6Reference timestamps are not monotonically increasing!z6Estimated timestamps are not monotonically increasing!z(Reference timestamps can not be below 0!z(Estimated timestamps can not be below 0!) isinstancenpndarray ValueErrortypendimsizeall)rrrF/home/ubuntu/.local/lib/python3.10/site-packages/mir_eval/alignment.pyvalidate;sV      ""rcCs,t||t||}t|t|fS)aCompute the absolute deviations between estimated and reference timestamps, and then returns the median and average over all events Examples -------- >>> reference_timestamps = mir_eval.io.load_events('reference.txt') >>> estimated_timestamps = mir_eval.io.load_events('estimated.txt') >>> mae, aae = mir_eval.align.absolute_error(reference_onsets, estimated_timestamps) Parameters ---------- reference_timestamps : np.ndarray reference timestamps, in seconds estimated_timestamps : np.ndarray estimated timestamps, in seconds Returns ------- mae : float Median absolute error aae: float Average absolute error )rr absmedianmean)rr deviationsrrrabsolute_errorss r333333?cCs&t||t||}t||kS)a5Compute the percentage of correctly predicted timestamps. A timestamp is predicted correctly if its position doesn't deviate more than the window parameter from the ground truth timestamp. Examples -------- >>> reference_timestamps = mir_eval.io.load_events('reference.txt') >>> estimated_timestamps = mir_eval.io.load_events('estimated.txt') >>> pc = mir_eval.align.percentage_correct(reference_onsets, estimated_timestamps, window=0.2) Parameters ---------- reference_timestamps : np.ndarray reference timestamps, in seconds estimated_timestamps : np.ndarray estimated timestamps, in seconds window : float Window size, in seconds (Default value = .3) Returns ------- pc : float Percentage of correct timestamps )rr rr)rrwindowrrrrpercentage_corrects rdurationc CsHt|||durat|}|dkrtd|t||kr,tdt|d|t||kr@tdt|d|tdg|g}t||gg}tdg|g}t||gg}n(|d|d}|dkrqtd|dd}|d d}|dd}|d d}t||}t||}tt||d} | |S) a Calculate the percentage of correct segments (PCS) metric. It constructs segments out of predicted and estimated timestamps separately out of each given timestamp vector and calculates the percentage of overlap between correct segments compared to the total duration. WARNING: This metrics behaves differently depending on whether "duration" is given! If duration is not given (default case), the computation follows the MIREX lyrics alignment challenge 2020. For a timestamp vector with entries (t1,t2, ... tN), segments with the following (start, end) boundaries are created: (t1, t2), ... (tN-1, tN). After the segments are created, the overlap between the reference and estimated segments is determined and divided by the total duration, which is the distance between the first and last timestamp in the reference. If duration is given, the segment boundaries are instead (0, t1), (t1, t2), ... (tN, duration). The overlap is computed in the same way, but then divided by the duration parameter given to this function. This method follows the original paper [#fujihara2011] more closely, where the metric was proposed. As a result, this variant of the metrics punishes cases where the first estimated timestamp is too early or the last estimated timestamp is too late, whereas the MIREX variant does not. On the other hand, the MIREX metric is invariant to how long the eventless beginning and end parts of the audio are, which might be a desirable property. Examples -------- >>> reference_timestamps = mir_eval.io.load_events('reference.txt') >>> estimated_timestamps = mir_eval.io.load_events('estimated.txt') >>> pcs = mir_eval.align.percentage_correct_segments(reference_timestamps, estimated_timestamps) Parameters ---------- reference_timestamps : np.ndarray reference timestamps, in seconds estimated_timestamps : np.ndarray estimated timestamps, in seconds duration : float Optional. Total duration of audio (seconds). WARNING: Metric is computed differently depending on whether this is provided or not - see documentation above! Returns ------- pcs : float Percentage of time where ground truth and predicted segments overlap Nrz0Positive duration needs to be provided, but got z$Expected largest reference timestampz to not be larger than duration z%Expected largest estimated timestamp rzCReference timestamps are all identical, can not compute PCS metric!r) rfloatr r max concatenatemaximumminimumsum) rrr ref_startsref_ends est_startsest_endsoverlap_starts overlap_endsoverlap_durationrrrpercentage_correct_segmentssP 1      r*cCsFt||||}d}d}d}d}d|tj||||d}t|S)aMetric based on human synchronicity perception as measured in the paper "User-centered evaluation of lyrics to audio alignment" [#lizemasclef2021] The parameters of this function were tuned on data collected through a user Karaoke-like experiment It reflects human judgment of how "synchronous" lyrics and audio stimuli are perceived in that setup. Beware that this metric is non-symmetrical and by construction it is also not equal to 1 at 0. Examples -------- >>> reference_timestamps = mir_eval.io.load_events('reference.txt') >>> estimated_timestamps = mir_eval.io.load_events('estimated.txt') >>> score = mir_eval.align.karaoke_perceptual_metric(reference_onsets, estimated_timestamps) Parameters ---------- reference_timestamps : np.ndarray reference timestamps, in seconds estimated_timestamps : np.ndarray estimated timestamps, in seconds Returns ------- perceptual_score : float Perceptual score, averaged over all timestamps gDqG?gݫl̿g-o?g -?g?)locscale)rrpdfr r)rroffsetsskewness localisationr,normalisation_factorperceptual_scoresrrrkaraoke_perceptual_metric s   r3cKs`t}tt||fi||d<t||\|d<|d<tt||fi||d<t|||d<|S)aCompute all metrics for the given reference and estimated annotations. Examples -------- >>> reference_timestamps = mir_eval.io.load_events('reference.txt') >>> estimated_timestamps = mir_eval.io.load_events('estimated.txt') >>> duration = max(np.max(reference_timestamps), np.max(estimated_timestamps)) + 10 >>> scores = mir_eval.align.evaluate(reference_onsets, estimated_timestamps, duration) Parameters ---------- reference_timestamps : np.ndarray reference timestamp locations, in seconds estimated_timestamps : np.ndarray estimated timestamp locations, in seconds **kwargs Additional keyword arguments which will be passed to the appropriate metric or preprocessing functions. Returns ------- scores : dict Dictionary of scores, where the key is the metric name (str) and the value is the (float) score achieved. pcmaeaaepcs perceptual) collections OrderedDictrrrr*r3)rrkwargsscoresrrrevaluate8s(  r=)r)N)__doc__r9typingrnumpyr scipy.statsr mir_eval.utilrr rrrrr*r3r=rrrrs1   8   ^ +