o 5ti)@sdZddlmZmZmZmZddlmZddlm Z ddl m Z m Z m Z ddlmZmZGd d d e ZGd d d e ZGd dde ZdS)uNThe implementation of chrF (Popović 2015) and chrF++ (Popović 2017) metrics.)ListSequenceOptionalDict)Counter) sum_of_lists)Score SignatureMetric)extract_all_char_ngramsextract_word_ngramscs&eZdZdZdeffdd ZZS) CHRFSignaturezA convenience class to represent the reproducibility signature for chrF. :param args: key-value dictionary passed from the actual metric instance. argscslt||jdddddd|j|drdnd |d s"d nd |d |d|dr/d nd ddS)z`CHRFSignature` initializer.cencnws)caseeffrrspace lowercaselcmixed eps_smoothingyesno char_order word_order whitespaceN)super__init___abbrupdateinfo)selfr __class__J/home/ubuntu/.local/lib/python3.10/site-packages/sacrebleu/metrics/chrf.pyr#s  zCHRFSignature.__init__)__name__ __module__ __qualname____doc__dictr# __classcell__r*r*r(r+r srcs2eZdZdZdedededeffdd ZZS) CHRFScorea9A convenience class to represent chrF scores. :param score: The chrF (chrF++) score. :param char_order: The character n-gram order. :param word_order: The word n-gram order. If equals to 2, the metric is referred to as chrF++. :param beta: Determine the importance of recall w.r.t precision. scorerr betacs:||_||_||_d|jd|j}t||dS)z`CHRFScore` initializer.chrF+N)r4rr r"r#)r'r3rr r4namer(r*r+r#,s zCHRFScore.__init__)r,r-r.r/floatintr#r1r*r*r(r+r2$s&r2cs>eZdZdZdZdZdZedZe Z eeeddddfde d e d e d e d e d e de eeeffdd Zedededee fddZdedeefddZdedefddZdee defddZdee defdd Zdeee defd!d"Zd#eedeeeeeffd$d%Zd&ed'edee fd(d)ZZS)*CHRFalComputes the chrF(++) metric given hypotheses and references. :param char_order: Character n-gram order. :param word_order: Word n-gram order. If equals to 2, the metric is referred to as chrF++. :param beta: Determine the importance of recall w.r.t precision. :param lowercase: Enable case-insensitivity. :param whitespace: If `True`, include whitespaces when extracting character n-grams. :param eps_smoothing: If `True`, applies epsilon smoothing similar to reference chrF++.py, NLTK and Moses implementations. Otherwise, it takes into account effective match order similar to sacreBLEU < 2.0.0. :param references: A sequence of reference documents with document being defined as a sequence of reference strings. If given, the reference n-grams will be pre-computed and cached for faster re-computation across many systems. rrz !"#$%&'()*+,-./:;<=>?@[\]^_`{|}~FNrr r4rr!r referencescsXt||_||_||_|j|j|_||_||_||_|dur*| ||_ dSdS)z`CHRF` initializer.N) r"r#r4rr orderrr!r_cache_references _ref_cache)r'rr r4rr!rr<r(r*r+r#Vs z CHRF.__init__ hyp_ngrams ref_ngramsreturncCsVd\}}|D]\}}||7}||vr|t|||7}q|r"|ndt||gS)a`Computes the match statistics between hypothesis and reference n-grams. :param hyp_ngrams: A `Counter` holding hypothesis n-grams. :param ref_ngrams: A `Counter` holding reference n-grams. :return: A list of three numbers denoting hypothesis n-gram count, reference n-gram count and the intersection count. )rrr)itemsminsumvalues)r@rA match_count hyp_countngcountr*r*r+_get_match_statisticsls   zCHRF._get_match_statisticssentcCsg}|D];}t|dkr||q|d|jvr(||dd|dg7}q|d|jvr<||d|ddg7}q||q|S)zSeparates out punctuations from beginning and end of words for chrF. Adapted from https://github.com/m-popovic/chrF :param sent: A string. :return: A list of words. r Nr)splitlenappend_PUNCTS)r'rL tokenizedwr*r*r+_remove_punctuations    zCHRF._remove_punctuationcCs|jr|S|S)zGiven a sentence, apply optional lowercasing. :param sent: The input sentence string. :return: The pre-processed output string. )rlower)r'rLr*r*r+_preprocess_segmentszCHRF._preprocess_segment statisticscCs6d}d}d}|jd}d\}}t|jD]Q}|d|d|d\} } } | dkr-| | n|} | dkr7| | n|} || | }||dkrNd|| | |n|7}| dkre| dkre|| 7}|| 7}|d7}q|jrpd||jS|dkryd}}n||}||}||rd|||}||||}d|SdS) aCompute the chrF score given the n-gram match statistics. :param statistics: A flattened list of 3 * (`char_order` + `word_order`) elements giving the [hyp, ref, match] counts for each order. :return: The final f_beta score between [0, 100]. gؗҜ<rr)rXrXr d)r4ranger=r)r'rWepsr3effective_orderfactoravg_precavg_recin_hypn_refn_matchprecrecdenomr*r*r+_compute_f_scores6  $ zCHRF._compute_f_scorestatscCst|||j|j|jS)zComputes the final score from already aggregated statistics. :param stats: A list or numpy array of segment-level statistics. :return: A `CHRFScore` object. )r2rhrr r4r'rir*r*r+_compute_score_from_statss zCHRF._compute_score_from_statscCs|t|S)zComputes the final score given the pre-computed corpus statistics. :param stats: A list of segment-level statistics :return: A `CHRFScore` object. )rkrrjr*r*r+_aggregate_and_computeszCHRF._aggregate_and_computerefscCshg}|D]+}t||j|j}|jdkr*||}t|jD] }|t||dq||qd|iS)zGiven a list of reference segments, extract the character and word n-grams. :param refs: A sequence of reference segments. :return: A list where each element contains n-grams per reference segment. rr rA)r rr!r rTr[rPr)r'rmngramsrefri ref_wordsnr*r*r+_extract_reference_infos   zCHRF._extract_reference_info hypothesis ref_kwargsc sg}d}t||j|j}|jdkr*||td|jd}|fdd|D|dD]$}g}t||D] \} } ||| | q7| |} | |krR| }|}q.|S)aGiven a (pre-processed) hypothesis sentence and already computed reference n-grams, returns the best match statistics across the references. :param hypothesis: Hypothesis sentence. :param ref_kwargs: A dictionary with key `ref_ngrams` which is a list where each sublist contains n-gram counters for a particular reference sentence. :return: A list of integers where each triplet denotes [hyp, ref, match] statistics. grr csg|]}t|qSr*)r).0rqhwordsr*r+ sz4CHRF._compute_segment_statistics..rA) r rr!r rTr[extendziprKrh) r'rsrt best_stats best_f_scoreall_hyp_ngrams_range _ref_ngramsrihrf_scorer*rvr+_compute_segment_statisticss&      z CHRF._compute_segment_statistics) r,r-r.r/ CHAR_ORDER WORD_ORDERBETAsetrQr_SIGNATURE_TYPEr9boolrrstrr# staticmethodrrrKrTrVr8rhr2rkrlrrrrr1r*r*r(r+r:8sT- &r:N)r/typingrrrr collectionsrutilsrbaser r r helpersr rrr2r:r*r*r*r+s