o 5tiA@sdZddlZddlZddlZddlmZmZddlmZm Z m Z m Z m Z ddl mZedZGdd d ZGd d d ZGd d d edZdS)zThe base `Score`, `Metric` and `Signature` classes to derive from. `Metric` is an abstract class that enforces the implementation of a set of abstract methods. This way, a correctly implemented metric will work seamlessly with the rest of the codebase. N)ABCMetaabstractmethod)AnyDictListOptionalSequence) __version__ sacrebleuc @sdeZdZdZdedefddZ    dd ed ed ed ed ef ddZ de dfddZ ddZ dS)ScorezA base score class to derive from. :param name: The name of the underlying metric. :param score: A floating point number for the final metric. namescorecCs"||_||_d|_d|_d|_dS)z`Score` initializer.gN)r r_mean_ci_verbose)selfr rrJ/home/ubuntu/.local/lib/python3.10/site-packages/sacrebleu/metrics/base.py__init__s  zScore.__init__r Frwidth score_only signatureis_jsonreturncCs2|jt|jd|d|d}|jd|d}|jdkrR|jd|d}|jd|d}d|d|} |d| d7}|rRt||d <t||d <| |d <|r\|jd |n|j} | d |} |jrv| d|j7} |j|d<|rz|S|r|d D] } | d\} } | || <qtj|dddS| S)aReturns a pretty representation of the score. :param width: Floating point decimal precision width. :param score_only: If `True`, and the format is not `json`, returns a single score string. :param signature: A string representation of the given `Signature` instance. :param is_json: If `True`, will output the score in JSON string. :return: A plain or JSON-formatted string representation. .f)r rrruμ = u ± z ()confidence_meanconfidence_var confidence|z =  verbose_score:F)indent ensure_ascii) r floatrrrrsplitjsondumps)rrrrrdscrr confidence_str full_scoreparamkeyvaluerrrformat&s6     z Score.formatscorescCs^tdd|D}t|}|d}||d}||||}}d|||_t||_dS)zTakes a list of scores and stores mean, stdev and 95% confidence interval around the mean. :param scores: A list of `Score` objects obtained from bootstrap resampling for example. cSsg|]}|jqSr)r.0xrrr cz%Score.estimate_ci..(r&g?N)sortedlenr statisticsmeanr)rr5 raw_scoresn lower_idx upper_idxlowerupperrrr estimate_ci[s zScore.estimate_cicC|S)z&Returns a human readable score string.r4rrrr__repr__mzScore.__repr__N)r FrF) __name__ __module__ __qualname____doc__strr)rintboolr4rrFrJrrrrr s( 5 r c@sTeZdZdZdefddZddedefdd Zd ed e fd d Z ddZ ddZ dS) SignaturezA convenience class to represent sacreBLEU reproducibility signatures. :param args: key-value dictionary passed from the actual metric instance. argsc Csddddddddd d |_d |vrtd |d }|d krd}t||ddd|dd|dd|dd|dd|ddd |_dS)z`Signature` initializer.v#tlSobsarrs) versionnrefstestlangsubsetoriglangr[r\seednum_refsz?Number of references unknown, please evaluate the metric first.var n_bootstrapNrdtest_setlangpairrcrb) r^r_r[r\rdr`rarcrb)_abbr ValueErrorr getinfo)rrTrerrrrxs6        zSignature.__init__FshortrcCsg}t|j}|d|dgD])}|j|}|dursz2Metric._check_corpus_score_args..z$Undefined line in hypotheses stream!z3`refs` should be a sequence of sequence of strings.z7Each element of `refs` should be a sequence of strings.r)rrLrurrPanyr)rrrrrrrr_check_corpus_score_argss$   "zMetric._check_corpus_score_argsstatsrcCdS)zComputes the final score given the pre-computed match statistics. :param stats: A list of segment-level statistics. :return: A `Score` instance. Nrrrrrr_aggregate_and_compute zMetric._aggregate_and_computecCr)zComputes the final score from already aggregated statistics. :param stats: A list or numpy array of segment-level statistics. :return: A `Score` object. Nrrrrr_compute_score_from_statsrz Metric._compute_score_from_statssentcCr)zA wrapper around the metric's tokenization and pre-processing logic. This should be implemented for reference caching to work correctly. :param sent: The input sentence. :return: The pre-processed output sentence. Nr)rrrrr_preprocess_segmentzMetric._preprocess_segmentcCr)a Given a list of reference segments, extract the required information (such as n-grams for BLEU and chrF). This should be implemented for the generic `_cache_references()` to work across all metrics. :param refs: A sequence of strings. Nr)rrrrr_extract_reference_info)rzMetric._extract_reference_info hypothesis ref_kwargscCr)aGiven a (pre-processed) hypothesis sentence and already computed reference info, returns the best match statistics across the references. The return type is usually a List of ints or floats. :param hypothesis: A pre-processed hypothesis sentence. :param ref_kwargs: A dictionary with reference-related information within. This is formulated as a dictionary as different metrics may require different information regarding a reference segment. Nr)rrrrrr_compute_segment_statistics3s z"Metric._compute_segment_statistics referencescs~g}t}t|D]!}dd|D}|t|fdd|D}||q t|dkr:t|d_|Sd_|S)a&Given the full set of document references, extract segment n-grams (or other necessary information) for caching purposes. :param references: A sequence of reference documents with document being defined as a sequence of reference strings. A particular reference segment can be '' or `None` to allow the use of variable number of references per segment. :return: A list where each element is a tuple of segment n-grams and reference lengths, as returned by `_extract_reference_info()`. cSsg|]}|dur|qSrrr6rrrr9Tsz,Metric._cache_references..csg|]}|qSr)rr6rIrrr9Zsr&rrf)setzipaddr=rvrrrre)rr ref_cachererlinesrrIr_cache_referencesBs   zMetric._cache_references hypothesescCs|r||}n |jr|j}ntdg}d}t||D]\}}|js,|dr,|d7}||}||||q|dkrNt dt dt d|S) aReads the corpus and returns sentence-level match statistics for faster re-computations esp. during statistical tests. :param hypotheses: A sequence of hypothesis strings. :param references: A sequence of reference documents with document being defined as a sequence of reference strings. If `None`, cached references will be used. :return: A list where each sublist corresponds to segment statistics. z.No references provided and the cache is empty.rz .r&dz5That's 100 lines that end in a tokenized period ('.')zQIt looks like you forgot to detokenize your test data, which may hurt your score.zpIf you insist your data is detokenized, or don't care, you can suppress this message with the `force` parameter.) rr RuntimeErrorrrendswithrrvr sacreloggerwarning)rrrrr tok_countrrrrr_extract_corpus_statisticsgs*   z!Metric._extract_corpus_statisticscCs.|||||gdd|D}||S)zCompute the metric for a single sentence against a single (or multiple) reference(s). :param hypothesis: A single hypothesis string. :param references: A sequence of reference strings. :return: A `Score` object. cSsg|]}|gqSrr)r7rrrrr9r:z)Metric.sentence_score..)rrr)rrrrrrrsentence_scores  zMetric.sentence_scorer&rhcCs\||||||}||}|dkr,ddlm}||_||||\|_}|||S)aCompute the metric for a corpus against a single (or multiple) reference(s). :param hypotheses: A sequence of hypothesis strings. :param references: A sequence of reference documents with document being defined as a sequence of reference strings. If `None`, cached references will be used. :param n_bootstrap: If > 1, provides 95% confidence interval around true mean using bootstrap resampling with `n_bootstrap` samples. :return: A `Score` object. r&r )_bootstrap_resample)rrr significancerrhrdrF)rrrrhr actual_scorer bs_scoresrrr corpus_scores     zMetric.corpus_scorecCs ||jS)zCreates and returns the signature for the metric. The creation of signatures is delayed as the number of references is resolved only at the point of reference caching.)_SIGNATURE_TYPE__dict__rIrrr get_signatures zMetric.get_signatureN)r&)rLrMrNrOrSrrrPrrrrrrrrrrrrrrrrrQrrrrrrr~s\      % - $r~) metaclass)rOr+loggingr>abcrrtypingrrrrrr^r getLoggerrr rSr~rrrrs  _S