o 5tiD @sdZddlZddlZddlmZddlmZmZmZm Z m Z ddl m Z m Z ddlmZmZmZdd lmZed Zd Zd d dddddddddd ZdefddZGdddeZGdddeZGdd d eZdS)!z>The implementation of the BLEU metric (Papineni et al., 2002).N) import_module)ListSequenceOptionalDictAny)my_log sum_of_lists)Score SignatureMetric)extract_all_word_ngrams sacrebleuztokenizer_none.NoneTokenizerztokenizer_zh.TokenizerZhztokenizer_13a.Tokenizer13az(tokenizer_intl.TokenizerV14Internationalztokenizer_char.TokenizerCharz#tokenizer_ja_mecab.TokenizerJaMecabz#tokenizer_ko_mecab.TokenizerKoMecabztokenizer_spm.TokenizerSPMz tokenizer_spm.Flores101Tokenizerz tokenizer_spm.Flores200Tokenizerztokenizer_spm.spBLEU1KTokenizer) nonezh13aintlcharja-mecabko-mecabspm flores101 flores200z spBLEU-1KnamecCs*t|dd\}}ttd|d|S)z6Dynamically import tokenizer as importing all is slow..r z .tokenizers.r) _TOKENIZERSrsplitgetattrr)r module_name class_namer#J/home/ubuntu/.local/lib/python3.10/site-packages/sacrebleu/metrics/bleu.py_get_tokenizer"s r%cs&eZdZdZdeffdd ZZS) BLEUSignaturezA convenience class to represent the reproducibility signature for BLEU. :param args: key-value dictionary passed from the actual metric instance. argscst||jddddd|d}tj|}|dur1|d}|dur(|}|d |d d 7}|j|d r:d nd|drAdnd|d|ddS)z`BLEUSignature` initializer.cetoks)caseeffr*smooth smooth_methodN smooth_value[z.2f] lowercaselcmixedeffective_orderyesnotokenizer_signature)super__init___abbrupdateBLEUSMOOTH_DEFAULTSinfo)selfr' smooth_str smooth_def smooth_val __class__r#r$r;/s(   zBLEUSignature.__init__)__name__ __module__ __qualname____doc__dictr; __classcell__r#r#rEr$r&*sr&csJeZdZdZdedeedeedeedededeffd d ZZS) BLEUScoreaA convenience class to represent BLEU scores. :param score: The BLEU score. :param counts: List of counts of correct ngrams, 1 <= n <= max_ngram_order :param totals: List of counts of total ngrams, 1 <= n <= max_ngram_order :param precisions: List of precisions, 1 <= n <= max_ngram_order :param bp: The brevity penalty. :param sys_len: The cumulative system length. :param ref_len: The cumulative reference length. scorecountstotals precisionsbpsys_lenref_lencstd|||_||_||_||_||_||_ddd|jD|_ |jr.|j|jnd|_ |j d|jdd|_ |j d |j dd |jd d7_ |j d |jd d 7_ dS)z`BLEUScore` initializer.r>/cSsg|]}|dqS)z.1fr#.0pr#r#r$ gz&BLEUScore.__init__..rz (BP = z.3f zratio = z hyp_len = dz ref_len = )N) r:r;rRrOrPrSrTrQjoinprec_strratio_verbose)rArNrOrPrQrRrSrTrEr#r$r;[s$zBLEUScore.__init__) rGrHrIrJfloatrintr;rLr#r#rEr$rMPs rMcseZdZUdZdddddZeeeefe d<e Z dZ dd d d ZeZd d dd ded ddf dededeededeededededeeeeffdd Zeddd efdeedeededededededefd d!Zd"edefd#d$Zd%eedefd&d'Zd%eeedefd(d)Zd*ed+eedefd,d-Zd.eedeeeffd/d0Zd1ed2edeefd3d4Z d1edeedeffd5d6 Z!Z"S)7r>awComputes the BLEU metric given hypotheses and references. :param lowercase: If True, lowercased BLEU is computed. :param force: Ignore data that looks already tokenized. :param tokenize: The tokenizer to use. If None, defaults to language-specific tokenizers with '13a' as the fallback default. :param smooth_method: The smoothing method to use ('floor', 'add-k', 'exp' or 'none'). :param smooth_value: The smoothing value for `floor` and `add-k` methods. `None` falls back to default value. :param max_ngram_order: If given, it overrides the maximum n-gram order (default: 4) when computing precisions. :param effective_order: If `True`, stop including n-gram orders for which precision is 0. This should be `True`, if sentence-level BLEU will be computed. :param trg_lang: An optional language code to raise potential tokenizer warnings. :param references: A sequence of reference documents with document being defined as a sequence of reference strings. If given, the reference n-grams and lengths will be pre-computed and cached for faster BLEU computation across many systems. Ng?r )rflooradd-kexpr?rrrr)rjakoFrfr3forcetokenizer/r0max_ngram_orderr6trg_lang referencesc st||_||_||_||_||_||_||_|j|j vs&Jd|dur:|j } |j|j vr9|j |j} n,|} |jdkrJ| dkrJt d|jdkrX| dkrXt d|jdkrf| d krft d t| |_|j|_| dur~|| |_dSdS) z`BLEU` initializer.z,Unknown smooth_method {self.smooth_method!r}Nrz7Consider using the 'zh' or 'spm' tokenizer for Chinese.rgrz>Consider using the 'ja-mecab' or 'spm' tokenizer for Japanese.rhrz.) r>r?rpmathrfanyrMrangelensum)rzr{rSrTr/r0r6rlrRrQ smooth_mteval eff_ordernrNr#r#r$ compute_bleusB  &zBLEU.compute_bleusentcCs|jr|}||S)zGiven a sentence, lowercases (optionally) and tokenizes it :param sent: The input sentence string. :return: The pre-processed output string. )r3lowerrurstrip)rArr#r#r$_preprocess_segment'szBLEU._preprocess_segmentstatsc CsN|j|dd|j|d|jdt|dt|d|j|j|j|jdS)zComputes the final score from already aggregated statistics. :param stats: A list or numpy array of segment-level statistics. :return: A `BLEUScore` object. rNrr )rzr{rSrTr/r0r6rl)rrlrcr/r0r6rArr#r#r$_compute_score_from_stats0szBLEU._compute_score_from_statscCs|t|S)zComputes the final BLEU score given the pre-computed corpus statistics. :param stats: A list of segment-level statistics :return: A `BLEUScore` instance. )rr rr#r#r$_aggregate_and_compute?szBLEU._aggregate_and_computehyp_lenref_lenscCsPd\}}|D]}t||}|dks||kr|}|}q||kr%||kr%|}q|S)aGiven a hypothesis length and a list of reference lengths, returns the closest reference length to be used by BLEU. :param hyp_len: The hypothesis length. :param ref_lens: A list of reference lengths. :return: The closest reference length. )rr)abs)rArr closest_diff closest_lenrTdiffr#r#r$_get_closest_ref_lenGs zBLEU._get_closest_ref_lenrefsc Csjd}g}|D])}t|d|j\}}|||dur|}q|D] \}}t|||||<q!q||dS)aXGiven a list of reference segments, extract the n-grams and reference lengths. The latter will be useful when comparing hypothesis and reference lengths for BLEU. :param refs: A sequence of strings. :return: A dictionary that will be passed to `_compute_segment_statistics()` through keyword arguments. Nr ) ref_ngramsr)rrlappenditemsmax) rArngramsrref this_ngramsrTngramcountr#r#r$_extract_reference_info[s  zBLEU._extract_reference_info hypothesis ref_kwargsc Cs|d|d}}t|d|j\}}|||}ddt|jD}|dd} |D]#\} } t| d} | | | 7<| |vrO|| t| || 7<q,||g|| S)aGiven a (pre-processed) hypothesis sentence and already computed reference n-grams & lengths, returns the best match statistics across the references. :param hypothesis: Hypothesis sentence. :param ref_kwargs: A dictionary with `refs_ngrams`and `ref_lens` keys that denote the counter containing all n-gram counts and reference lengths, respectively. :return: A list of integers with match statistics. rrr cSsg|]}dqS)rr#)rWir#r#r$rYsz4BLEU._compute_segment_statistics..N)rrlrrrrmin) rArrrr hyp_ngramsrrTrzr{ hyp_ngram hyp_countrr#r#r$_compute_segment_statisticsvs    z BLEU._compute_segment_statisticscs|jstdt||S)zCompute the metric for a single sentence against a single (or multiple) reference(s). :param hypothesis: A single hypothesis string. :param references: A sequence of reference strings. :return: a `BLEUScore` object. zFIt is recommended to enable `effective_order` for sentence-level BLEU.)r6rsrtr:sentence_score)rArrnrEr#r$rs zBLEU.sentence_score)#rGrHrIrJr?rstrrrb__annotations__rrp TOKENIZERSrqrrr&_SIGNATURE_TYPEMAX_NGRAM_ORDERboolrcrr; staticmethodrrMrrrrrrrrrrLr#r#rEr$r>ps  8  R  &%r>)rJr}logging importlibrtypingrrrrrutilsr r baser r rhelpersr getLoggerrsrrrr%r&rMr>r#r#r#r$s4   &