o .wi( @sddlZddlmZmZddlmZmZmZmZddl Z ddl m Z m Z ddl m Z ddlmZmZmZddlmZddlmZmZesGd gZd Zd Zd Zeeee feee feee feee feee feee ffZGd dde ZdS)N)IteratorSequence)AnyListOptionalUnion)Tensortensor)Metric)_chrf_score_compute_chrf_score_update_prepare_n_grams_dicts)_MATPLOTLIB_AVAILABLE)_AX_TYPE_PLOT_OUT_TYPECHRFScore.plot)charword)predstargetmatching)total_preds_char_n_gramstotal_preds_word_n_gramstotal_target_char_n_gramstotal_target_word_n_gramstotal_matching_char_n_gramstotal_matching_word_n_gramscseZdZUdZdZeed<dZeed<dZeed<dZ e ed<d Z e ed <d Z e eeed <     d2dedede dededededd ffdd Zdeedeeedd fddZdeeeeefffddZdefd d!Zd"edd fd#d$Zed%ed&edefd'd(Zed%ed&ed)edefd*d+Zdeeeeefeffd,d-Z d3d.e eeeefd/e e!de"fd0d1Z#Z$S)4 CHRFScoreaCalculate `chrf score`_ of machine translated text with one or more references. This implementation supports both ChrF score computation introduced in `chrF score`_ and `chrF++ score`_ introduced in `chrF++ score`_. This implementation follows the implementations from https://github.com/m-popovic/chrF and https://github.com/mjpost/sacrebleu/blob/master/sacrebleu/metrics/chrf.py. As input to ``forward`` and ``update`` the metric accepts the following input: - ``preds`` (:class:`~Sequence`): An iterable of hypothesis corpus - ``target`` (:class:`~Sequence`): An iterable of iterables of reference corpus As output of ``forward`` and ``compute`` the metric returns the following output: - ``chrf`` (:class:`~torch.Tensor`): If `return_sentence_level_score=True` return a list of sentence-level chrF/chrF++ scores, else return a corpus-level chrF/chrF++ score Args: n_char_order: A character n-gram order. If ``n_char_order=6``, the metrics refers to the official chrF/chrF++. n_word_order: A word n-gram order. If ``n_word_order=2``, the metric refers to the official chrF++. If ``n_word_order=0``, the metric is equivalent to the original ChrF. beta: parameter determining an importance of recall w.r.t. precision. If ``beta=1``, their importance is equal. lowercase: An indication whether to enable case-insensitivity. whitespace: An indication whether keep whitespaces during n-gram extraction. return_sentence_level_score: An indication whether a sentence-level chrF/chrF++ score to be returned. kwargs: Additional keyword arguments, see :ref:`Metric kwargs` for more info. Raises: ValueError: If ``n_char_order`` is not an integer greater than or equal to 1. ValueError: If ``n_word_order`` is not an integer greater than or equal to 0. ValueError: If ``beta`` is smaller than 0. Example: >>> from torchmetrics.text import CHRFScore >>> preds = ['the cat is on the mat'] >>> target = [['there is a cat on the mat', 'a cat is on the mat']] >>> chrf = CHRFScore() >>> chrf(preds, target) tensor(0.8640) Fis_differentiableThigher_is_betterfull_state_updateplot_lower_boundg?plot_upper_boundNsentence_chrf_score@ n_char_order n_word_orderbeta lowercase whitespacereturn_sentence_level_scorekwargsreturnc  stjd i|t|tr|dkrtd||_t|tr"|dkr&td||_|dkr1td||_||_||_ ||_ t |||_ | D]!\\}} } td| dD]} || || } |j| tdddqUqH|j rw|jd gd ddSdS) NzMExpected argument `n_char_order` to be an integer greater than or equal to 1.rzMExpected argument `n_word_order` to be an integer greater than or equal to 0.z.Expected argument `beta` to be greater than 0.r!sum)dist_reduce_fxr$cat)super__init__ isinstanceint ValueErrorr(r)r*r+r,r-floatn_order_get_text_n_gram_iteratorrange_get_state_name add_stater ) selfr(r)r*r+r,r-r. n_gram_level n_gram_ordertextn state_name __class__r4S/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/torchmetrics/text/chrf.pyr6js, zCHRFScore.__init__rrcCsxt||g||j|j|j|j|j|j|jr |j ndR}| |dd|j dur:|d|_ dSdS)z*Update state with predictions and targets.N) r _convert_states_to_dictsr(r)r;r*r+r,r-r$_update_states_from_dicts)r@rrn_grams_dicts_tupler4r4rHupdates0   zCHRFScore.updatecCsR|jdurtg||j|jRt|jfStg||j|jRS)zCalculate chrF/chrF++ score.N)r$r rJr;r*torchr3r@r4r4rHcomputes  zCHRFScore.computecCsztttt|j|j}|D]&\\}}}td|dD]}|||}| |||}t |||||<qqt | S)z_Convert global metric states to the n-gram dictionaries to be passed in ``_chrf_score_update``.r0) dictzip_DICT_STATES_NAMESr r(r)r<r=_get_dict_namer>getattrtuplevalues)r@ n_grams_dictsrArBrCrD dict_namerEr4r4rHrJs  z"CHRFScore._convert_states_to_dictsrLc Csjttt|}|D]'\\}}}td|dD]}|||}||||}t|||||qq dS)z]Update global metric states based on the n-gram dictionaries calculated on the current batch.r0N)rQrRrSr<r=rTr>setattr) r@rLrXrArBrCrDrYrEr4r4rHrKs z#CHRFScore._update_states_from_dictsrCrAcCsd|d|dS)z*Return a dictionary name w.r.t input args.total___n_gramsr4)rCrAr4r4rHrTszCHRFScore._get_dict_namerDcCsd|d|d|dS)z,Return a metric state name w.r.t input args.r[r\_gramsr4)rCrArDr4r4rHr>szCHRFScore._get_state_namecCsttt|j|jgtS)zKGet iterator over char/word and reference/hypothesis/matching n-gram level.) itertoolsproductrR_N_GRAM_LEVELSr(r) _TEXT_LEVELSrOr4r4rHr<sz#CHRFScore._get_text_n_gram_iteratorvalaxcCs |||S)aPlot a single or multiple values from the metric. Args: val: Either a single result from calling `metric.forward` or `metric.compute` or a list of these results. If no value is provided, will automatically call `metric.compute` and plot that result. ax: An matplotlib axis object. If provided will add plot to that axis Returns: Figure and Axes object Raises: ModuleNotFoundError: If `matplotlib` is not installed .. plot:: :scale: 75 >>> # Example plotting a single value >>> from torchmetrics.text import CHRFScore >>> metric = CHRFScore() >>> preds = ['the cat is on the mat'] >>> target = [['there is a cat on the mat', 'a cat is on the mat']] >>> metric.update(preds, target) >>> fig_, ax_ = metric.plot() .. plot:: :scale: 75 >>> # Example plotting multiple values >>> from torchmetrics.text import CHRFScore >>> metric = CHRFScore() >>> preds = ['the cat is on the mat'] >>> target = [['there is a cat on the mat', 'a cat is on the mat']] >>> values = [ ] >>> for _ in range(10): ... values.append(metric(preds, target)) >>> fig_, ax_ = metric.plot(values) )_plot)r@rcrdr4r4rHplots *r)r%r&r'FFF)NN)%__name__ __module__ __qualname____doc__rbool__annotations__rr r"r:r#r$rrrr8rr6rstrrMrrVrP_DICT_STATES_TYPESrJrK staticmethodrTr>rr<rrrf __classcell__r4r4rFrHr5s`  ,     "$  "r) r_collections.abcrrtypingrrrrrNrr torchmetricsr !torchmetrics.functional.text.chrfr r r torchmetrics.utilities.importsrtorchmetrics.utilities.plotrr__doctest_skip__rarbrSrVrQr8rnrr4r4r4rHs$   >