o yi@slddlmZmZmZmZmZmZddlZddlmZm Z ddl m Z m Z m Z ddlmZGdddeZdS))AnyListOptionalSequenceTupleUnionN)Tensortensor) _ter_compute _ter_update_TercomTokenizer)Metricc seZdZUdZdZeed<dZeed<dZeed<e ed<e ed<dZ e e e ed <    dd ed ed ededede f fdd ZdeeeefdeeeeefddfddZdee ee e fffddZZS)TranslationEditRatea5Calculate Translation edit rate (`TER`_) of machine translated text with one or more references. This implementation follows the one from `SacreBleu_ter`_, which is a near-exact reimplementation of the Tercom algorithm, produces identical results on all "sane" outputs. As input to ``forward`` and ``update`` the metric accepts the following input: - ``preds`` (:class:`~Sequence`): An iterable of hypothesis corpus - ``target`` (:class:`~Sequence`): An iterable of iterables of reference corpus As output of ``forward`` and ``compute`` the metric returns the following output: - ``ter`` (:class:`~torch.Tensor`): if ``return_sentence_level_score=True`` return a corpus-level translation edit rate with a list of sentence-level translation_edit_rate, else return a corpus-level translation edit rate Args: normalize: An indication whether a general tokenization to be applied. no_punctuation: An indication whteher a punctuation to be removed from the sentences. lowercase: An indication whether to enable case-insesitivity. asian_support: An indication whether asian characters to be processed. return_sentence_level_score: An indication whether a sentence-level TER to be returned. kwargs: Additional keyword arguments, see :ref:`Metric kwargs` for more info. Example: >>> preds = ['the cat is on the mat'] >>> target = [['there is a cat on the mat', 'a cat is on the mat']] >>> ter = TranslationEditRate() >>> ter(preds, target) tensor(0.1538) Fis_differentiablehigher_is_betterfull_state_updatetotal_num_edits total_tgt_lenN sentence_terT normalizeno_punctuation lowercase asian_supportreturn_sentence_level_scorekwargsc stjd i|t|tstd|dt|ts#td|dt|ts0td|dt|ts=td|dt|||||_||_|jdt ddd |jd t ddd |jri|jd gd d dSdS)Nzs