o yi@sddlZddlmZmZmZmZmZmZmZddl Z ddl m Z ddl m Z ddl mZmZmZmZmZmZddlmZddlmZddlmZesMd gZGd d d eZdS) N)AnyDictListOptionalSequenceTupleUnion)Tensor)_load_tokenizer_and_model)$_ALLOWED_INFORMATION_MEASURE_LITERAL_get_dataloader_get_special_tokens_map_infolm_compute_infolm_update_InformationMeasure)Metric) dim_zero_cat)_TRANSFORMERS_AVAILABLEInfoLMcs"eZdZUdZdZdZeeed<eeed<eeed<eeed<    d$de e e j fde dededee dee dee e ejfdeedededededee efffdd Zde e ee fde e ee fdd fd d!Zde eeeefffd"d#ZZS)%ru Calculate `InfoLM`_ - i.e. calculate a distance/divergence between predicted and reference sentence discrete distribution using one of the following information measures: - `KL divergence`_ - `alpha divergence`_ - `beta divergence`_ - `AB divergence`_ - `Rényi divergence`_ - L1 distance - L2 distance - L-infinity distance - `Fisher-Rao distance`_ `InfoLM`_ is a family of untrained embedding-based metrics which addresses some famous flaws of standard string-based metrics thanks to the usage of pre-trained masked language models. This family of metrics is mainly designed for summarization and data-to-text tasks. The implementation of this metric is fully based HuggingFace ``transformers``' package. As input to ``forward`` and ``update`` the metric accepts the following input: - ``preds`` (:class:`~Sequence`): An iterable of hypothesis corpus - ``target`` (:class:`~Sequence`): An iterable of reference corpus As output of ``forward`` and ``compute`` the metric returns the following output: - ``infolm`` (:class:`~torch.Tensor`): If `return_sentence_level_score=True` return a tuple with a tensor with the corpus-level InfoLM score and a list of sentence-level InfoLM scores, else return a corpus-level InfoLM score Args: model_name_or_path: A name or a model path used to load ``transformers`` pretrained model. By default the `"bert-base-uncased"` model is used. temperature: A temperature for calibrating language modelling. For more information, please reference `InfoLM`_ paper. information_measure: A name of information measure to be used. Please use one of: ['kl_divergence', 'alpha_divergence', 'beta_divergence', 'ab_divergence', 'renyi_divergence', 'l1_distance', 'l2_distance', 'l_infinity_distance', 'fisher_rao_distance'] idf: An indication of whether normalization using inverse document frequencies should be used. alpha: Alpha parameter of the divergence used for alpha, AB and Rényi divergence measures. beta: Beta parameter of the divergence used for beta and AB divergence measures. device: A device to be used for calculation. max_length: A maximum length of input sequences. Sequences longer than ``max_length`` are to be trimmed. batch_size: A batch size used for model processing. num_threads: A number of threads to use for a dataloader. verbose: An indication of whether a progress bar to be displayed during the embeddings calculation. return_sentence_level_score: An indication whether a sentence-level InfoLM score to be returned. Example: >>> from torchmetrics.text.infolm import InfoLM >>> preds = ['he read the book because he was interested in world history'] >>> target = ['he was interested in world history because he read the book'] >>> infolm = InfoLM('google/bert_uncased_L-2_H-128_A-2', idf=False) >>> infolm(preds, target) tensor(-0.1784) FTpreds_input_idspreds_attention_masktarget_input_idstarget_attention_maskbert-base-uncased? kl_divergenceN@rmodel_name_or_path temperatureinformation_measureidfalphabetadevice max_length batch_size num_threadsverbosereturn_sentence_level_scorekwargsc  stjdi| ||_||_||_||_||_||_t |p d|_ | |_ | |_ | |_ | |_t||\|_|_t||||_|pE|jjj|_t|j|_|jdgdd|jdgdd|jdgdd|jdgdddS) Ncpurcat)dist_reduce_fxrrr)super__init__rrrr r!r"torchr#_devicer%r&r'r(r tokenizermodelrinformation_measure_clsconfigr$r special_tokens_map add_state)selfrrrr r!r"r#r$r%r&r'r(r) __class__r-L/home/ubuntu/.local/lib/python3.10/site-packages/torchmetrics/text/infolm.pyr/rs( zInfoLM.__init__predstargetreturncCsNt|||j|j\}}}}|j||j||j||j|dS)z*Update state with predictions and targets.N)rr2r$rappendrrr)r8r<r=rrrrr-r-r;updates    z InfoLM.updatec Cstt|jt|j|j|j|jd}tt|jt|j|j|j|jd}t |j |||j |j|j |j |j}|jr>||fS|S)zLCalculate selected information measure using the pre-trained language model.) input_idsattention_maskr r% num_workers)r rrrr r%r&rrrr3rr4r6r'r(mean)r8preds_dataloadertarget_dataloader info_lm_scorer-r-r;computes6 zInfoLM.compute) rrrTNNNNrrTF)__name__ __module__ __qualname____doc__is_differentiablehigher_is_betterrr __annotations__rstrosPathLikefloatr boolrr0r#intrrr/rr@rrH __classcell__r-r-r9r;r%sb E           .'& )rQtypingrrrrrrrr0r 4torchmetrics.functional.text.helper_embedding_metricr #torchmetrics.functional.text.infolmr r r rrrtorchmetrics.metricrtorchmetrics.utilities.datartorchmetrics.utilities.importsr__doctest_skip__rr-r-r-r;s $