o }oi @s.ddlZddlmZdgZGdddeZdS)N)MetricSequencePerplexitycsBeZdZdZd fdd Zd dejdejfdd Zd d ZZ S)ra This class computes mean perplexity across the batches of sequences. You have to provide ``log_probs`` (float tensor of shape [batch_size x seq_length x vocab_size]) and ``labels`` (int tensor of shape [batch_size x seq_length] with values from the range [0, vocab_size-1]) to the :meth:`update` method. If some of the sequences are shorter than seq_length, you can also provide an optional argument ``mask`` (bool tensor of shape [batch_size x seq_length]) which masks out tokens not participating in perplexity computation. See :doc:`PyTorch Lightning Metrics` for the metric usage instructions. Args: dist_sync_on_step: Synchronize metric state across processes at each ``forward()`` before returning the value at the step. process_group: Specify the process group on which synchronization is called. default: ``None`` (which selects the entire world) dist_sync_fn: Callback that performs the allgather operation on the metric state. When ``None``, DDP will be used to perform the allgather. FNcsNtj|||d|jdtjdtjddd|jdtjdtjddddS) N)dist_sync_on_step process_group dist_sync_fnperplexities_sumg)dtypesum)defaultdist_reduce_fx num_sequencesr)super__init__ add_statetorchtensorfloat64int64)selfrrr __class__d/home/ubuntu/.local/lib/python3.10/site-packages/nemo/collections/nlp/metrics/sequence_perplexity.pyr,s  zSequencePerplexity.__init__ log_probslabelscCs|dur t|}|j|jur||j}|d|dd}||jdd |jdd}|}|j | 7_ |j |7_ dS)N)dim) r ones_likertogather unsqueezesqueezer expr numelr)rrrmasktarget_log_probs avg_neg_llpplrrrupdate6s   zSequencePerplexity.updatecCs|jdrdS|j|jS)z{ Returns perplexity across all workers and resets to 0 :attr:`perplexities_sum` and :attr:`num_sequences`. rN)r eqr)rrrrcomputeCs  zSequencePerplexity.compute)FNN)N) __name__ __module__ __qualname____doc__rrTensorr)r+ __classcell__rrrrrs   )r torchmetricsr__all__rrrrrs