o wi7 @s ddlmZmZmZmZddlZddlZddlZddlm Z ddl m Z ddl m Z ddlmZddlmZgdZd d Zdd eed eedefddZ dd eed eedeeeeeeffddZdd eed eedeeeeffddZGddde ZdS))ListOptionalTupleUnionN)Metric)AbstractCTCDecoding)AbstractMultiTaskDecoding)AbstractRNNTDecoding)logging)word_error_rateword_error_rate_detailWERcCs6tt|j}|j|g|d|||ddS)N)listrangendimpermute)tensor dim_indexall_dimsr]/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/nemo/collections/asr/metrics/wer.pymove_dimension_to_the_fronts(rF hypotheses referencesreturnc Csd}d}t|t|krtdt|t|t||D]%\}}|r,t|}t|}n|}|}|t|7}|t||7}q|dkrOd||} | Std} | S)au Computes Average Word Error rate between two texts represented as corresponding lists of string. Hypotheses and references must have same length. Args: hypotheses (list): list of hypotheses references(list) : list of references use_cer (bool): set True to enable cer Returns: wer (float): average word error rate rIn word error rate calculation, hypotheses and reference lists must have the same number of elements. But I got:{0} and {1} correspondingly?inf) len ValueErrorformatziprsplit editdistanceevalfloat) rruse_cerscoreswordshrh_listr_listwerrrrr #s(   r cCsd}d}dddd}t|t|krtdt|t|t||D]x\}}|r2t|}t|} n|}|} t| dkrVt|dkrSt|} |d| 7<n>d} n;|ratj||dd} nt||} | d| d| d} |d| d7<|d| d7<|d| d7<|| 7}|t| 7}q#|dkrd ||} d |d|} d |d|}d |d|}nt d t d t d t d f\} } }}| || ||fS) a Computes Average Word Error Rate with details (insertion rate, deletion rate, substitution rate) between two texts represented as corresponding lists of string. Hypotheses and references must have same length. Args: hypotheses (list): list of hypotheses references(list) : list of references use_cer (bool): set True to enable cer Returns: wer (float): average word error rate words (int): Total number of words/charactors of given reference texts ins_rate (float): average insertion error rate del_rate (float): average deletion error rate sub_rate (float): average substitution error rate r) substitutions insertions deletionsrr0T return_dictr1r/rr) rr r!r"rr#jiwercercompute_measuresr&)rrr'r(r) ops_countr*r+r,r-errorsmeasuresr.ins_ratedel_ratesub_raterrrr LsF      $r cCs2d}d}g}t|t|krtdt|t|t||D]c\}}|r.t|}t|} n|}|} t| dkrNt|dkrMt|} |tdn*|r]tj ||dd} | d} n t ||} | d} | d| d | d } || || 7}|t| 7}q|dkrd ||} || fStd} || fS) a Computes Word Error Rate per utterance and the average WER between two texts represented as corresponding lists of string. Hypotheses and references must have same length. Args: hypotheses (list): list of hypotheses references(list) : list of references use_cer (bool): set True to enable cer Returns: wer_per_utt (List[float]): word error rate per utterance avg_wer (float): average word error rate rrrTr2r5r.r0r1r/r) rr r!r"rr#appendr&r4r5r6)rrr'r(r) wer_per_uttr*r+r,r-r8r9eravg_werrrrword_error_rate_per_uttsB       rAcseZdZUdZdZeed<      ddeee e fffdd Z dd e j d e j d e j d e j dee j dee j f ddZddZZS)r a This metric computes numerator and denominator for Overall Word Error Rate (WER) between prediction and reference texts. When doing distributed training/evaluation the result of ``res=WER(predictions, predictions_lengths, targets, target_lengths)`` calls will be all-reduced between all workers using SUM operations. Here ``res`` contains three numbers ``res=[wer, total_levenstein_distance, total_number_of_words]``. If used with PytorchLightning LightningModule, include wer_numerator and wer_denominators inside validation_step results. Then aggregate (sum) then at the end of validation epoch to correctly compute validation WER. Example: def validation_step(self, batch, batch_idx): ... wer_num, wer_denom = self.__wer(predictions, predictions_len, transcript, transcript_len) self.val_outputs = {'val_loss': loss_value, 'val_wer_num': wer_num, 'val_wer_denom': wer_denom} return self.val_outputs def on_validation_epoch_end(self): ... wer_num = torch.stack([x['val_wer_num'] for x in self.val_outputs]).sum() wer_denom = torch.stack([x['val_wer_denom'] for x in self.val_outputs]).sum() tensorboard_logs = {'validation_loss': val_loss_mean, 'validation_avg_wer': wer_num / wer_denom} self.val_outputs.clear() # free memory return {'val_loss': val_loss_mean, 'log': tensorboard_logs} Args: decoding: An instance of CTCDecoding or RNNTDecoding. use_cer: Whether to use Character Error Rate instead of Word Error Rate. log_prediction: Whether to log a single decoded sample per call. batch_dim_index: Index corresponding to batch dimension. (For RNNT.) dist_dync_on_step: Whether to perform reduction on forward pass of metric. Returns: res: a tuple of 3 zero dimensional float32 ``torch.Tensor` objects: a WER score, a sum of Levenstein's distances for all prediction - reference pairs, total number of words in all references. Tfull_state_updateFrdecodingc  stj||d|_|_|_|_|_d_tjt r(fdd_n&tjt r6fdd_ntjt rDfdd_n t dt jjdtdd d d jd tdd d d dS) N)dist_sync_on_stepsync_on_computecsjj||dS)N)encoder_outputencoded_lengths)rCrnnt_decoder_predictions_tensor predictionspredictions_lengthspredictions_mask input_idsselfrrszWER.__init__..csjj||jdS)N)decoder_outputsdecoder_lengthsfold_consecutive)rCctc_decoder_predictions_tensorrSrIrNrrrPs csjj|||ddS)NF)encoder_hidden_statesencoder_input_maskdecoder_input_idsreturn_hypotheses)rCdecode_predictions_tensor)rJprediction_lengthsrLrMrNrrrPs z-WER metric does not support decoding of type r(rsumF)defaultdist_reduce_fx persistentr))super__init__rCr'log_predictionrSbatch_dim_indexdecode isinstancer rr TypeErrortype add_statetorchr) rOrCr'rarSrbrDrEkwargs __class__rNrr`s     z WER.__init__NrJrKtargetstargets_lengthsrLrMcKsd}d} g} tV|} |} |jdkr"t| |j} t| jdD]} | | }| | d| }|j |}| |q)|dkrW|||||ng}Wdn1scwY|r|jrtdtd| dtd|djt|| D]1\}}t|tr|d}|jrt|j}t|}n |j}|}|t|7}| t||7} qtj| |jj|jjd|_tj||j j|j jd|_ dS)a} Updates metric state. Args: predictions: an integer torch.Tensor of shape ``[Batch, Time, {Vocabulary}]`` (if ``batch_dim_index == 0``) or ``[Time, Batch]`` (if ``batch_dim_index == 1``) prediction_lengths: an integer torch.Tensor of shape ``[Batch]`` targets: an integer torch.Tensor of shape ``[Batch, Time]`` (if ``batch_dim_index == 0``) or ``[Time, Batch]`` (if ``batch_dim_index == 1``) target_lengths: an integer torch.Tensor of shape ``[Batch]`` predictions_lengths: an integer torch.Tensor of shape ``[Batch]`` rN zWER reference:zWER predicted:)devicedtype)!rhno_gradlongcpurbrrshapeitemnumpytolistrCdecode_ids_to_strr=numelrcrar infotextr"rdrr'r#rr$r%rr(rorpr))rOrJrKrlrmrLrMrir)r(rtgt_lenths_cpu_tensortargets_cpu_tensorindtgt_lentarget referencerr*r+r,r-rrrupdate$sD                z WER.updatecCs*|j}|j}||||fS)N)r(detachr&r))rOr(r)rrrcomputedsz WER.compute)FTTrFT)NN)__name__ __module__ __qualname____doc__rBbool__annotations__rrr rr`rhTensorrrr __classcell__rrrjrr s8  $ 1 @r )F)typingrrrrr$r4rh torchmetricsr2nemo.collections.asr.parts.submodules.ctc_decodingr8nemo.collections.asr.parts.submodules.multitask_decodingr3nemo.collections.asr.parts.submodules.rnnt_decodingr nemo.utilsr __all__rstrr&r intr rAr rrrrs,      * ,H>