o wi@sfddlZddlZddlmZmZmZmZmZmZm Z m Z ddl Z e dZ ddee dee de deee eefffd d Z ddee dee d e de deee e ff d d Z   d deee deee deeee dededef ddZdededeeeffddZ d!deee deee deee deeee dffddZdZdS)"N)DictIterableListOptionalSequenceTupleTypeVarUnionSymbolFrefhyp sclite_modereturnc Cstttt|t|B}dd|D}g}g}|D] }|||q|D] }|||q)t|||}t||d<z |dt||d<W|St yj|ddkrad|d<Y|St d|d<Y|Sw) at Compute the edit distance between sequences ``ref`` and ``hyp``. Both sequences can be strings or lists of strings or ints. Optional ``sclite_mode`` sets INS/DEL/SUB costs to 3/3/4 for compatibility with sclite tool. Returns a dict with keys: * ``ins`` -- the number of insertions (in ``hyp`` vs ``ref``) * ``del`` -- the number of deletions (in ``hyp`` vs ``ref``) * ``sub`` -- the number of substitutions * ``total`` -- total number of errors * ``ref_len`` -- the number of symbols in ``ref`` * ``err_rate`` -- the error rate (total number of errors divided by ``ref_len``) cSi|]\}}||qSr.0kvrrP/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/kaldialign/__init__.py z!edit_distance..ref_lentotalerr_raterginf) dict enumeratesortedsetitemsappend _kaldialign edit_distancelenZeroDivisionErrorfloat) r r r int2symsym2intrefihypisymansrrrr# s(   r# eps_symbolc Cstttt|t|B|hB}dd|D}g}g}|D] }|||q |D] }|||q,||} t||| |} g} tt | D]} | || | d|| | dfqJ| S)a Compute the alignment between sequences ``ref`` and ``hyp``. Both sequences can be strings or lists of strings or ints. ``eps_symbol`` is used as a blank symbol to indicate insertion or deletion. Optional ``sclite_mode`` sets INS/DEL/SUB costs to 3/3/4 for compatibility with sclite tool. Returns a list of pairs of alignment symbols. The presence of ``eps_symbol`` in the first pair index indicates insertion, and in the second pair index, deletion. Mismatched symbols indicate substitution. cSrrrrrrrrGrzalign..r) rrrrr r!r"alignranger$) r r r-r r'r(aibir+eps_int alignmentaliidxrrrr/3s"(r/'refshypshyps2 replicationsseedcCs.ddlm}m}m}t|t|ks!Jdt|dt|d|dks)Jd|dks1Jdt|ts;t|tr?Jdt|||\}}}|||}||||d \} } t| | } |d ura| St|t|ksxJdt|dt|d |||} || ||d \} }||| ||d }| t| ||d S) aq Compute a boostrapping of WER to extract the 95% confidence interval (CI) using the bootstrap method of Bisani and Ney [1]. The implementation is based on Kaldi's ``compute-wer-bootci`` script [2]. Args: refs: A list of reference sequences (str, list[str], list[list[[int]]) hyps: A list of hypothesis sequences from system1 (str, list[str], list[list[int]]) hyps2: A list of hypothesis sequences from system2 (str, list[str], list[list[int]]). When provided, we'll compute CI for both systems as well as the probability of system2 improving over system1. replications: The number of replications to use for bootstrapping. seed: The random seed to reproduce the results. Returns: A dict with results. When scoring a single system (``hyp2_seqs=None``), the keys are: - "wer" (mean WER estimate), - "ci95" (95% confidence interval size), - "ci95min" (95% confidence interval lower bound) - "ci95max" (95% confidence interval upper bound) When scoring two systems, the keys are "system1", "system2", and "p_s2_improv_over_s1". The first two keys contain dicts as described for the single-system case, and the last key's value is a float in the range [0, 1]. [1] Bisani, M., & Ney, H. (2004, May). Bootstrap estimates for confidence intervals in ASR performance evaluation. In 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing (Vol. 1, pp. I-409). IEEE. [2] https://github.com/kaldi-asr/kaldi/blob/master/src/bin/compute-wer-bootci.cc r)_get_boostrap_wer_interval _get_edits _get_p_improvz"Inconsistent number of reference (z) and hypothesis (z ) sequences.z2The number of replications must be greater than 0.zThe seed must be 0 or greater.z=The input must be a list of strings or list of lists of ints.)r;r<Nz.) sequences for the second system (hyp2_seqs).)system1system2p_s2_improv_over_s1) r"r=r>r?r$ isinstancestr_convert_to_int_build_results)r8r9r:r;r<r=r>r?edit_sym_per_hypmeanintervalans1edit_sym_per_hyp2mean2 interval2p_improvrrrbootstrap_wer_ci\sJ$     rOrHrIcCs||||||dS)N)werci95ci95minci95maxr)rHrIrrrrFs rFhyp2.csz||g}|dur ||ttdd|D}tt|}dd|Dfdd|D}|dur9|dt|S)Ncss(|]}|D] }|D]}|Vq qqdSNr)rsourceseqsymbolrrr s&z"_convert_to_int..cSrrrrrrrrrz#_convert_to_int..cg|] }fdd|DqS)crZ)csg|]}|qSrr)ritemr(rr sz9_convert_to_int....r)rrWr\rrr]z._convert_to_int...r)rrVr\rrr]r^z#_convert_to_int..)r!rrrrr tuple)r r rTsourcessymbolsr'intsrr\rrEs   rEz0.9.3)F)Nr7rrU)mathrandomtypingrrrrrrrr r"r boolrDintr&r#r/rOrFrE __version__rrrrsn( - ,   J