o Gicx@sRddlmZddlmZddlmZddlZddlmZddlmZddl m Z ddl Z ddl Z ddl Z dPd ed ed efd dZdPd ed ed efddZd ed ejfddZdQd edeed dfddZddZeddddfdededed ed!ed"ed#ed efd$d%Z  dRd&ejd'ejd(ejd)eeeefd*eef d+d,Zd-ed.e jd/ejd efd0d1Z dSd-ed.e jd/e jd2ed ef d3d4Zd5ed6ed.e jd7ejd8ejd ef d9d:Z  ; ed?ed@ed ef dAdBZ  <  ; ed?ed@ed efdEdFZ dGe jdHejdIejdJedKedLedMed eejejejejffdNdOZ!dS)V)Optional)Tuple)UnionN)Fsa) index_select) SymbolTableFfsaopenfstreturncCs|jdks Jg}g}t|jddD]!\}}t|tjr+|jtjkr+| |qt|t j r6| |qt j |j|||dS)aConvert an Fsa to a string. This version prints out all integer labels and integer ragged labels on the same line as each arc, the same format accepted by Fsa.from_str(). Note: The returned string can be used to construct an Fsa with Fsa.from_str(), but you would need to know the names of the auxiliary labels and ragged labels. Args: openfst: Optional. If true, we negate the scores during the conversion. Returns: A string representation of the Fsa. Finclude_scores)r extra_labels ragged_labels)arcsnum_axessortednamed_tensor_attr isinstancetorchTensordtypeint32appendk2 RaggedTensor_k2 fsa_to_str)r r rrnamevaluer!z-1 )rrrintrrrshape row_splitscpuvaluestolistrjoin) r$r.r/ansr5beginendlabelslabelr!r!r"convert_aux_label_to_symbols*      z+to_dot..convert_aux_label_to_symbolLRz8.5,111Portraitz0.4z0.25)rankdirsizecenter orientationranksepnodesepr>circlebold14)r4stylefontsize doublecircler0)r graph_attrraux_labels_symzuε labels_symz.2f0./)r>N)graphviz Exceptionprintlenr4r%r$rrrrrr3rrstrr+setziprr7scoresr8nodeaddrPreplacerRgetrstripedge)r r*rWr$rr?rOdefault_node_attrfinal_state_attr final_statedotseeniarcweight src_state dst_stater> aux_labelr!r!r"to_dotms    +.       rpc sZt}D]}t|jdksJ||jqt|}t|}tfdd| D}|D]7}g}D] }|t ||q5t |dt j rNt |}nt |dtjsXJtjj|dd}t|||q/t} D]}|D] \}} | |qrql| D]+}|dkrqD]!}t ||d}|durt||rt |||ksJqt|||qq|S)a_Create an FsaVec from a list of FSAs We use the following rules to set the attributes of the output FsaVec: - For tensor attributes, we assume that all input FSAs have the same attribute name and the values are concatenated. - For non-tensor attributes, if any two of the input FSAs have the same attribute name, then we assume that their attribute values are equal and the output FSA will inherit the attribute. Args: fsas: A list of `Fsa`. Each element must be a single FSA. Returns: An instance of :class:`Fsa` that represents a FsaVec. r c3s"|] \}}D]}|VqqdSrVr!).0r_r fsasr!r" s  z!create_fsa_vec..r)axis propertiesN)listrZr4rrrcreate_fsa_vecrr\rgetattrrrrcatrrraggedsetattrnamed_non_tensor_attrr`r%) rtragged_arc_listr ragged_arcsfsa_vectensor_attr_namesrr7r non_tensor_attr_namesrrr!rsr"rysF       ryinfTgư>dab log_semiringbeamtreat_epsilons_speciallydeltanpathc Cst|j|j|||||S)aCheck if the Fsa `a` appears to be equivalent to `b` by randomly checking some symbol sequences in them. Caution: It works only on CPU. Args: a: One of the input FSA. It can be either a single FSA or an FsaVec. Must be top-sorted and on CPU. b: The other input FSA. It must have the same NumAxes() as a. Must be top-sorted and on CPU. log_semiring: The semiring to be used for all weight measurements; if false then we use 'max' on alternative paths; if true we use 'log-add'. beam: beam > 0 that affects pruning; the algorithm will only check paths within `beam` of the total score of the lattice (for tropical semiring, it's max weight over all paths from start state to final state; for log semiring, it's log-sum probs over all paths) in `a` or `b`. treat_epsilons_specially: We'll do `intersection` between generated path and a or b when check equivalence. Generally, if it's true, we will treat epsilons as epsilon when doing intersection; Otherwise, epsilons will just be treated as any other symbol. delta: Tolerance for path weights to check the equivalence. If abs(weights_a, weights_b) <= delta, we say the two paths are equivalent. npath: The number of paths will be generated to check the equivalence of `a` and `b` Returns: True if the Fsa `a` appears to be equivalent to `b` by randomly generating `npath` paths from one of them and then checking if the symbol sequence exists in the other one and if the total weight for that symbol sequence is the same in both FSAs. )ris_rand_equivalentr)rrrrrrrr!r!r"r5s0rrowscolsr7rD min_col_indexcCs|j|jkr dksJJ||kr!|ks$JJ|dur?t|ts/J||k}||}||}||}|durTtjt||g|||j|jdStjt||g||j|jdS)a1This is a utility function that creates a (torch) sparse matrix likely intended to represent posteriors. The likely usage is something like (for example):: post = k2.create_sparse(fsa.seqframe, fsa.phones, fsa.get_arc_post(True,True).exp(), min_col_index=1) (assuming `seqframe` and `phones` were integer-valued attributes of `fsa`). Args: rows: Row indexes of the sparse matrix (a torch.Tensor), which must have values >= 0; likely `fsa.seqframe`. Must have row_indexes.dim == 1. Will be converted to `dtype=torch.long` cols: Column indexes of the sparse matrix, with the same shape as `rows`. Will be converted to `dtype=torch.long` values: Values of the sparse matrix, likely of dtype float or double, with the same shape as `rows` and `cols`. size: Optional. If not None, it is assumed to be a tuple containing `(num_frames, highest_phone_plus_one)` min_col_index: If provided, before the sparse tensor is constructed we will filter out elements with `cols[i] < min_col_index`. Will likely be 0 or 1, if set. This is necessary if `col_indexes` may have values less than 0, or if you want to filter out 0 values (e.g. as representing blanks). Returns: Returns a torch.Tensor that is sparse with coo (coordinate) format, i.e. `layout=torch.sparse_coo` (which is actually the only sparse format that torch currently supports). rN)rDdevice requires_grad)rr) ndimnumelrr3rsparse_coo_tensorstackrr)rrr7rDr kept_indexesr!r!r" create_sparseis( ((rsrc dest_arcsarc_mapc Cst|}|jddD]?\}}t|tjr)t||}t|||d}t|||q t|t j s1J|j tj ks9J|j |ddd\}}t|||q |D] \}}t|||qNt j||j||S)aCreate an Fsa object, including autograd logic and propagating properties from the source FSA. This is intended to be called from unary functions on FSAs where the arc_map is a Tensor of int32 (i.e. not ragged). Args: src: The source Fsa, i.e. the arg to the unary function. dest_arcs: The raw output of the unary function, as output by whatever C++ algorithm we used. arc_map: A map from arcs in `dest_arcs` to the corresponding arc-index in `src`, or -1 if the arc had no source arc (e.g. added epsilon self-loops). Returns: Returns the resulting Fsa, with properties propagated appropriately, and autograd handled. Fr  default_valuerrvneed_value_indexes)rrrrrfloat get_fillerrr}rrrrindexr~autograd_utilsphantom_index_select_scoresr^) rrrdestrr filler new_valuerrr!r!r"fsa_from_unary_function_tensors"  r remove_fillerc Csnt|}|jddD]\}}|r`t|tjr`|jtjkr`||}|dkrM|}t tdr?||t t |j dk|dk<n||t |j dk|dk@<t jj|||d}t||||q t|tjry|jtjtjfvsqJt j||}nt|t jsJ|jtjksJ||}||jd}t|||q |D] \}}t|||qt j||j||S)aCreate an Fsa object, including autograd logic and propagating properties from the source FSA. This is intended to be called from unary functions on FSAs where the arc_map is an instance of k2.RaggedTensor (with dtype torch.int32). Args: src: The source Fsa, i.e. the arg to the unary function. dest_arcs: The raw output of the unary function, as output by whatever C++ algorithm we used. arc_map: A map from arcs in `dest_arcs` to the corresponding arc-index in `src`, or -1 if the arc had no source arc (e.g. :func:`remove_epsilon`). remove_filler: If true, for each attribute that is linear in `src` and ragged in the result, after turning it into a ragged tensor we will remove all items that are equal to the filler for that attribute (0 by default; see Fsa.get_filler()). Attribute values on final-arcs that are equal to -1 will also be treated as fillers and removed, if remove_filler==True. Returns: Returns the resulting Fsa, with properties propagated appropriately, and autograd handled. Fr r0 logical_andrr )rrrrrrrrcloner%whererr=rr|rr}remove_values_eqfloat32float64 index_and_sumr remove_axisrr~rphantom_index_and_sum_scoresr^) rrrrrrr rrr!r!r"fsa_from_unary_function_raggeds@      ra_fsab_fsa a_arc_map b_arc_mapc Cst|}|D]l\}}t||}t||rF|jtjkr$td|t ||} | jtjks1Jt |||dt | ||d} t ||| qt |tj rTt |||d} nt |tjs\J|jtjksdJ|j|ddd\} } t ||| q|D]>\}} t||st | tj rt||}t | ||d} nt | tjsJ| jtjksJ| j|ddd\} } t ||| qy|D] \}}t |||q|D]\}} t||st ||| q|S)aBCreate an Fsa object, including autograd logic and propagating properties from the source FSAs. This is intended to be called from binary functions on FSAs where the arc_map is a Tensor of int32 (i.e. not ragged). Caution: Only the attributes with dtype `torch.float32` will be merged, other kinds of attributes with the same name are discarded. Args: a_fsa: The source Fsa, i.e. the arg to the binary function. b_fsa: The other source Fsa. dest_arcs: The raw output of the binary function, as output by whatever C++ algorithm we used. a_arc_map: A map from arcs in `dest_arcs` to the corresponding arc-index in `a_fsa` or -1 if the arc had no source arc (e.g. added epsilon self-loops). a_arc_map: A map from arcs in `dest_arcs` to the corresponding arc-index in `b_fsa` or -1 if the arc had no source arc (e.g. added epsilon self-loops). Returns: Returns the resulting Fsa, with properties propagated appropriately, and autograd handled. zjWe don't support propagating two attributes with the same name that are not real-valued, in intersection: rrFr)rrrrr%rrrAttributeErrorrzrr}rrrrrrr~) rrrrrout_fsara_valuerb_valuer rrr!r!r"fsa_from_binary_function_tensor(sV             r2acyclic max_symbol min_num_arcs max_num_arcscCst||||}t|S)aGenerate a random Fsa. Args: acyclic: If true, generated Fsa will be acyclic. max_symbol: Maximum symbol on arcs. Generated arc symbols will be in range [-1,max_symbol], note -1 is kFinalSymbol; must be at least 0; min_num_arcs: Minimum number of arcs; must be at least 0. max_num_arcs: Maximum number of arcs; must be >= min_num_arcs. )r random_fsar)rrrr random_arcsr!r!r"rs r min_num_fsas max_num_fsascCst||||||}t|S)aGenerate a random FsaVec. Args: min_num_fsas: Minimum number of fsas we'll generated in the returned FsaVec; must be at least 1. max_num_fsas: Maximum number of fsas we'll generated in the returned FsaVec; must be >= min_num_fsas. acyclic: If true, generated Fsas will be acyclic. max_symbol: Maximum symbol on arcs. Generated arcs' symbols will be in range [-1,max_symbol], note -1 is kFinalSymbol; must be at least 0; min_num_arcs: Minimum number of arcs in each Fsa; must be at least 0. max_num_arcs: Maximum number of arcs in each Fsa; must be >= min_num_arcs. )rrandom_fsa_vecr)rrrrrrrr!r!r"rs rtokensr^countseos min_token max_token max_orderc Cst|||||||S)a For "query" sentences, this function gets the mean and variance of scores from the best matching words-in-context in a set of provided "key" sentences. This matching process matches the word and the words preceding it, looking for the highest-order match it can find (it's intended for approximating the scores of models that see only left-context, like language models). The intended application is in estimating the scores of hypothesized transcripts, when we have actually computed the scores for only a subset of the hypotheses. CAUTION: This function only runs on CPU for now. Args: tokens: A ragged tensor of int32_t with 2 or 3 axes. If 2 axes, this represents a collection of key and query sequences. If 3 axes, this represents a set of such collections. 2-axis example:: [ [ the, cat, said, eos ], [ the, cat, fed, eos ] ] 3-axis example:: [ [ [ the, cat, said, eos ], [ the, cat, fed, eos ] ], [ [ hi, my, name, is, eos ], [ bye, my, name, is, eos ] ], ... ] where the words would actually be represented as integers, The eos symbol is required if this code is to work as intended (otherwise this code will not be able to recognize when we have reached the beginnings of sentences when comparing histories). bos symbols are allowed but not required. scores: A one dim torch.tensor with scores.size() == tokens.NumElements(), this is the item for which we are requesting best-matching values (as means and variances in case there are multiple best matches). In our anticipated use, these would represent scores of words in the sentences, but they could represent anything. counts: An one dim torch.tensor with counts.size() == tokens.NumElements(), containing 1 for words that are considered "keys" and 0 for words that are considered "queries". Typically some entire sentences will be keys and others will be queries. eos: The value of the eos (end of sentence) symbol; internally, this is used as an extra padding value before the first sentence in each collection, so that it can act like a "bos" symbol. min_token: The lowest possible token value, including the bos symbol (e.g., might be -1). max_token: The maximum possible token value. Be careful not to set this too large the implementation contains a part which takes time and space O(max_token - min_token). max_order: The maximum n-gram order to ever return in the `ngram_order` output; the output will be the minimum of max_order and the actual order matched; or max_order if we matched all the way to the beginning of both sentences. The main reason this is needed is that we need a finite number to return at the beginning of sentences. Returns: Returns a tuple of four torch.tensor (mean, var, counts_out, ngram_order) mean: For query positions, will contain the mean of the scores at the best matching key positions, or zero if that is undefined because there are no key positions at all. For key positions, you can treat the output as being undefined (actually they are treated the same as queries, but won't match with only themselves because we don't match at singleton intervals). var: Like `mean`, but contains the (centered) variance of the best matching positions. counts_out: The number of key positions that contributed to the `mean` and `var` statistics. This should only be zero if `counts` was all zero. ngram_order: The n-gram order corresponding to the best matching positions found at each query position, up to a maximum of `max_order`; will be `max_order` if we matched all the way to the beginning of a sentence. )rget_best_matching_stats)rr^rrrrrr!r!r"rsYr)FrV)NN)T)Trrr)rrTrrr)"typingrrrrr ropsr symbol_tablerr k2.raggedrboolr[r#r'rr)rpryrr3rr RaggedArcrrrrrrrr!r!r!r"s       > 7  > / U \