o }oi94@sddlmZddlmZmZmZmZddlZddlm Z ddl m Z ddl m Z ddlmZddlmZmZmZmZdd lmZdd lmZGd d d eZGd ddee ZdS))abstractmethod)AnyOptionalTupleUnionN) DictConfig)GraphIntersectDenseConfig) CtcK2Mixin)MLLoss)create_sparse_wrappedget_tot_objf_and_finite_maskinvert_permutation load_graph)k2)loggingcseZdZdZedddddedfdeded ed ee d ed e d ee defde dede ffdd Z ed&ddZ d'ddddddde deejejededff ddZ d'ddddddde deejejededff dd Zd!dd"ed#ejdeejejffd$d%ZZS)(MAPLossa Maximum a Posteriori Probability criterion. It implements Lattice-Free Maximum Mutual Information (LF-MMI) and LF-boosted-MMI (LF-bMMI) losses. Based on https://github.com/k2-fsa/snowfall/blob/master/snowfall/objectives/mmi.py cfg takes precedence over all optional parameters We keep explicit parameter setting to be able to create an instance without the need of a config. NdefaultTF num_classesblank reductioncfg topo_typetopo_with_self_loopstoken_lmk2.Fsaintersect_prunedintersect_conf boost_coeffc stj||||||d|dur(|d|}|d|}|d| } |d| } | |_|r0|jn|j|_| |_d|_|durEt ddSt |t rNt |n||_|jdurZtd||jdS)N)rrrrrrrrrrztoken_lm is empty. Trainable token_lm is not supported yet. Please call .update_graph(token_lm) before using.zlm_graph is empty.)super__init__getr"_intersect_calc_scores_impl_pruned%_intersect_calc_scores_impl_exact_opt_intersect_calc_scores_implrgraph_compilerrwarning isinstancestrrlm_graph ValueError update_graph selfrrrrrrrrrr __class__Z/home/ubuntu/.local/lib/python3.10/site-packages/nemo/collections/asr/parts/k2/map_loss.pyr ;s4     zMAPLoss.__init__graphcCst)N)NotImplementedError)r-r2r0r0r1r+iszMAPLoss.update_graph dense_fsa_veczk2.DenseFsaVec num_graphs den_graph return_latsreturncCsn|j}||jkr ||jksJ|jd}||ksJ|}|}t||g}tj|tjd}tj |g|tjd} t || g d |} t|| } tj|tjd dd} | dd d |} tj| ||jj| |rydndd} | jd d d }|ddd}|ddd}|rtj|tjd |d}||t| |t| |dfS||ddfS) zInner intersection method. Does joint (simultaneous) exact intersection of dense_fsa_vec against num_graphs and den_graph. Optiolally returns the numerator and the denominator lattices. rdtype seqframe_idxN)a_fsasb_fsas output_beam a_to_b_mapseqframe_idx_nameTF log_semiringuse_double_scores)deviceshapedim0clonercattorcharangeint32tensorstacktreshapeto index_fsarepeatintersect_denserrAget_tot_scores)r-r4r5r6r7rGnum_fsasnum_den_graphsnum_graphs_indexesden_graph_indexesnum_den_graphs_indexesnum_den_reordered_graphsrB num_den_latsnum_den_tot_scoresnum_tot_scoresden_tot_scores lat_slicer0r0r1r#ns>     z-MAPLoss._intersect_calc_scores_impl_exact_optc Cs|j}||jkr ||jksJ|jd}||ksJtj|||jj|r'dndd}tj|||jj|jj|jj |jj |r?dndd}|j ddd} |j ddd} |rY| | ||fS| | ddfS) zInner intersection method. Does exact intersection of dense_fsa_vec against num_graphs and pruned intersection against den_graph. Optiolally returns the numerator and the denominator lattices. rr>N)r?r@rArC)r?r@ search_beamrAmin_active_statesmax_active_statesrCTFrD) rGrHrIrrVrrAintersect_dense_prunedrcrdrerW) r-r4r5r6r7rGrXnum_latsden_latsr`rar0r0r1r"s0      z*MAPLoss._intersect_calc_scores_impl_prunedemissions_graphssupervision_graphs supervisionscCs|jdk}|||d|d|\}}}}t|dddfjtjd} d|_||| } t| |j\} } |r|dur?|dusAJ| |j j d|j j ddf} |j d}tt||j|j|jg|dd| dd}~tt||j|j|jg|dd| dd}~tj||d }~~t||j\}}| |@}|jd kr|j|| || |n|j|| }||fS| }|jd kr| | n| }||fS) acIntersects emissions_graphs with supervision_graphs and calculates lattice scores. This version implicitly assumes supervision_graphs to be a pair of the numerator and the denominator FSAs. It can also calculate accuracy between the numerator and the denominator lattices to use it as additional loss. Can be overridden. rrr<Nr9FT)indicesvaluessize min_col_index)r<r=none)rr$r rSrLlong_MAPLoss__batch_orderr rrIscoresrHrirow_idsr r index_selectr>phones get_arc_postexpsparsesumcoalesceabsto_dense)r-rirjrkboostedr`rargrhinverted_batch_order tot_scoresmmi_tot_scoresmmi_valid_maskrnrt num_sparse den_sparseacc_lossacc_tot_scoresacc_valid_mask valid_mask total_lossr0r0r1_intersect_calc_scoressR      zMAPLoss._intersect_calc_scoresr2r)T)__name__ __module__ __qualname____doc__rrintr(rrboolrfloatr r+rrLTensorr#r"rr __classcell__r0r0r.r1r0s    -  ? &rcsveZdZdZdddddedfdeded ed eed ed e d ee defde dede ffdd Z dddZ ZS) CtcMmiLossaMMI loss with custom CTC topologies. Available topologies: - `default`, with or without self-loops - `compact`, with or without self-loops - `shared_blank`, with or without self-loops - `minimal`, without self-loops cfg takes precedence over all optional parameters We keep explicit parameter setting to be able to create an instance without the need of a config. NrTFrrrrrrrrrrrrc s$tj||||||||| | d dS)N) rrrrrrrrrr)rr r,r.r0r1r s  zCtcMmiLoss.__init__r2cCs||_|j}t|drt|d|j}||jdkr+td|d|jddl m }||j|j |j |j |d|_dS)N aux_labelsr<z1lm_graph is not compatible with the num_classes: z, r)MmiGraphCompiler) aux_graph)r)rJhasattrdelattrlabelsmaxrr*unique-nemo.collections.asr.parts.k2.graph_compilersrrrrr%)r-r2r)rcompilerr0r0r1r+4s     zCtcMmiLoss.update_graphr)rrrrrrr(rrrrrr r+rr0r0r.r1rs>   r)abcrtypingrrrrrL omegaconfr%nemo.collections.asr.parts.k2.classesr)nemo.collections.asr.parts.k2.loss_mixinsr %nemo.collections.asr.parts.k2.ml_lossr #nemo.collections.asr.parts.k2.utilsr r r rnemo.core.utils.k2_guardr nemo.utilsrrrr0r0r0r1s       _