o i?@sdZddlZddlZedZzddlmZWney4ddl Z e j de idddlmZYnwGdd d Z d dZ dd d Zd dZddZddZdS)aiCTC segmentation. This file contains the core functions of CTC segmentation. to extract utterance alignments within an audio file with a given transcription. For a description, see: "CTC-Segmentation of Large Corpora for German End-to-end Speech Recognition" https://arxiv.org/abs/2007.09127 or https://link.springer.com/chapter/10.1007%2F978-3-030-60276-5_27 Nctc_segmentation)cython_fill_table include_dirs) setup_argsc@seZdZdZdZdZdZdZdZdZ dZ dZ d Z d Z d Zd Zd Zd Zd ZdZdZdZdZeddZeddZddZddZddZddZdS)CtcSegmentationParametersuDefault values for CTC segmentation. May need adjustment according to localization or ASR settings. The character set is taken from the model dict, i.e., usually are generated with SentencePiece. An ASR model trained in the corresponding language and character set is needed. If the character set contains any punctuation characters, "#", the Greek char "ε", or the space placeholder, adapt these settings. _i@ig?·rFTuε#u.,»«•❍·u▁NcCs*|jr|jr|j|jd}|S|j}|S)aBDerive index duration from frame duration and subsampling. This value can be fixed by setting ctc_index_duration, which causes frame_duration_ms and subsampling_factor to be ignored. Legacy function. This function will be removed in later versions and replaced by index_duration. i)subsampling_factorframe_duration_msindex_duration)selftrU/home/ubuntu/.local/lib/python3.10/site-packages/ctc_segmentation/ctc_segmentation.pyindex_duration_in_secondsCs z3CtcSegmentationParameters.index_duration_in_secondscCs t|j}|dt|j7}|S)z`zSCtcSegmentationParameters.update_excluded_characters...) char_list)rrrrr]s zHCtcSegmentationParameters.update_excluded_characters..zExcluded characters: N)joinexcluded_charactersloggerdebugr!rr!rupdate_excluded_charactersZs  z4CtcSegmentationParameters.update_excluded_characterscKs|jdi|dS)z(Set all parameters as attribute at init.Nr)set)rkwargsrrr__init__esz"CtcSegmentationParameters.__init__cKs>|D]}|dst||r||durt||||qdS)zUpdate CtcSegmentationParameters. Args: **kwargs: Key-value dict that contains all properties with their new values. Unknown properties are ignored. _N) startswithhasattrsetattr)rr(keyrrrr'is zCtcSegmentationParameters.setcCs>d}|jD]}|j|}||d|d7}q|d7}|S)z"Print all attribute as dictionary.zCtcSegmentationParameters( =z, ))__dict__keys)routput attributevaluerrr__repr__xs  z"CtcSegmentationParameters.__repr__)__name__ __module__ __qualname____doc__max_prob skip_probmin_window_sizemax_window_sizerscore_min_mean_over_Lspaceblankreplace_spaces_with_blanksrrbacktrack_from_max_tself_transitionstart_of_ground_truthr#tokenized_meta_symbolr r r propertyrrr&r)r'r6rrrrr#s:     rc Cs|j}d}|jd|j}tdt|d|dd|jddt||jdkr5|j|jkr5td|j } t j t ||jdt|gt j d }||jt j t|gt jd }t||t j t j|t jd ||j|j\} } |jr}|jdd } td |d d | fd | t t|g} t |jdg} dg|jd} z| dks| dkrd }t j}|j}t|jd D]^}|| |fdkr!|| | |dkr|| d |nd}| dkr|| || || |ffn|j}|| | f|| d || d |f}t|||krt||}|}t||}q| dkr4t|| || |f|n|j}|| | f|| d | f}t|||kr| dkrtd|d D]}|| | |j| | |<qZ|| || | <|| |f}|j|| || | <| d |8} | d |8} n|| || | <|j| || | <| d 8} | dks| dksWn-tytd|d9}||jkrtdt|Yq8t dt dw | | | fS)zExtract character-level utterance alignments. :param config: an instance of CtcSegmentationParameters :param lpz: probabilities obtained from CTC output :param ground_truth: ground truth text in the form of a label sequence :return: rzCTC segmentation of z chars to z.2fz s audio (z indices).zAudio is shorter than text!Tdtyperz/Max. joint probability to align text to audio: Nz at time index rzPIndexError: Backtracking was not successful, the window size might be too small.rzIncreasing the window size to: zMaximum window size reached.zCheck data and character list!)!rAshaperr$infolenr<r;AssertionErrorr=npzerosminfloat32fillint64rastypearrayrrCr%maxinfrangeabsr rD IndexErrorwarningr>strerror)configlpz ground_truthrAoffsetaudio_duration window_sizetableoffsetsrctimings char_probs state_listmin_smin_switch_prob_delta max_lpz_probs switch_probest_switch_prob stay_prob est_stay_prob char_indexrrrrs   ((       +    cCst|jtkr d|_|dur||_|j|j}|j}g}|D]<}||js+||j7}|t|d|D]#}| rK|j rK||jsJ||j7}q6||jvrY||j vrY||7}q6q||jsf||j7}t d||t|dtdd|jD}tt||gtjd} tt|D]2} t|D]+} | | dkrq|| | | d} | |j|} | |jvr|j| } | | | | f<qq| |fS)aPrepare the given text for CTC segmentation. Creates a matrix of character symbols to represent the given text, then creates list of char indices depending on the models char list. :param config: an instance of CtcSegmentationParameters :param text: iterable of utterance transcriptions :param char_list: a set or list that includes all characters/symbols, characters not included in this list are ignored :return: label matrix, character index matrix rNrground_truth: cSsg|]}t|qSr)rM)rrgrrrrrz prepare_text..rJ)typerAr]r rEendswithr@appendrMisspacerBr#r$r%rWrOonesrTrYreplaceindex)r_textr rArautt_begin_indicesuttr max_char_lenground_truth_matirnspanrsrrr prepare_textsL             rc Cs2|jg}g}|D]8}|d|jks||jg7}|t|d|D]}||jvr?|jr:||js:||jg7}||g7}q$q|d|jksN||jg7}t d||t|dd}t t||gt j d}tdt|D]}|||jkr|j||df<qu|j||} | ||df<qu||fS)zPrepare the given tokenized text for CTC segmentation. :param config: an instance of CtcSegmentationParameters :param text: string with tokens separated by spaces :return: label matrix, character index matrix rJrrtr)rEr@rwrMsplitr rB beginswithrFr$r%rOryrTrYrAr{) r_r|rar}r~tokenrrrrsrrrprepare_tokenized_text$s6       rcCsdg}g}|D]}|d|jks||jg7}|t|d||7}q|d|jks3||jg7}td||t|dtj|tjd dd}||fS)aNPrepare the given token list for CTC segmentation. This function expects the text input in form of a list of numpy arrays: [np.array([2, 5]), np.array([7, 9])] :param config: an instance of CtcSegmentationParameters :param text: list of numpy arrays with tokens :return: label matrix, character index matrix rJrrtrH) rArwrMtolistr$r%rOrVrTreshape)r_r|rar}r~rrrrprepare_token_listLs   rcsfdd}g}td}tt|D]a}|||d} |||dd} tt| |j} tt| |j} |j} | | krA|}n+| | | krP|| | }ntd}t| | | D]}t ||||| }q\| | | |fq|S)aUtterance-wise alignments from char-wise alignments. :param config: an instance of CtcSegmentationParameters :param utt_begin_indices: list of time indices of utterance start :param char_probs: character positioned probabilities obtained from backtracking :param timings: mapping of time indices to seconds :param text: list of utterances :return: segments, a list of: utterance start and end [s], and its confidence score csX||dd}|dkrt|dd|S|dkr*t|dd|SdS)zCompute start and end time of utterance. :param index: frame index value :param align_type: one of ["begin", "end"] :return: start/end time of utterance in seconds rrbeging?endN)rWrQ)r{ align_typemiddlerhrr compute_timews z2determine_utterance_segments..compute_timerrrrg) rOfloat64rYrMrroundrr?meanrQrw)r_r}rirhr|rsegmentsmin_probrstartrstart_tend_tnmin_avgrrrrdetermine_utterance_segmentsls$    r)N)r:loggingnumpyrO getLoggerr$ctc_segmentation_dynr ImportError pyximportinstall get_includerrrrrrrrrrs"   _ m5(