o i3@sdZddlZddlZddlZddlZddlZddlZddlmZmZm Z m Z ddl m Z ddl mZddlmZddZd d Zd d Zed krSeejdddSdS)a This program performs CTC segmentation to align utterances within audio files. Inputs: `--data-json`: A json containing list of utterances and audio files `--model`: An already trained ASR model Output: `--output`: A plain `segments` file with utterance positions in the audio files. Selected parameters: `--min-window-size`: Minimum window size considered for a single utterance. The current default value should be OK in most cases. Larger values might give better results; too large values cause IndexErrors. `--subsampling-factor`: If the encoder sub-samples its input, the number of frames at the CTC layer is reduced by this factor. `--frame-duration`: This is the non-overlapping duration of a single frame in milliseconds (the inverse of frames per millisecond). `--set-blank`: In the rare case that the blank token has not the index 0 in the character dictionary, this parameter sets the index of the blank token. `--gratis-blank`: Sets the transition cost for blank tokens to zero. Useful if there are longer unrelated segments between segments. `--replace-spaces-with-blanks`: Spaces are replaced with blanks. Helps to model pauses between words. May increase length of ground truth. May lead to misaligned segments when combined with the option `--gratis-blank`. N)CtcSegmentationParametersctc_segmentationdetermine_utterance_segments prepare_text)load_trained_model) ASRInterface)LoadInputsAndTargetscCstjdtjtjd}|jdddd|jdtdd d |jd d d dd|jdtddgdd|jdtddd |jddtddd |jdtddd |jdtdd|jd td!d|jd"tdd#d$|jd%tdd&d |jd'dtd(d)|jd*tdd+d |jd,tdd-d |jd.tdd/d |jd0tdd1d |jd2tdd3d |jd4tdd5d |jd6tdd7d |jd8tdd9d |jd:tdd;d |jdd$|S)?zGet default arguments.zXAlign text to audio using CTC segmentation.using a pre-trained speech recognition model.) descriptionconfig_file_parser_classformatter_classz--configTzDecoding config file path.)is_config_filehelpz--ngpurz$Number of GPUs (max. 1 is supported))typedefaultr z--dtype)float16float32float64rz,Float precision (only available in --api v2))choicesrr z --backendpytorchzBackend library)rrrr z --debugmode Debugmodez --verbosez-VzVerbose optionz--preprocess-confNz-The configuration file for the pre-processingz --data-jsonz+Json of recognition data for audio and text)rr z --utt-textzText separated into utterancesz--modelzModel file parameters to read)rrequiredr z --model-confzModel config filez --num-encsz Number of encoders in the model.)rrr z--subsampling-factorzSubsampling factor. If the encoder sub-samples its input, the number of frames at the CTC layer is reduced by this factor. For example, a BLSTMP with subsampling 1_2_2_1_1 has a subsampling factor of 4.z--frame-durationz;Non-overlapping duration of a single frame in milliseconds.z--min-window-sizez-Minimum window size considered for utterance.z--max-window-sizez-Maximum window size considered for utterance.z--use-dict-blankz DEPRECATED.z --set-blankz7Index of model dictionary for blank token (default: 0).z--gratis-blankzSet the transition cost of the blank token to zero. Audio sections labeled with blank tokens can then be skipped without penalty. Useful if there are unrelated audio segments between utterances.z--replace-spaces-with-blankszFill blanks in between words to better model pauses between words. Segments can be misaligned if this option is combined with --gratis-blank. May increase length of ground truth.z--scoring-lengthzFChanges partitioning length L for calculation of the confidence score.z--outputwzOutput segments file) configargparseArgumentParserYAMLConfigFileParserArgumentDefaultsHelpFormatteradd add_argumentintstrFileType)parserr#H/home/ubuntu/.local/lib/python3.10/site-packages/espnet/bin/asr_align.py get_parserCs r%cCs0t}||\}}|jdkrtjtjddn|jdkr&tjtjddn tjtjddtd|j dkrF|j dkrFt d|j d d }|j dkr_d }t j d }|d ur^tdn|j dkrntdtdtdt j ddtd|j|jdkrt||nt dtdd S)zRun the main decoding function.rz>%(asctime)s (%(module)s:%(lineno)d) %(levelname)s: %(message)s)levelformatzSkip DEBUG/INFO messagesrrz--dtype z" does not support the CPU backend.cpucudaCUDA_VISIBLE_DEVICESNz CUDA_VISIBLE_DEVICES is not set.zDecoding only supports ngpu=1.zpython path = PYTHONPATHz(None)z backend = rzOnly pytorch is supported.)r%parse_known_argsverboselogging basicConfigINFODEBUGWARNwarningngpudtype ValueErrorosenvirongeterrorsysexitinfobackend ctc_align)argsr"extradevicecvdr#r#r$mainsF           rEc Cstt|j\}}t|tsJtddd|jdur|jn|jddid}td|t|dr5|j j j }n t|d r@|j j j }nd }td |td |j j j d |vr^td|j|dt|jd}t|d}Wdn1s~wYt|jddde}|}d} i} i} |D]N} g} g}| t|kr|| | r| || || ddd||| d|| d| d7} | t|kr|| | s| | | <|| | <qWdn1swYt}d}d}|jdur|j}|jdur|j}t|dr||d|_n||_||_ |j!dur/|j!|_!|j"dur9|j"|_"|j#|_#|j$durHtd|j%durR|j%|_&|j'durc|j'r`d|_'nd|_'|j(rjd|_)|j)rw|j'rwt*d|j+dur|j+|_,td|d|t-|dD]\}} td| |t|| || fg}||\}}|d}t./"|0t.1||2d}|j 3|d45}Wdn 1swYt6|| | \}}t7|||\}}}t8d |t9||||| | }t-|D])\} }| | | d| d|dd!d|dd!d|d"d#d$ }|j:;|q qdS)%awESPnet-specific interface for CTC segmentation. Parses configuration, infers the CTC posterior probabilities, and then aligns start and end of utterances using CTC segmentation. Results are written to the output file given in the args. :param args: given configuration :param device: for inference; one of ['cuda', 'cpu'] :return: 0 on success asrTFNtrain)mode load_outputsort_in_input_lengthpreprocess_confpreprocess_argszDecoding device=encencoderUnknownzEncoder module: zCTC module: rnnz8No BLSTM model detected; memory consumption may be high.)rCrbuttsrzutf-8)encodingr r index_durationizNThe option --use-dict-blank is deprecated. If needed, use --set-blank instead.zBlanks are inserted between words, and also the transition cost of blank is zero. This configuration may lead to misalignments!zFrame timings: zms * z(%d/%d) Aligning z state_list = z.2fr(z.9f )hasattrrM __class__ __module__rNctcr4toevalopen data_jsonjsonloadutt_text readlineskeyslen startswithappendfindrsubsampling_factorframe_durationrWframe_duration_msmin_window_sizemax_window_size char_listuse_dict_blank set_blankblankreplace_spaces_with_blanks gratis_blankblank_transition_cost_zeror;scoring_lengthscore_min_mean_over_L enumeratetorchno_gradencode as_tensor unsqueeze log_softmaxr)numpyrrdebugroutputwrite) rArCrY train_argsload_inputs_and_targetsencoder_modulefjslinesitext segment_namesnametext_per_audiosegment_names_per_audioconfigrlrnidxbatchfeatlabel enc_outputlpzground_truth_matutt_begin_indicestimings char_probs state_listsegmentsboundary utt_segmentr#r#r$r@s        $               r@__main__r)__doc__rcr/r8r<rr{rrrr#espnet.asr.pytorch_backend.asr_initrespnet.nets.asr_interfacerespnet.utils.io_utilsrr%rEr@__name__argvr#r#r#r$s"$   x,}