o iC@sdZddlZddlZddlZddlZddlZddlZddlZddl m Z ddl m Z ddl mZd ddZd d Zed krHeejd ddSdS)z4End-to-end speech translation model training script.N) __version__) strtobool)BATCH_COUNT_CHOICESTcCs|durtjdtjtjd}|jdddd|jddd d|jd dd d|jd dtd d|jddgddd|jddtddgdd|jdt|dd|jddtdd|jd|d d!|jd"dtd#d|jd$td%d&|jd'd(d)d*d+d,|jd-d.td/d0d1|jd2d3d4td5d|jd6dtd*d7d8|jd9d:td;d|jdtdd?d1|jd@tddAd1|jdBtddCd1|jdDdEtgdFdGd|jdHdItdJd|jdKdItdLd|jdMdItdNd|jdOdItdPd|jdQdRdSdTdU|jdVdRdSdWdU|jdXddSdYdU|jdZtdd[d1|jd\td]d^d1|jd_dItd`d|jdadItdbd|jdcdItddd|jdetddfd1|jdgtddhd1|jdidItdjd|jdkdltdmd|jdndotdpd|jdqd4td*drd8|jdsdtt dud|jdvdwdxd4tdyd|jdzd4td{d|jd|d4td}d|jd~d4tdd|jdd4tdd|jdddtddd|jdddtddd|jdd4tdd|jdtdd*dd|jddtgddd|jddtdd|jddtdd|jddtdd|jddtdd|jddtdd|jddItdd|jddtddgdd|jddtdd|jdddtdd|jddtd*dd8|jddtd*dd8|jddtdd|jddtdd|jddtdd|jdt dRdd1|jddRt d*dd8|jddtd*dd8|jddddƄdd|jddtd*dd8|jddddƄdd|jddRt dd|jddRt dd|jdtddd1|jdt ddd1|jdt dd)d1|jdt dRd)d1|jdtddd1|jdtddd1|jdtdId)d1|jdtdd)d1|S)zGet default arguments.NzFTrain a speech translation (ST) model on one CPU, one or multiple GPUs) descriptionconfig_file_parser_classformatter_classz--configTzconfig file path)is_config_filehelpz --config2zCsecond config file path that overwrites the settings in `--config`.z --config3zRthird config file path that overwrites the settings in `--config` and `--config2`.z--ngpuz5Number of GPUs. If not given, use all visible devices)defaulttyper z --train-dtypefloat32)float16r float64O0O1O2O3zData type for training (only pytorch backend). O0,O1,.. flags require apex. See https://nvidia.github.io/apex/amp.html#opt-levels)r choicesr z --backendchainerpytorchzBackend library)r r rr z--outdirzOutput directory)r requiredr z --debugmode Debugmodez--dict Dictionary)rr z--seedz Random seedz --debugdirzOutput directory for debugging)r r z--resumez-r?z!Resume the training from snapshot)r nargsr z --minibatchesz-Nz-1z&Process only N minibatches (for debug))r r r z --verbosez-VrzVerbose optionz--tensorboard-dirzTensorboard log dir path)r r rr z--report-interval-itersdzReport interval iterationsz--save-interval-itersz!Save snapshot interval iterationsz --train-jsonz#Filename of train label data (json)z --valid-jsonz(Filename of validation label data (json)z--model-modulezBmodel defined module (default: espnet.nets.xxx_backend.e2e_st:E2E)z --ctc_typebuiltin)rgtnctccudnnctcz-Type of CTC implementation to calculate loss.z --mtlalphagzjMultitask learning coefficient, alpha: alpha*ctc_loss + (1-alpha)*att_lossz --asr-weightzMultitask learning coefficient for ASR task, weight: asr_weight*(alpha*ctc_loss + (1-alpha)*att_loss) + (1-asr_weight-mt_weight)*st_lossz --mt-weightzMultitask learning coefficient for MT task, weight: mt_weight*mt_loss + (1-mt_weight-asr_weight)*st_lossz --lsm-weightzLabel smoothing weightz --report-cerF store_truezCompute CER on development set)r actionr z --report-werzCompute WER on development setz --report-bleuzCompute BLEU on development setz--nbestzOutput N-best hypothesesz --beam-sizez Beam sizez --penaltyzIncertion penaltyz --maxlenratiozInput length ratio to obtain max output length. If maxlenratio=0.0 (default), it uses a end-detect function to automatically find maximum hypothesis lengthsz --minlenratioz.Input length ratio to obtain min output lengthz--rnnlmzRNNLM model file to readz --rnnlm-confzRNNLM model config file to readz --lm-weightz RNNLM weight.z --sym-spacezz Space symbolz --sym-blankz Blank symbolz --sortagradzFHow many epochs to use sortagrad for. 0 = deactivated, -1 = all epochsz --batch-countautozKHow to count batch_size. The default (auto) will find how to count by args.z --batch-sizez --batch-seqsz-bz*Maximum seqs in a minibatch (0 to disable)z --batch-binsz*Maximum bins in a minibatch (0 to disable)z--batch-frames-inz2Maximum input frames in a minibatch (0 to disable)z--batch-frames-outz3Maximum output frames in a minibatch (0 to disable)z--batch-frames-inoutz9Maximum input+output frames in a minibatch (0 to disable)z --maxlen-inz--batch-seq-maxlen-ini MLzPWhen --batch-count=seq, batch size is reduced if the input sequence length > ML.)r r metavarr z --maxlen-outz--batch-seq-maxlen-outzPWhen --batch-count=seq, batch size is reduced if the output sequence length > MLz--n-iter-processeszNumber of processes of iteratorz--preprocess-confz-The configuration file for the pre-processing)r r rr z--optadadelta)r)adamnoam Optimizerz --accum-gradzNumber of gradient accumurationz--epsg:0yE>zEpsilon constant for optimizerz --eps-decayg{Gz?zDecaying ratio of epsilonz--lrgMbP?zLearning rate for optimizerz --lr-decayg?zDecaying ratio of learning ratez--weight-decayzWeight decay ratioz --criterionacclossz"Criterion to perform epsilon decayz --thresholdg-C6?zThreshold to stop iterationz--epochsz-ezMaximum number of epochsz--early-stop-criterionzvalidation/main/accz=Value to monitor to trigger an early stopping of the trainingz --patiencezINumber of epochs to wait without improvement before stopping the trainingz --grad-clipzGradient norm threshold to clipz--num-save-attentionz*Number of samples of attention to be savedz--num-save-ctcz0Number of samples of CTC probability to be savedz --grad-noisezFThe flag to switch to use noise injection to gradients during trainingz--context-residualzHThe flag to switch to use context vector residual in the decoder networkz --enc-initz,Pre-trained ASR model to initialize encoder.z--enc-init-modszenc.enc.cfdddDS)Ncg|] }dkrt|qSrstr.0modsG/home/ubuntu/.local/lib/python3.10/site-packages/espnet/bin/st_train.py |0get_parser....,splitr:r<r:r=|zget_parser..zr?r@rArBr:r<r:r=rDrEz in the decoder with a target language ID (the first token in the target sequence)z --stats-filez,The stats file for the feature normalizationz--apply-uttmvnz2Apply utterance level mean variance normalization.z--uttmvn-norm-meansz--uttmvn-norm-varsz --fbank-fsi>z5The sample frequency used for the mel-fbank creation.z--n-melsPz!The number of mel-frequency bins.z --fbank-fminz --fbank-fmax) configargparseArgumentParserYAMLConfigFileParserArgumentDefaultsHelpFormatteradd add_argumentintr6floatrr)parserrr<r<r= get_parsers   rPc Cst}||\}}|jdkr|jdkrtd|jd|jdkr0|jdvr0td|jddd lm}|j d urCd |jd }n|j }||}| || |}||_ d |j vr_d|_d|j vrgd|_t |_ |jdkrxtjtjddn tjtjddtd|jd urtjd}|d urt|d}n.tdztjddgtjtjd} Wntjtfyd}Yn wt| jdd}||_n |jdkrt d|j}t!d|t!dtjddt!d |j"t#"|j"t$j#"|j"|j%d ur;t&|j%d! } | '} Wd n 1s wYd"d#| D} | (dd$| )d%| |_*nd |_*t!d&|j|jdkrXdd'l+m,} | |d Std())zRun the main training function.rr z/chainer backend does not support --train-dtype z.Use --dtype float32.r)rrrrr z--train-dtype z" does not support the CPU backend.)dynamic_importNz espnet.nets.z_backend.e2e_st:E2Echainer_backendpytorch_backendrz>%(asctime)s (%(module)s:%(lineno)d) %(levelname)s: %(message)s)levelformatzSkip DEBUG/INFO messagesCUDA_VISIBLE_DEVICESrAz CUDA_VISIBLE_DEVICES is not set.z nvidia-smiz-L)stdoutstderr rzsThere are some bugs with multi-GPU processing in PyTorch 1.2+ (see https://github.com/pytorch/pytorch/issues/21108)zngpu: zpython path = PYTHONPATHz(None)zrandom seed = %drbcSs g|] }|dddqS)zutf-8 r)decoderC)r8entryr<r<r=r>s zmain..r$zz backend = )trainzOnly pytorch are supported.)-rPparse_known_argsbackend train_dtypeNotImplementedErrorngpu ValueErrorespnet.utils.dynamic_importrQ model_module add_arguments parse_argsrversionverboselogging basicConfigINFOWARNwarningosenvirongetlenrC subprocessrunPIPECalledProcessErrorFileNotFoundErrorrXr]debuginfoseedrandomnpdictopen readlinesinsertappend char_listespnet.st.pytorch_backend.str_)cmd_argsrOargs_rQrg model_classcvdrdpf dictionaryrr_r<r<r=mains                       r__main__r)NT)__doc__rlrqr}rusysrGnumpyr~espnetrespnet.utils.cli_utilsrespnet.utils.training.batchfyrrPr__name__argvr<r<r<r=s&    $h