o }oi @sddlmZddlmZmZmZmZmZ   ddedede deeeeefeefd ef d d Z ded efd d Z ded efddZ ded efddZ dS))ArgumentParser)AnyDictListOptionalUnionadamN parent_parser optimizer default_lrdefault_opt_argsreturncCsX|durg}t|gddd}|jdt|dd|jdt|d d|jd |d td d |S)aExtends existing argparse with support for optimizers. # Example of adding optimizer args to command line : python train_script.py ... --optimizer "novograd" --lr 0.01 --opt_args betas=0.95,0.5 weight_decay=0.001 Args: parent_parser (ArgumentParser): Custom CLI parser that will be extended. optimizer (str): Default optimizer required. default_lr (float): Default learning rate that should be overriden during training. default_opt_args (list(str)): List of overriding arguments for the instantiated optimizer. Returns: ArgumentParser: Parser extended by Optimizers arguments. NTresolveparentsadd_helpconflict_handlerz --optimizerz(Name of the optimizer. Defaults to Adam.)typedefaulthelpz--lrzLearning rate of the optimizer.z --opt_args+zOverriding arguments for the optimizer. Must follow the pattern : name=value separated by spaces.Example: --opt_args weight_decay=0.001 eps=1e-8 betas=0.9,0.999)rnargsrr)r add_argumentstrfloat)r r r r parserrH/home/ubuntu/.local/lib/python3.10/site-packages/nemo/utils/arguments.pyadd_optimizer_argss rcCst|gddd}|jdtdddd|jdtddd d|jd tddd d|jd tddd d|jdtdddd|jdtdddd|S)zExtends existing argparse with default LR scheduler args. Args: parent_parser (ArgumentParser): Custom CLI parser that will be extended. Returns: ArgumentParser: Parser extended by LR Scheduler arguments. Frrz--warmup_stepsNzNumber of warmup stepsrrequiredrrz--warmup_ratioz>Number of warmup steps as a percentage of total training stepsz --hold_stepszNumber of hold LR stepsz --hold_ratioz?Number of hold LR steps as a percentage of total training stepsz--min_lrgzMinimum learning ratez --last_epochz1Last epoch id. -1 indicates training from scratch)rrintrr rrrradd_scheduler_args;s,  r$cCsNt|gddd}|jdtdddd|jd tdd d d|jd tdd d|S)zExtends existing argparse with default ASR collection args. Args: parent_parser (ArgumentParser): Custom CLI parser that will be extended. Returns: ArgumentParser: Parser extended by NeMo ASR Collection arguments. Frrz --asr_modelTzbad_quartznet15x5.yamlrz--train_datasetNztraining dataset pathz--eval_datasetzevaluation dataset pathrr rrrrr#rrr add_asr_args\s  r(cCst|gddd}|jdtddd|jdtddd d |jd d tdd d|jddtddgdd|jddtdd|jddddd|S)zExtends existing argparse with default NLP collection args. Args: parent_parser (ArgumentParser): Custom CLI parser that will be extended. Returns: ArgumentParser: Parser extended by NeMo NLP Collection arguments. Frrz --data_dirz4data directory to training or/and evaluation datasetr&z --config_fileNz$Huggingface model configuration filerz--pretrained_model_namezbert-base-uncasedzpretrained model name)rrr rz--tokenizer_namenemobert sentencepiecezTokenizer type)rrchoicesrz--tokenizer_modelz!Tokenizer file for sentence piece)rrrz--do_lower_case store_truezlower case data)actionr rr'r#rrr add_nlp_argsls    r.)rNN)argparsertypingrrrrrrrrr$r(r.rrrrs&  (!