o ॵiE@sddlZddlmZddlmZmZmZddlmZm Z ddl Z ddl Z ddl m Z ddlmZddlmZde eeeffd d ZeGd d d ZeGd ddZeGdddZeddGdddeeeZddZdede eeedffddZdS)N)deepcopy) dataclassfieldfields)ListUnion)CliArgumentParser)Config)DEFAULT_DATASET_NAMESPACEvaluescCsZt|tr |dn|}i}|pgD]}t|dkrq|d\}}t|||<q|S)N,r=) isinstancestrsplitlenstrip parse_value)r pairs_paramskvkeyvaluerU/home/ubuntu/.local/lib/python3.10/site-packages/modelscope/trainers/training_args.pyset_flatten_values rc@seZdZUedddidZeed<edddidZeed<edddidZeed <eddd idZ eed <eddd idZ eed <edddidZ eed<ee ddidZ eed<ee ddidZeed<edddidZeed<dS) DatasetArgsNhelpzNThe dataset name used for training, can be an id in the datahub or a local dirdefaultmetadatatrain_dataset_namezOThe subset name used for evaluating, can be an id in the datahub or a local dirval_dataset_namez.The subset name used for training, can be Nonetrain_subset_namez0The subset name used for evaluating, can be Noneval_subset_namezThe split of train dataset train_splitzThe split of val dataset val_splitz'The dataset namespace used for trainingtrain_dataset_namespacez)The dataset namespace used for evaluatingval_dataset_namespaceaThe json file to parse all datasets from, used in a complex dataset scenario,the json format should be like: [ { "dataset": { # All args used in the MsDataset.load function "dataset_name": "xxx", ... }, # All columns used, mapping the column names in each dataset in same names. "column_mapping": { "text1": "sequence1", "text2": "sequence2", "label": "label", }, # float or str, float means to split the dataset into train/val, # or just str(train/val) "split": 0.8, } ] dataset_json_file)__name__ __module__ __qualname__rr!r__annotations__r"r#r$r%r&r r'r(r)rrrrrsb rc@sreZdZUedddddZeed<edddidZeed<eddd idZeed <edd d ddZ eed <dS) ModelArgsNzThe task code to be usedtaskrcfg_noderrzA model id or model dirmodelzthe revision of modelmodel_revisionzJThe mode type, if load_model_config is False, user need to fill this fieldz model.type model_type) r*r+r,rr/rr-r2r3r4rrrrr.js. r.c@seZdZUedddidZeed<edddd dZeed <ed d d d dZeed<edddd dZ e ed<edddd dZ e ed<edddd dZ eed<ed ddd dZ eed<edddd dZe ed<eddd d dZe ed!<ed"d#d$d dZeed%<ed&d'd(d dZeed)<ed*d+d,d dZeed-<ed.d/d0d dZeed1<ed2d3d4d dZeed5<ed6d7d8ed9dZeed:<ed6d;dd?d@gdAdBdZeedC<ed ddDidZeedE<ed"dFdGdHdZeedI<ed>dJdKgdAdLdZeedM<edNdOdPdHdZeedQ<ed6dRdSdHdZeedT<ed>dUdVgdAdLdZ eedW<edNdXdYdHdZ!eedZ<edd[d\dHdZ"e ed]<ed6d^d_dHdZ#eed`<edadbdcdHdZ$eedd<ed6dedfdHdZ%eedg<edNdhdidHdZ&eedj<eddkdldHdZ'e edm<ed6dndodHdZ(eedp<ed6dqdrdHdZ)eeds<edtdudvdHdZ*e edw<edxdydzdHdZ+eed{<eddkd|dHdZ,e ed}<ed6dnd~dHdZ-eed<ed6dqddHdZ.eed<edtduddHdZ/e ed<edxdyddHdZ0eed<d6S) TrainArgs*rzThe random seedrseedz#train.dataloader.batch_size_per_gpuz:The `batch_size_per_gpu` argument for the train dataloader)r1rper_device_train_batch_sizerz train.dataloader.workers_per_gpuz7The `workers_per_gpu` argument for the train dataloadertrain_data_workerFztrain.dataloader.shufflez/The `shuffle` argument for the train dataloader train_shuffleztrain.dataloader.drop_lastz1The `drop_last` argument for the train dataloadertrain_drop_lastz(evaluation.dataloader.batch_size_per_gpuz9The `batch_size_per_gpu` argument for the eval dataloaderper_device_eval_batch_sizez%evaluation.dataloader.workers_per_gpuz6The `workers_per_gpu` argument for the eval dataloadereval_data_workerzevaluation.dataloader.shufflez.The `shuffle` argument for the eval dataloader eval_shufflezevaluation.dataloader.drop_lastz0The `drop_last` argument for the eval dataloadereval_drop_lastztrain.max_epochszThe training epochs max_epochsz./train_targetztrain.work_dirz%The directory to save models and logswork_dirg-C6 ?ztrain.optimizer.lrz"The learning rate of the optimizerlrLinearLRztrain.lr_scheduler.typezThe lr_scheduler type in torch lr_schedulerAdamWztrain.optimizer.typez+The optimizer type in PyTorch, like `AdamW` optimizerNztrain.optimizerzThe optimizer params)r1r cfg_setteroptimizer_paramsztrain.lr_schedulerzThe lr scheduler paramslr_scheduler_paramsby_epochz&train.lr_scheduler.options.lr_strategyzThe lr decay strategy)rLby_stepno)r1rchoices lr_strategyzThe local rank local_rankz+The interval of iter of logging informationztrain.logging.intervalr0logging_intervalz5Eval strategy, can be `by_epoch` or `by_step` or `no`zevaluation.period.eval_strategy)rr1rO eval_strategyz Eval intervalzevaluation.period.interval eval_intervalzThe metric name for evaluationzevaluation.metrics eval_metricsz>Checkpointing strategy, can be `by_epoch` or `by_step` or `no`z%train.checkpoint.period.save_strategy save_strategyz9The interval of epoch or iter of saving checkpoint periodz train.checkpoint.period.interval save_intervalz;Save the checkpoint(if it's the best) after the evaluation.ztrain.checkpoint.best.save_bestsave_best_checkpointz%The metric used to measure the model.z train.checkpoint.best.metric_keymetric_for_best_modelmaxzDThe rule to measure the model with the metric, can be `max` or `min`ztrain.checkpoint.best.rulemetric_rule_for_best_modelzBThe max number of checkpoints to keep, older ones will be deleted.z*train.checkpoint.period.max_checkpoint_nummax_checkpoint_numzGThe max number of best checkpoints to keep, worse ones will be deleted.z(train.checkpoint.best.max_checkpoint_nummax_checkpoint_num_bestz$Push to hub after each checkpointingz#train.checkpoint.period.push_to_hub push_to_hubzr?r@rBrCrrDfloatrFrHrrJrKrPrQrRrSrUrVrWrXrYrZr\r]r^r_r`rarbrdrerfrgrhrirrrrr5s  r5F)initc@sJeZdZUedddidZeed<ddZdd d Zdd d Z d dZ dS) TrainingArgsFrzeUse the configuration of the model, default will only use the parameters in the CLI and the dataclassruse_model_configcKsDt||_t|D]}|j|vrt||j||jq i|_dSN)listkeys manual_argsrnamesetattr _unknown_args)selfkwargsfrrr__init__s    zTrainingArgs.__init__Nc Cst|}||\}}dd|D}i}tdt|dD]}t||d|||dd<qt|}|j|j7_|j |t | D]\}} |durZt ||rZt ||| qG|S) zcConstruct a TrainingArg class by the parameters of CLI. Returns: Self cSs g|] }|dvrd|vr|qS))\ z --local-rank=r).0itemrrr s z*TrainingArgs.parse_cli..rrT-N)rparse_known_argsrangerrreplacevarsrsrvupdateritemshasattrru) rw parser_argsparserargsunknown_unknowni args_dictrrrrr parse_clis "  zTrainingArgs.parse_clic Cst}t}|dur|j}t|D]B}|jd}|jdp#dd}|durK|j|jvs0|sJt |t r8|g}|D]}| ||t ||jiq:qt ||j||j<q| |j ||fS)zyConvert the TrainingArgs to the `Config` Returns: The Config, and extra parameters in dict. Nr1rIcS|Srprxrrrz(TrainingArgs.to_config..)r addictDictrorr getrtrsrrmerge_from_dictgetattrrv)rwignore_default_configcfgrryr1rI_noderrr to_configs&    zTrainingArgs.to_configcCs(t|}|D] }|j|kr|SqdSrp)rrt)rwr_fieldsryrrr get_metadatas  zTrainingArgs.get_metadatarp) r*r+r,rrorkr-rzrrrrrrrrns    rncsddlm}g}g}t|d}t|}|D]}|jdi|d}|j}|dfdd|D} ddlm } dd lm } dd lm } fd d|j D} i}| D]}t|d | rn| |d j||d<qY|d ||d<qY| |}|jd d| |d|d}|d}t|tr|dvsJ|dkr||q||qt|trd|krd ksJJ|j|d}||d||dqWdn1swYddlm}||||fS)aW The filename format: [ { "dataset": { "dataset_name": "xxx", ... }, "column_mapping": { "text1": "sequence1", "text2": "sequence2", "label": "label", } "usage": 0.8, } ] r) MsDatasetrdatasetcolumn_mappingcsg|]}|vr|qSrr)r}column keep_columnsrrrsz+build_dataset_from_file..)Features)Value) ClassLabelcsg|] }|dvr|qS)rr)r}ryrrrrsrTcSrrprrrrrr"rz)build_dataset_from_file..)remove_columnsfeaturesusage)trainvalr) train_sizetestN)concatenate_datasetsr) modelscoperopenjsonload to_hf_dataset column_namesrrdatasetsrrrrrrdtypemaprename_columnsrappendrltrain_test_splitr)filenamer train_seteval_setryds_jsondsr all_columnsrrrrr new_featuresrds_dictrrrrbuild_dataset_from_filesZ              &  'rrreturncCstdddddddd}||vr||Sd|vsd|vr$|ddddStd|r.t|Std|r8t|S|S) NTF)TruetrueFalsefalseNonenonenull"'rz^\d+$z4[+-]?(?=\d*[.eE])(?=\.?\d)\d*\.?\d*(?:[eE][+-]?\d+)?)rrematchrjrl)r const_maprrrr6s&  r)rcopyr dataclassesrrrtypingrrrr'modelscope.trainers.cli_argument_parserrmodelscope.utils.configr modelscope.utils.constantr rrrr.r5rnrrlrkrrrrrs,     O#L"A