o 5tip @sNddlZddlZddlZddlmZddlmZddlmZGdddZ dS)N)List)Run)Validatec@s<eZdZdZddZdejfddZdejddfd d ZdS) HarnessCLIz-Main CLI parser that manages all subcommands.csntjddtdtjd_jjfdddjjdd d d _t jt jt jdS) Nzlm-evalz!Language Model Evaluation Harnessa% quick start: # Basic evaluation lm-eval run --model hf --model_args pretrained=gpt2 --tasks hellaswag # List available tasks lm-eval ls tasks # Validate task configurations lm-eval validate --tasks hellaswag,arc_easy legacy compatibility: The harness maintains backward compatibility with the original interface. If no command is specified, 'run' is automatically inserted: lm-eval --model hf --tasks hellaswag # Equivalent to 'lm-eval run --model hf --tasks hellaswag' For documentation, visit: https://github.com/EleutherAI/lm-evaluation-harness/blob/main/docs/interface.md )prog descriptionepilogformatter_classcs jS)N)_parser print_help)argsselfH/home/ubuntu/.local/lib/python3.10/site-packages/lm_eval/_cli/harness.py&s z%HarnessCLI.__init__..funccommandzAvailable commandsCOMMAND)desthelpmetavar) argparseArgumentParsertextwrapdedentRawDescriptionHelpFormatterr set_defaultsadd_subparsers _subparsersrcreaterrr rr r__init__ s  zHarnessCLI.__init__returncCslttjdkrtjd|jjvrtjddnttjdkr1dtjvr1|jjdtd|j S)z&Parse arguments using the main parser.runr) lensysargvr choicesinsertr exitr parse_argsr rrrr-.s   zHarnessCLI.parse_argsr NcCs||dS)zBMain execution method that handles subcommands and legacy support.Nr)rr rrrexecute:szHarnessCLI.execute) __name__ __module__ __qualname____doc__r"r Namespacer-r.rrrrr s ! r) rr(rlm_eval._cli.lsrlm_eval._cli.runrlm_eval._cli.validaterrrrrrs