o 6ti @s<dZddlZddlZddlZddlZddlZddlZddlmZddZe dkreZ ej e j dZee j dd  ZeeZWdn1sLwYd Zed Zd ZeeeD]Zed edjdZe ddZe d^ZZ dZ!e j"re j#rd$e dZ!n6e D]3Z%z e&e%dZ'Wne(yZ)z e*dee*e%e)dZ)[)wwe% ddZ+e!e+e'd7Z!qe!eZ,e j#re,dZ,ede j-deeede,dZ.e j/dedZ0e*dede0ee0ddd Z1ej2e.e1e3dddd Wdn 1swYqcdSdS)!z< Take in a YAML, and output all other splits with this YAML N)tqdmcCsVt}|jddd|jddd|jddd|jd dd|jd d d|S) Nz--base_yaml_pathT)requiredz--save_prefix_pathzeroshot)defaultz--cotFz --fewshotz --task_prefix)argparseArgumentParser add_argument parse_args)parserr W/home/ubuntu/.local/lib/python3.10/site-packages/lm_eval/tasks/bbh/_generate_configs.pyr sr __main__zutf-8)encodingzQ: {{input}} A:z(?<=answer is )(.*)(?=.)z lukaemon/bbhzNhttps://raw.githubusercontent.com/suzgunmirac/BIG-Bench-Hard/main/cot-prompts/z.txtz ----- z rtaskzLet's think step by step.z Let's think step by step. bbh__)includer dataset_name description doc_to_text/z.yamlzSaving yaml for subset z to winfT")width allow_unicode default_style)4__doc__rosredatasetsrequestsyamlrr __name__argspathsplitbase_yaml_pathbase_yaml_nameopenf full_load base_yamlbase_doc_to_textcompile answer_regex dataset_pathget_dataset_infoskeysrgetcontentdecoderesppromptrfew_shotprefix_doc_to_textfewshotcotjoinshotsearchanswer Exceptioneprintexampler task_prefix yaml_dictsave_prefix_pathfile_save_path yaml_filedumpfloatr r r r s