o wiK @sddlZddlZddlZddlmZmZmZddlZ ddZ dde de deee efd e fd d Z e d kroe d ZejddddejddddejdejddejddddeZe ejejejejdSdS)N)AnyDictOptionalcCs6t|jdkrtd|j|jd}tt|}|S)z(Get HuggingFace model class from config.z:More than one model architecture available, choosing 1st: r)len architecturesprintgetattr transformers) hf_config model_name model_classr_/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/tests/setup/models/create_hf_model.pyget_hf_model_class,s   rFmodel_name_or_path output_dirconfig_updates overwritecCstj|r|std|ddStj|}tj|}t|}|dur+| |t|||}t|tj |ddtd|d| || |tddS) z6Create HuggingFace model with optional config updates.zOutput directory z1 exists and overwrite flag is not set so exiting.NT)exist_okzSaving model to z...zOK.) ospathisdirrr AutoConfigfrom_pretrained AutoTokenizerrupdatemakedirssave_pretrained)rrrrr tokenizerr modelrrrcreate_hf_model5s       r!__main__zHCreate a HuggingFace model (random initialization) for testing purposes.z--model_name_or_pathTz8Model name or local path with model config and tokenizer)requiredhelpz --output_dirzOutput directoryz--config_updatesz>Parameter updates in JSON format to overwrite for model config)typer$z --overwrite store_truezOverwrite file if it exists)actionr$)NF)argparsejsonrtypingrrrr rstrboolr!__name__ArgumentParserparser add_argumentloads parse_argsargsrrrrrrrrsT