o in* @sddlZddlZddlZddlZddlmZddlmZddlm Z ddl m Z m Z m Z mZmZmZmZmZmZddlZddlZddlmZddlmZddlmZmZmZdd lmZdd l m!Z!dd l"m#Z#m$Z$dd l%m&Z'd dl(m)Z)d dl*m+Z+d dl,m-Z-d dl.m/Z/m0Z0m1Z1m2Z2m3Z3m4Z4m5Z5m6Z6dZ7dZ8dZ9dZ:dZ;dZdZ?dZ@ejAZBejCZDejEe;edd ZGejEd"e@dd ZHe&jIe'd#e=dd e&IeGe&IeFe&IeHdSd%d&ZJe/jKfd'eeLd(eeLd$e eLe ffd)d*ZMdTd'eeLd,eNd$e eLe ffd-d.ZOd/e d$e fd0d1ZPe dUdd2ddd3d4eeeLe fd5eeLd6eLd7eeNd8eNf d9d:ZQd;eee eLfd$dfddVd?eLd$eeSeSffd@dAZTedWd/eLdCe+d+d$eeLfdDdEZUed/eLdCe+dd$eeSfdFdEZUdTd/eLdCeNd$eeeLeeSffdGdEZUdUdHeSd$dfdIdJZVdUdKe dLeeLd$ee fdMdNZWdXdOeeSeXfdPeSd$eLfdQdRZYdS)YN)InterpolationError)contextmanager)Path) TYPE_CHECKINGAnyDictIterableListOptionalTupleUnionoverload) NoSuchOption)split_arg_string)ConfigConfigValidationError require_gpu)gpu_is_available) get_command)Printermsg)app)about)Literal)validate)ENV_VARSSimpleFrozenDict import_fileis_compatible_versionlogger make_tempdirregistry run_commandz.tar.gzz-py3-none-any.whlz project.ymlzpython -m spacyspacyz=spaCy Command-line Interface DOCS: https://spacy.io/api/cli zCommand-line interface for spaCy projects and templates. You'd typically start by cloning a project template to a local directory and fetching its assets like datasets etc. See the project's z for the available commands. zSuite of helpful commands for debugging and profiling. Includes commands to check and validate your config files, training and evaluation data, and custom model implementations. z$Commands for benchmarking pipelines.z8Commands for initializing configs and pipeline packages.)namehelp benchmarkT)r%r&no_args_is_helpdebuginitprojectreturncCs tjtt}|tddS)N) prog_name)r"cliget_allrrCOMMAND)commandr2C/home/ubuntu/.local/lib/python3.10/site-packages/spacy/cli/_util.py setup_cliRs r4argsenv_varcsp|r tj|dnd}tt|t|dd}|r(fdd|D}td|r2tdti|S)aGenerate a dictionary of config overrides based on the extra arguments provided on the CLI, e.g. --training.batch_size to override "training.batch_size". Arguments without a "." are considered invalid, since the config only allows top-level sections to exist. env_vars (Optional[str]): Optional environment variable to read from. RETURNS (Dict[str, Any]): The parsed dict, keyed by nested config setting. T)is_clicsg|]}|vr|qSr2r2).0k env_overridesr2r3 isz*parse_config_overrides..zConfig overrides from CLI: %sz'Config overrides from env variables: %s)osenvironget_parse_overridesrr r)list)r5r6 env_string cli_overrideskeysr2r;r3parse_config_overridesZs    rFFr8cCsi}|rg|d}d|d}|dr[|}|dd}d|vr0|r&t|tj|ddd d |vrC|d d\}}|d d }n|rL|ddrOd }n|d}t|||<n tj|ddd |s|S)NrzInvalid config override ''z--r7.z#: can't override top-level sectionsexits=-_truez: name should start with --)pop startswithreplacerrfailsplit_parse_override)r5r8resultopterrorig_optvaluer2r2r3rAps*     rArZcCs(zt|WStyt|YSwN)srsly json_loads ValueErrorstr)rZr2r2r3rUs    rUr7)titledesc show_config hint_fill file_pathr`rarbrcc cs$zdVWdStyw}zd|dur|n|j}|jr)|s!|jn|jd|}|j|d||d}t|t|j|rgd|j vrg|durQt |dkrQ|nd}tjdd d tt d |d |d t dWYd}~dSd}~wty}ztjd|ddWYd}~dSd}~ww)aHelper to show custom config validation errors on the CLI. file_path (str / Path): Optional file path of config file, used in hints. title (str): Override title of custom formatted error. desc (str): Override description of custom formatted error. show_config (bool): Whether to output the config the error refers to. hint_fill (bool): Show hint about filling config. Nz r7)r`rarbzvalue_error.missingrMz config.cfgz|If your config contains missing values, you can run the 'init fill-config' command to fill in all the defaults, if possible:T)spacedz init fill-config  z rIzConfig validation errorrJ)rr`ra from_errorrrSprinttextstrip error_typesr_r0sysexitr)rdr`rarbrcerX config_pathr2r2r3show_validation_errors4  rp code_pathc Csv|dur9t|stjd|ddztd|WdSty8}ztjd||ddWYd}~dSd}~wwdS)zHelper to import Python file provided in training commands / commands using the config. This makes custom registered functions available. NzPath to Python code not foundrIrJ python_codezCouldn't load Python code: )rexistsrrSr Exception)rqrnr2r2r3 import_codes $ruNCould not run 'git'. Make sure it's installed and the executable is available.errorcCsjztddd}Wnt||j}|r|dsdS|ddd}t|d t|d fS) aGet the version of git and raise an error if calling 'git --version' fails. error (str): The error message to show. RETURNS (Tuple[int, int]): The version as a (major, minor) tuple. Returns (0, 0) if the version couldn't be determined. z git --versionT)capturez git version)rr NrHrrI)r# RuntimeErrorstdoutrjrQrTint)rwretr{versionr2r2r3get_git_versions r.intifycCdSr[r2rZrr2r2r3string_to_listrcCrr[r2rr2r2r3rrcCs|sgS|dr|dr|dd}g}|dD]5}|}|dr1|dr1|dd}|drA|drA|dd}|}|rKt|}||q|S)aSParse a comma-separated string to a list and account for various formatting options. Mostly used to handle CLI arguments that take a list of comma-separated values. value (str): The value to parse. intify (bool): Whether to convert values to ints. RETURNS (Union[List[str], List[int]]): A list of strings or ints. []rI,rG")rQendswithrTrjr|append)rZrrVpr2r2r3rs      use_gpucCsd|durt}nt|| d}|dkr!|d|t|dS|dtr0|ddSdS)zConfigure the GPU and log info.N)no_printprettyrz Using GPU: z Using CPUz.To switch to GPU 0, use the option: --gpu-id 0)rinforr)rsilent local_msgr2r2r3 setup_gpus  rpathsuffixcCs|s|gS|g}g}t}|D]7}t||vrq|t||jddr*q|r6||q|durC|jd|sCq| |q| |S)aGiven a directory and a suffix, recursively find all files matching the suffix. Directories or files with names beginning with a . are ignored, but hidden flags on filesystems are not checked. When provided with a suffix `None`, there is no suffix-based filtering.rrHN) is_dirsetr_addpartsrQextenditerdirrrsort)rrpathslocsseenr2r2r3walk_directorys$  rnumberndigitscCs"t|tr |d|dSt|S)zFormats a number (float or int) rounding to `ndigits`, without truncating trailing 0s, as happens with `round(number, ndigits)`rHf) isinstancefloatr_)rrr2r2r3_format_number0s r)r,N)Fr[)rv).)r)Zhashlibr>shutilrl configparserr contextlibrpathlibrtypingrrrrr r r r r r\typerclickrclick.shell_completionr thinc.apirrr thinc.utilr typer.mainrwasabirrweaselr project_clir7rcompatrschemasrutilrrrrr r!r"r# SDIST_SUFFIX WHEEL_SUFFIX PROJECT_FILEr0NAMEHELP PROJECT_HELP DEBUG_HELPBENCHMARK_HELP INIT_HELPArgumentArgOptionOptTyper benchmark_cli debug_cliinit_cli add_typerr4CONFIG_OVERRIDESr_rFboolrArUrprur|rrrrrrr2r2r2r3s   ,         (        $  +  " ( $