o i3@sddlZddlmZddlmZmZmZmZmZm Z ddl Z ddl Z ddl m Z ddlmZddlmZmZmZddlmZmZmZmZmZdd lmZmZmZmZd d lmZmZm Z m!Z!m"Z"m#Z#d d lm$Z$e$j%d dddde!ddede!e&dddde"ddddde"ddddde"ddddfde j'de(d ed!e)d"e)d#e)f d$d%Z*edddded&d ede(d'ee(efd!e)d"e)d(e)d)e)d*e(d+dfd,d-Z+defd edee(d*e(d+dfd.d/Z,edddfd0ee(d1e)d"e)d(e)d+df d2d3Z-d0e e(d4e e(de(d+dfd5d6Z.d ed7ee(efd+e)fd8d9Z/d ed7ee(efd+dfd:d;Z0ed<d ed7ee(efd*e(d+ee(effd=d>Z1d ed?ee(d+eee(ee(ffd@dAZ2dS)BN)Path)AnyDictIterableListOptionalSequence)msg) locale_escape)SimpleFrozenDictSimpleFrozenListcheck_spacy_env_vars) get_checksumget_hashis_cwd join_commandload_project_config)parse_config_overrides run_command split_command working_dir)COMMAND PROJECT_FILE PROJECT_LOCKArgOpt_get_parent_command)apprunT)allow_extra_argsignore_unknown_options)context_settingszName of command defined in the )helpzELocation of project directory. Defaults to current working directory.F)r$exists file_okayz--forcez-Fz/Force re-running steps, even if nothing changedz--dryz-Dz+Perform a dry run and don't execute scriptsz--helpz+Show help message and available subcommandsctx subcommand project_dirforcedry show_helpcCsBt|}|s|st|||dSt|j}t||||||ddS)aLRun a named command or workflow defined in the project.yml. If a workflow name is specified, all commands in the workflow are run, in order. If commands define dependencies and/or outputs, they will only be re-run if state has changed. DOCS: https://github.com/explosion/weasel/tree/main/docs/cli.md#rocket-run ) overridesr*r+parent_commandN)rprint_run_helprargs project_run)r'r(r)r*r+r,r.r-r2B/home/ubuntu/.local/lib/python3.10/site-packages/weasel/cli/run.pyproject_run_clis  r4)r-r*r+captureskip_requirements_checkr.r-r5r6r.returnc Cst||d}dd|dgD} |di} tt| t| ||| vrEtd|d| |D] } t|| ||||dq6dS| |} | d gD] } || sod |d | } d }|sed nd}tj | ||dqOt t |C}t |t || }|s|std| ddn t| d||d|st|| WddSWddSWddS1swYdS)aRun a named script defined in the project.yml. If the script is part of the default pipeline (defined in the "run" section), DVC is used to execute the command, so it can determine whether to rerun it. It then calls into "exec" to execute it. project_dir (Path): Path to project directory. subcommand (str): Name of command to run. overrides (Dict[str, Any]): Optional config overrides. force (bool): Force re-running, even if nothing changed. dry (bool): Perform a dry run and don't execute commands. capture (bool): Whether to capture the output and errors of individual commands. If False, the stdout and stderr will not be redirected, and if there's an error, sys.exit will be called with the return code. You should use capture=False when you want to turn over execution to the command, and capture=True when you want to run the command more like a function. skip_requirements_check (bool): No longer used, deprecated. )r-cSi|]}|d|qSnamer2.0cmdr2r2r3 Qzproject_run..commands workflowszRunning workflow '')r-r*r+r5depsz)Missing dependency specified by command 'z': zGMaybe you forgot to run the 'weasel assets' command or a previous step?rNexitsz Skipping 'r:z': nothing changedscript)r+r5)rgetvalidate_subcommandlistkeysr infor1r%failrrdivider check_rerun run_commandsupdate_lockfile)r)r(r-r*r+r5r6r.configr@rAr=deperrerr_help err_exits current_dirrerunr2r2r3r14sL          "r1c st|}|dg}dd|D|di}t|rdn|}|rttt||td|d|d||vrU|d }|rStd |d dSdS||vr||}td t|d fd dt|D} t | |d|d} td| dSdStd|d} | rtt | d |rtdt td|d|t dd|D|rtdt td|d|t dd| DdSdS)aJSimulate a CLI help prompt using the info available in the project.yml. project_dir (Path): The project directory. subcommand (Optional[str]): The subcommand or None. If a subcommand is provided, the subcommand help is shown. Otherwise, the top-level help and a list of available commands is printed. r@cSr8r9r2r;r2r2r3r>r?z"print_run_help..rAzUsage:  run  r$ z Workflow consisting of z commands:cs2g|]\}}|dd||ddfqS)r. r$rXrG)r<istepr@r2r3 s z"print_run_help..z run [COMMAND] z --helpzFor command details, run: titlezAvailable commands in cSs g|] }|d|ddfqS)r:r$rXr]r;r2r2r3ras zAvailable workflows in z run [WORKFLOW] cSsg|] \}}|d|fqS)z -> )join)r<r:stepsr2r2r3rasN)rrGrrHrIrJprintlen enumerater tabler ritems) r)r(r.rQconfig_commandsrA project_loc help_textrd steps_datahelp_cmdrbr2r`r3r/tsH     r/r@silentcCs|D]?}t|}t|r|ddvrtj|d<nt|r.|ddvr.tjddg|dd}|s9tdt||sAt||d qdS) abRun a sequence of commands in a subprocess, in order. commands (List[str]): The string commands. silent (bool): Don't print the commands. dry (bool): Perform a dry run and don't execut anything. capture (bool): Whether to capture the output and errors of individual commands. If False, the stdout and stderr will not be redirected, and if there's an error, sys.exit will be called with the return code. You should use capture=False when you want to turn over execution to the command, and capture=True when you want to run the command more like a function. r)pythonpython3)pippip3z-mrrrNzRunning command: )r5)rrfsys executablererr)r@ror+r5ccommandr2r2r3rOs  rOrAcCs|s|stjdtdd||vrN||vrPg}|dvr!|d|r.|dd||r;|dd|tjd |d td |ddd Sd Sd S) zCheck that a subcommand is valid and defined. Raises an error otherwise. commands (Sequence[str]): The available commands. subcommand (str): The subcommand. z$No commands or workflows defined in rrD)assetsassetz-Did you mean to run: python -m weasel assets?zAvailable commands: z, zAvailable workflows: z Can't find command or workflow 'z' in r\N)r rLrappendrc)r@rAr(help_msgr2r2r3rHs   rHrwcCst|ddrdS|t}|sdSt|}|d|vrdS||d}|dgs-dSt||}t|t|kS)a+Check if a command should be rerun because its settings or inputs/outputs changed. project_dir (Path): The current project directory. command (Dict[str, Any]): The command, as defined in the project.yml. strict_version (bool): RETURNS (bool): Whether to re-run the command. no_skipFTr:outs)rGrr%srsly read_yamlget_lock_entryr)r)rw lock_pathdataentry lock_entryr2r2r3rNs      rNcCsN|t}|st|ii}nt|}t||||d<t||dS)a>Update the lockfile after running a command. Will create a lockfile if it doesn't yet exist and will add an entry for the current command, its script and dependencies/outputs. project_dir (Path): The current project directory. command (Dict[str, Any]): The command, as defined in the project.yml. r:N)rr%r~ write_yamlrr)r)rwrrr2r2r3rPs  rP)r.cCs^t||dg}t||dg}t||dg}|d|d|d|g||dS)aGet a lockfile entry for a given command. An entry includes the command, the script (command steps) and a list of dependencies and outputs with their paths and file hashes, if available. The format is based on the dvc.lock files, to keep things consistent. project_dir (Path): The current project directory. command (Dict[str, Any]): The command, as defined in the project.yml. RETURNS (Dict[str, Any]): The lockfile entry. rCoutputsoutputs_no_cacherYr:rF)r=rFrCr}) get_fileinforG)r)rwr.rCr}outs_ncr2r2r3rs  rpathscCs>g}|D]}||}|rt|nd}|||dq|S)a%Generate the file information for a list of paths (dependencies, outputs). Includes the file path and the file's checksum. project_dir (Path): The current project directory. paths (List[str]): The file paths. RETURNS (List[Dict[str, str]]): The lockfile entry for a file. N)pathmd5)r%rrz)r)rrr file_pathrr2r2r3r)s r)3rtpathlibrtypingrrrrrrr~typerwasabir wasabi.utilr utilr r rrrrrrrrrrmainrrrrrrrrwcwdContextstrboolr4r1r/rOrHrNrPrrr2r2r2r3s         $   A 0 $       .