o i!@sdZddlZddlmZddlmZmZmZmZddl m Z ddl m Z m Z mZmZmZdd lmZmZmZmZmZmZd Zd Zd Zd edededeZeeeeddddeddeddedddddedddddedddd dfd!ed"eed#ed$ed%ef d&d'Z d5dddd(d!ed"eed#ed$ed%ed)df d*d+Z!    d6d,ed-eeefd"eed#ed$ed%ed)efd.d/Z"d5d0eed"eed)dfd1d2Z#d!ed)dfd3d4Z$dS)7ztThis module contains helpers and subcommands for integrating Weasel with Data Version Control (DVC). https://dvc.orgN)Path)AnyDictListOptional)msg)get_hash join_commandload_project_config run_command working_dir)COMMANDNAME PROJECT_FILEArgOptappzdvc.yaml.dvcdvcz6# This file is auto-generated by Weasel based on your z. If you've # edited your z-, you can regenerate this file by running: #  zELocation of project directory. Defaults to current working directory.TF)helpexists file_okayzName of workflow defined in z(. Defaults to first workflow if not set.)r --verbosez-VzPrint more info--quietz-qzPrint less infoz--forcez-FzForce update DVC config project_dirworkflowverbosequietforcecCst|||||ddS)aAuto-generate Data Version Control (DVC) config. A DVC project can only define one pipeline, so you need to specify one workflow defined in the project.yml. If no workflow is specified, the first defined workflow is used. The DVC config will only be updated if the project.yml changed. DOCS: https://github.com/explosion/weasel/tree/main/docs/cli.md#repeat-dvc rr r!N)project_update_dvc)rrrr r!r$B/home/ubuntu/.local/lib/python3.10/site-packages/weasel/cli/dvc.pyproject_update_dvc_clisr&r"returncCsRt|}t||||||d}d}|rtdt|dStdtd|dS)aUpdate the auto-generated Data Version Control (DVC) config file. A DVC project can only define one pipeline, so you need to specify one workflow defined in the project.yml. Will only update the file if the checksum changed. project_dir (Path): The project directory. workflow (Optional[str]): Optional name of workflow defined in project.yml. If not set, the first workflow will be used. verbose (bool): Print more info. quiet (bool): Print less info. force (bool): Force update DVC config. r"z0To execute the workflow with DVC, run: dvc reprozUpdated DVC config from zNo changes found in z, no update neededN)r update_dvc_configrgoodrinfo)rrrr r!configupdatedhelp_msgr$r$r%r#*s r#pathr+c Cst||di}t|}t|||s|d}t|}|}|t} | rX| j ddd} | dd} Wdn1sGwY| |krT|sTd S| g} d d |d gD} g}|rn|d |ru|d||D]y}| |}|dg}|dg}|dg}|s|s|sqyddtdd|g}dddd|DD}dddd|DD}dddd|DD}dg|d|dt|d}|d r|d!g|||||}| t|qy| stjd"d#d$t|| D] }d%|}t|qWdn 1swY| j d&dd!} | }| dd| d|d'td'|Wdd(S1sKwYd(S))a~Re-run the DVC commands in dry mode and update dvc.yaml file in the project directory. The file is auto-generated based on the config. The first line of the auto-generated file specifies the hash of the config dict, so if any of the config values change, the DVC config is regenerated. path (Path): The path to the project directory. config (Dict[str, Any]): The loaded project.yml. verbose (bool): Whether to print additional info (via DVC). quiet (bool): Don't output anything (via DVC). force (bool): Force update, even if hashes match. RETURNS (bool): Whether the DVC config file was updated. workflowsrrutf8)encodingz# NFcSsi|]}|d|qS)namer$).0cmdr$r$r% msz%update_dvc_config..commandsrrdepsoutputsoutputs_no_cachepythonz-mprojectruncSg|] }|D]}|qqSr$r$r5clcr$r$r% z%update_dvc_config..cSg|]}d|gqS)z-dr$r5pr$r$r%rCcSr?r$r$r@r$r$r%rCrDcSrE)z-or$rFr$r$r%rCrHcSr?r$r$r@r$r$r%rCrDcSrE)z-Or$rFr$r$r%rCrHz-nz-wz --no-execno_skipz--always-changedzhNo usable commands for DVC found. This can happen if none of your commands have dependencies or outputs.rexitszdvc zr+ T) ensure_dvcgetlistkeyscheck_workflowsr resolve DVC_CONFIGropenreadlinestripreplaceunlinkappendrstrr rfailr r readseekwriteDVC_CONFIG_COMMENT)r.r+rrr r!r/workflow_names config_hashdvc_config_pathfref_hash dvc_commandsconfig_commandsflagsr4commandr9r:r; project_cmddeps_cmd outputs_cmdoutputs_nc_cmddvc_cmdfull_cmdrB dvc_commandcontentr$r$r%r(Hst           "      r(r/c Csx|s tjdtddd|dur(||vr(tjd|dtdd |dd|s:td td |d d dSdS)zValidate workflows provided in project.yml and check that a given workflow can be used to generate a DVC config. workflows (List[str]): Names of the available workflows. workflow (Optional[str]): The name of the workflow to convert. zNo workflows defined in zA. To generate a DVC config, define at least one list of commands.rrJNz Workflow 'z' not defined in z. Available workflows: z, zLNo workflow specified for DVC pipeline. Using the first workflow defined in z: 'r')rr[rjoinwarn)r/rr$r$r%rQs* rQcCs`z tjddgtjdWntytjddddYnw|ds.tjd d ddd Sd S) zyEnsure that the "dvc" command is available and that the current project directory is an initialized DVC project. rz --version)stdoutztTo use Weasel with DVC (Data Version Control), DVC needs to be installed and the 'dvc' command needs to be availablezYou can install the Python package from pip (pip install dvc) or conda (conda install -c conda-forge dvc). For more details, see the documentation: https://dvc.org/doc/installrrJrz(Project not initialized as a DVC projectzTo initialize a DVC project, you can run 'dvc init' in the project directory. For more details, see the documentation: https://dvc.org/doc/command-reference/initN) subprocessr>DEVNULL Exceptionrr[r)rr$r$r%rMs     rM)N)NFFF)%__doc__rupathlibrtypingrrrrwasabirutilr r r r r mainrrrrrrrSDVC_DIRUPDATE_COMMANDr_rhcwdrZboolr&r#r(rQrMr$r$r$r%s     !  V