o mit. @sdZddlZddlZddlmZddlmZddlZddlm Z zddl m Z Wne y9ZzededZ[wwzddlmZWne ySZzed edZ[wwz&ddlZe  d'd ed ejfd dZe  d'd ed ejfddZWn e yedYnwz,ddlZddlmZe  d'd ed ejfddZe  d'd ed ejfddZWn e yedYnwz&ddlmZe  d'd ed efddZe  d'd ed efddZWn e yedYnwGdddZe  d(d ed e e!e"ee#e$e%ffddZe d'd ed efddZe d'd efddZed efddZ&ed ed e e!e"ee#e$e%ffd dZ&e d'd ed efd!dZed'd efd"dZd#d$Z'  d)d%d&Z(dS)*aW&B Integration for Metaflow. This integration lets users apply decorators to Metaflow flows and steps to automatically log parameters and artifacts to W&B by type dispatch. - Decorating a step will enable or disable logging for certain types within that step - Decorating the flow is equivalent to decorating all steps with a default - Decorating a step after decorating the flow will overwrite the flow decoration Examples can be found at wandb/wandb/functional_tests/metaflow Nwraps)Path) telemetry)currentzrError: `metaflow` not installed >> This integration requires metaflow! To fix, please `pip install -Uqq metaflow`) typedispatchzrError: `fastcore` not installed >> This integration requires fastcore! To fix, please `pip install -Uqq fastcore`FnamedatacOH|r|rdSdS|r"||dtd|dt|ddSdSNdatasets:latestUsing artifact:  () use_artifactwandbtermlogtyperr r runtestingargskwargsra/home/ubuntu/SoloSpeech/.venv/lib/python3.10/site-packages/wandb/integration/metaflow/metaflow.py _wandb_use&   rc Os|r|rdSdS|rFtj|dd}||dd}|j|ddWdn1s,wY||td|d t|d dSdS) Nz pd.DataFramedatasetrz.parquetwbpyarrow)engineLogging artifact: rr)rArtifactnew_file to_parquet log_artifactrr) rr r rrrrartifactfrrr wandb_track7s   r+z[Warning: `pandas` not installed >> @wandb_log(datasets=True) may not auto log your dataset!cOr Nmodelsr rrrrrr r-rrrrrrrrTrc O|r|rdSdS|rEtj|dd}||dd}t||Wdn1s+wY||td|dt|ddSdS) Nz nn.Modulemodelr .pklr!r$rr)rr%r&torchsaver(rr rr r-rrrrr)r*rrrr+e   zXWarning: `pytorch` not installed >> @wandb_log(models=True) may not auto log your model!) BaseEstimatorcOr r,rr.rrrrrc Or/) Nr6r0r r1r!r$rrrr%r&pickledumpr(rrr4rrrr+r5zXWarning: `sklearn` not installed >> @wandb_log(models=True) may not auto log your model!c@s$eZdZddZddZddZdS) ArtifactProxyc s4|jiittfddtjDddS)Ncsi|]}|t|qSr)getattr).0pflowrr sz*ArtifactProxy.__init__..)r?inputsoutputsbaseparams)__dict__updatesetdirrparameter_names)selfr?rr>r__init__s zArtifactProxy.__init__cCs||j|<t|j||SN)rBsetattrr?)rJkeyvalrrr __setattr__s zArtifactProxy.__setattr__cCs2||jvr||jvrt|j||j|<t|j|SrL)rCrBr;r?rA)rJrNrrr __getattr__s zArtifactProxy.__getattr__N)__name__ __module__ __qualname__rKrPrQrrrrr:s r:cOs|rdS|||idS)Nscalar)log)rr rrrrrrrr+s cOsv|r|rdSdS|r9tj|dd}|r||n |r$||||td|dt|ddSdS)Nrrr r$rr) rr%is_diradd_diris_fileadd_filer(rr)rr r rrrrr)rrrr+s     c Or/) Ngenericotherr r1r!r$rrr7) rr othersrrrrr)r*rrrr+s   c OsLz t||g|Ri|WStjy%td|dt|dYdSw)NzThis artifact (z, z) does not exist in the wandb datastore!If you created an instance inline (e.g. sklearn.ensemble.RandomForestClassifier), then you can safely ignore thisOtherwise you may want to check your internet connection!)rr CommErrorprintrrr rrrrr wandb_uses racOsdSrLrr`rrrrascOr r rrrrrrs   cOr )Nr]r rrrr)rr r]rrrrrrrrs   cGstdd|DdS)Ncss|] }|dur|VqdSrLr)r<arrr szcoalesce..)next)argrrrcoalescesrfcs0t|fdd|durS|S)aAutomatically log parameters and artifacts to W&B by type dispatch. This decorator can be applied to a flow, step, or both. - Decorating a step will enable or disable logging for certain types within that step - Decorating the flow is equivalent to decorating all steps with a default - Decorating a step after decorating the flow will overwrite the flow decoration Args: func: (`Callable`). The method or class being decorated (if decorating a step or flow respectively). datasets: (`bool`). If `True`, log datasets. Datasets can be a `pd.DataFrame` or `pathlib.Path`. The default value is `False`, so datasets are not logged. models: (`bool`). If `True`, log models. Models can be a `nn.Module` or `sklearn.base.BaseEstimator`. The default value is `False`, so models are not logged. others: (`bool`). If `True`, log anything pickle-able. The default value is `False`, so files are not logged. settings: (`wandb.sdk.wandb_settings.Settings`). Custom settings passed to `wandb.init`. The default value is `None`, and is the same as passing `wandb.Settings()`. If `settings.run_group` is `None`, it will be set to `{flow_name}/{run_id}. If `settings.run_job_type` is `None`, it will be set to `{run_job_type}/{step_name}` cstr&}|jD]}tt||r#t|ds#t||t||q |Stdr-Stdfdd }|_d|_ |S)N _base_funcsettingsc sFt|tjjjs t}|jt|jtj dtj tjjj j d|jt|j tjtjjj j dtj|da}tj|d }d|j_Wdn1sNwYt|}|j|j|g|Ri||jD]\}}t|||dqn|jD]\}}t|||dqWddS1swYdS)N/) run_groupsource) run_job_typerlrh)rT)r r-r]r) isinstancersdkwandb_settingsSettingsrFrfrkr flow_namerun_idSourceINITrm step_nameinit wb_telemetrycontextfeaturemetaflowr:configrDrAitemsrarBr+) rJrirrrtelproxyrr )r funcr-r]rrwrapperFsN     "z-wandb_log..decorator..wrapper)r r-r]ri) inspectisclassrEcallabler;hasattrrMrrg_kwargs)rclsattrrr decoratorr-r]ri)rrr7s&    *zwandb_log..decoratorNr)rr r-r]rirrr wandb_log s Dr)FNF)NF)NFFFN))__doc__rr8 functoolsrpathlibrr wandb.sdk.librrxr{r ImportErrore Exception fastcore.allrpandaspdstr DataFramerr+r_r2torch.nnnnModule sklearn.baser6r:dictlistrGintfloatboolrarfrrrrrs(